Transparency and reproducibility in artificial intelligence

Benjamin Haibe-Kains1,2,3,4,5 ✉, George Alexandru Adam, Ahmed Hosny, Farnoosh Khodakarami, Massive Analysis Quality Control (MAQC) Society Board of Directors*, Levi Waldron, Bo Wang, Chris McIntosh, Anna Goldenberg, Anshul Kundaje, Casey S. Greene, Tamara Broderick, Michael M. Hoffman, Jeffrey T. Leek, Keegan Korthauer, Wolfgang Huber, Alvis Brazma, Joelle Pineau, Robert Tibshirani, Trevor Hastie, John P. A. Ioannidis, John Quackenbush & Hugo J. W. L. Aerts

[1]  B. Carlson Putting oncology patients at risk. , 2012, Biotechnology healthcare.

[2]  Yolanda Gil,et al.  Enhancing reproducibility for computational methods , 2016, Science.

[3]  Daniel L Rubin,et al.  A curated mammography data set for use in computer-aided detection and diagnosis research , 2017, Scientific Data.

[4]  J. Ioannidis,et al.  Reproducible research practices, transparency, and open access data in the biomedical literature, 2015–2017 , 2018, PLoS biology.

[5]  Odd Erik Gundersen,et al.  State of the Art: Reproducibility in Artificial Intelligence , 2018, AAAI.

[6]  M. Hutson Artificial intelligence faces reproducibility crisis. , 2018, Science.

[7]  Matt Crane,et al.  Questionable Answers in Question Answering Research: Reproducibility and Variability of Published Results , 2018, TACL.

[8]  David W. Aha,et al.  On Reproducible AI: Towards Reproducible Research, Open Science, and Digital Scholarship in AI Publications , 2018, AI Mag..

[9]  Taghi M. Khoshgoftaar,et al.  A survey on Image Data Augmentation for Deep Learning , 2019, Journal of Big Data.

[10]  Rick L. Stevens,et al.  Toward unrestricted use of public genomic data , 2019, Science.

[11]  Kathryn J Fowler,et al.  Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers-From the Radiology Editorial Board. , 2019, Radiology.

[12]  C. Loewe Hemodynamically Significant Coronary Stenosis: Detection with CT Myocardial Perfusion Imaging versus Machine Learning Coronary CT Fractional Flow Reserve. , 2019, Radiology.

[13]  John P. A. Ioannidis,et al.  Exploration, Inference, and Prediction in Neuroscience and Biomedicine , 2019, Trends in Neurosciences.

[14]  David S. Melnick,et al.  International evaluation of an AI system for breast cancer screening , 2020, Nature.