Jennifer J Ware1, Marcus R Munafò. 1. School of Social and Community Medicine, University of Bristol, Bristol, UK; MRC Integrative Epidemiology Unit (IEU) at the University of Bristol, Bristol, UK; UK Centre for Tobacco and Alcohol Studies, University of Bristol, Bristol, UK.
Abstract
BACKGROUND AND AIMS: The low reproducibility of findings within the scientific literature is a growing concern. This may be due to many findings being false positives which, in turn, can misdirect research effort and waste money. METHODS: We review factors that may contribute to poor study reproducibility and an excess of 'significant' findings within the published literature. Specifically, we consider the influence of current incentive structures and the impact of these on research practices. RESULTS: The prevalence of false positives within the literature may be attributable to a number of questionable research practices, ranging from the relatively innocent and minor (e.g. unplanned post-hoc tests) to the calculated and serious (e.g. fabrication of data). These practices may be driven by current incentive structures (e.g. pressure to publish), alongside the preferential emphasis placed by journals on novelty over veracity. There are a number of potential solutions to poor reproducibility, such as new publishing formats that emphasize the research question and study design, rather than the results obtained. This has the potential to minimize significance chasing and non-publication of null findings. CONCLUSIONS: Significance chasing, questionable research practices and poor study reproducibility are the unfortunate consequence of a 'publish or perish' culture and a preference among journals for novel findings. It is likely that top-down change implemented by those with the ability to modify current incentive structure (e.g. funders and journals) will be required to address problems of poor reproducibility.
BACKGROUND AND AIMS: The low reproducibility of findings within the scientific literature is a growing concern. This may be due to many findings being false positives which, in turn, can misdirect research effort and waste money. METHODS: We review factors that may contribute to poor study reproducibility and an excess of 'significant' findings within the published literature. Specifically, we consider the influence of current incentive structures and the impact of these on research practices. RESULTS: The prevalence of false positives within the literature may be attributable to a number of questionable research practices, ranging from the relatively innocent and minor (e.g. unplanned post-hoc tests) to the calculated and serious (e.g. fabrication of data). These practices may be driven by current incentive structures (e.g. pressure to publish), alongside the preferential emphasis placed by journals on novelty over veracity. There are a number of potential solutions to poor reproducibility, such as new publishing formats that emphasize the research question and study design, rather than the results obtained. This has the potential to minimize significance chasing and non-publication of null findings. CONCLUSIONS: Significance chasing, questionable research practices and poor study reproducibility are the unfortunate consequence of a 'publish or perish' culture and a preference among journals for novel findings. It is likely that top-down change implemented by those with the ability to modify current incentive structure (e.g. funders and journals) will be required to address problems of poor reproducibility.
Authors: Paul M Thompson; Ole A Andreassen; Alejandro Arias-Vasquez; Carrie E Bearden; Premika S Boedhoe; Rachel M Brouwer; Randy L Buckner; Jan K Buitelaar; Kazima B Bulayeva; Dara M Cannon; Ronald A Cohen; Patricia J Conrod; Anders M Dale; Ian J Deary; Emily L Dennis; Marcel A de Reus; Sylvane Desrivieres; Danai Dima; Gary Donohoe; Simon E Fisher; Jean-Paul Fouche; Clyde Francks; Sophia Frangou; Barbara Franke; Habib Ganjgahi; Hugh Garavan; David C Glahn; Hans J Grabe; Tulio Guadalupe; Boris A Gutman; Ryota Hashimoto; Derrek P Hibar; Dominic Holland; Martine Hoogman; Hilleke E Hulshoff Pol; Norbert Hosten; Neda Jahanshad; Sinead Kelly; Peter Kochunov; William S Kremen; Phil H Lee; Scott Mackey; Nicholas G Martin; Bernard Mazoyer; Colm McDonald; Sarah E Medland; Rajendra A Morey; Thomas E Nichols; Tomas Paus; Zdenka Pausova; Lianne Schmaal; Gunter Schumann; Li Shen; Sanjay M Sisodiya; Dirk J A Smit; Jordan W Smoller; Dan J Stein; Jason L Stein; Roberto Toro; Jessica A Turner; Martijn P van den Heuvel; Odile L van den Heuvel; Theo G M van Erp; Daan van Rooij; Dick J Veltman; Henrik Walter; Yalin Wang; Joanna M Wardlaw; Christopher D Whelan; Margaret J Wright; Jieping Ye Journal: Neuroimage Date: 2015-12-04 Impact factor: 6.556