Martin A Lindquist1, Amanda Mejia. 1. From the Department of Biostatistics, Johns Hopkins University, Baltimore, Maryland.
Abstract
OBJECTIVE: The need for appropriate multiple comparisons correction when performing statistical inference is not a new problem. However, it has come to the forefront in many new modern data-intensive disciplines. For example, researchers in areas such as imaging and genetics are routinely required to simultaneously perform thousands of statistical tests. Ignoring this multiplicity can cause severe problems with false positives, thereby introducing nonreproducible results into the literature. METHODS: This article serves as an introduction to hypothesis testing and multiple comparisons for practical research applications, with a particular focus on its use in the analysis of functional magnetic resonance imaging data. RESULTS: We discuss hypothesis testing and a variety of principled techniques for correcting for multiple tests. We also illustrate potential pitfalls problems that can occur if the multiple comparisons issue is not dealt with properly. We conclude, by discussing effect size estimation, an issue often linked with the multiple comparisons problem. CONCLUSIONS: Failure to properly account for multiple comparisons will ultimately lead to heightened risks for false positives and erroneous conclusions.
OBJECTIVE: The need for appropriate multiple comparisons correction when performing statistical inference is not a new problem. However, it has come to the forefront in many new modern data-intensive disciplines. For example, researchers in areas such as imaging and genetics are routinely required to simultaneously perform thousands of statistical tests. Ignoring this multiplicity can cause severe problems with false positives, thereby introducing nonreproducible results into the literature. METHODS: This article serves as an introduction to hypothesis testing and multiple comparisons for practical research applications, with a particular focus on its use in the analysis of functional magnetic resonance imaging data. RESULTS: We discuss hypothesis testing and a variety of principled techniques for correcting for multiple tests. We also illustrate potential pitfalls problems that can occur if the multiple comparisons issue is not dealt with properly. We conclude, by discussing effect size estimation, an issue often linked with the multiple comparisons problem. CONCLUSIONS: Failure to properly account for multiple comparisons will ultimately lead to heightened risks for false positives and erroneous conclusions.
Authors: Nikolaus Kriegeskorte; Martin A Lindquist; Thomas E Nichols; Russell A Poldrack; Edward Vul Journal: J Cereb Blood Flow Metab Date: 2010-06-23 Impact factor: 6.200
Authors: Katherine S Button; John P A Ioannidis; Claire Mokrysz; Brian A Nosek; Jonathan Flint; Emma S J Robinson; Marcus R Munafò Journal: Nat Rev Neurosci Date: 2013-04-10 Impact factor: 34.870
Authors: Amanda F Mejia; Mary Beth Nebel; Haochang Shou; Ciprian M Crainiceanu; James J Pekar; Stewart Mostofsky; Brian Caffo; Martin A Lindquist Journal: Neuroimage Date: 2015-02-28 Impact factor: 6.556
Authors: Nasr A I Farooqi; Martina Scotti; Ji Min Lew; Kelly N Botteron; Sherif Karama; James T McCracken; Tuong-Vi Nguyen Journal: Psychoneuroendocrinology Date: 2018-08-07 Impact factor: 4.905
Authors: Gregory Garbès Putzel; Giovanni Battistella; Anna F Rumbach; Laurie J Ozelius; Mert R Sabuncu; Kristina Simonyan Journal: Cereb Cortex Date: 2018-01-01 Impact factor: 5.357
Authors: Diana Petroi; Joseph R Duffy; Andrew Borgert; Edythe A Strand; Mary M Machulda; Matthew L Senjem; Clifford R Jack; Keith A Josephs; Jennifer L Whitwell Journal: Brain Lang Date: 2020-02-27 Impact factor: 2.381