Literature DB >> 32425159

Dataset decay and the problem of sequential analyses on open datasets.

William Hedley Thompson1,2, Jessey Wright1,3, Patrick G Bissett1, Russell A Poldrack1.   

Abstract

Open data allows researchers to explore pre-existing datasets in new ways. However, if many researchers reuse the same dataset, multiple statistical testing may increase false positives. Here we demonstrate that sequential hypothesis testing on the same dataset by multiple researchers can inflate error rates. We go on to discuss a number of correction procedures that can reduce the number of false positives, and the challenges associated with these correction procedures.
© 2020, Thompson et al.

Entities:  

Keywords:  computational biology; human; meta-research; multiple comparison correction; multiple comparisons; neuroscience; open data; sequential testing; systems biology

Mesh：

Year:  2020        PMID： 32425159      PMCID： PMC7237204          DOI： 10.7554/eLife.53498

Source DB:  PubMed          Journal:  Elife        ISSN： 2050-084X            Impact factor:   8.140


  21 in total

1.  P-curve: a key to the file-drawer.

Authors:  Uri Simonsohn; Leif D Nelson; Joseph P Simmons
Journal:  J Exp Psychol Gen       Date:  2013-07-15

2.  An Agenda for Purely Confirmatory Research.

Authors:  Eric-Jan Wagenmakers; Ruud Wetzels; Denny Borsboom; Han L J van der Maas; Rogier A Kievit
Journal:  Perspect Psychol Sci       Date:  2012-11

3.  Cross-validation failure: Small sample sizes lead to large error bars.

Authors:  Gaël Varoquaux
Journal:  Neuroimage       Date:  2017-06-24       Impact factor: 6.556

4.  The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective.

Authors:  John K Kruschke; Torrin M Liddell
Journal:  Psychon Bull Rev       Date:  2018-02

Review 5.  Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning.

Authors:  Tal Yarkoni; Jacob Westfall
Journal:  Perspect Psychol Sci       Date:  2017-08-25

Review 6.  The WU-Minn Human Connectome Project: an overview.

Authors:  David C Van Essen; Stephen M Smith; Deanna M Barch; Timothy E J Behrens; Essa Yacoub; Kamil Ugurbil
Journal:  Neuroimage       Date:  2013-05-16       Impact factor: 6.556

7.  Hidden multiplicity in exploratory multiway ANOVA: Prevalence and remedies.

Authors:  Angélique O J Cramer; Don van Ravenzwaaij; Dora Matzke; Helen Steingroever; Ruud Wetzels; Raoul P P P Grasman; Lourens J Waldorp; Eric-Jan Wagenmakers
Journal:  Psychon Bull Rev       Date:  2016-04

8.  How open science helps researchers succeed.

Authors:  Erin C McKiernan; Philip E Bourne; C Titus Brown; Stuart Buck; Amye Kenall; Jennifer Lin; Damon McDougall; Brian A Nosek; Karthik Ram; Courtney K Soderberg; Jeffrey R Spies; Kaitlin Thaney; Andrew Updegrove; Kara H Woo; Tal Yarkoni
Journal:  Elife       Date:  2016-07-07       Impact factor: 8.140

Review 9.  The Human Connectome Project: a data acquisition perspective.

Authors:  D C Van Essen; K Ugurbil; E Auerbach; D Barch; T E J Behrens; R Bucholz; A Chang; L Chen; M Corbetta; S W Curtiss; S Della Penna; D Feinberg; M F Glasser; N Harel; A C Heath; L Larson-Prior; D Marcus; G Michalareas; S Moeller; R Oostenveld; S E Petersen; F Prior; B L Schlaggar; S M Smith; A Z Snyder; J Xu; E Yacoub
Journal:  Neuroimage       Date:  2012-02-17       Impact factor: 6.556

10.  PSYCHOLOGY. Estimating the reproducibility of psychological science.

Authors: 
Journal:  Science       Date:  2015-08-28       Impact factor: 47.728
View more
  8 in total

Review 1.  Psychology needs to get tired of winning.

Authors:  Gerald J Haeffel
Journal:  R Soc Open Sci       Date:  2022-06-22       Impact factor: 3.653

2.  Data overuse in aging research: Emerging issues and potential solutions.

Authors:  Daniel K Mroczek; Sara J Weston; Eileen K Graham; Emily C Willroth
Journal:  Psychol Aging       Date:  2021-04-29

3.  Prevalence of neurocognitive and perceived speech deficits in patients with head and neck cancer before treatment: Associations with demographic, behavioral, and disease-related factors.

Authors:  Vitória Piai; Femke Jansen; Kristoffer Dahlslätt; Irma M Verdonck-de Leeuw; Judith Prins; René Leemans; Chris H J Terhaard; Johannes A Langendijk; Robert J Baatenburg de Jong; Johannes H Smit; Robert Takes; Roy P C Kessels
Journal:  Head Neck       Date:  2021-11-19       Impact factor: 3.821

4.  Protecting against researcher bias in secondary data analysis: challenges and potential solutions.

Authors:  Jessie R Baldwin; Jean-Baptiste Pingault; Tabea Schoeler; Hannah M Sallis; Marcus R Munafò
Journal:  Eur J Epidemiol       Date:  2022-01-13       Impact factor: 8.082

5.  Standardizing workflows in imaging transcriptomics with the abagen toolbox.

Authors:  Ross D Markello; Aurina Arnatkeviciute; Jean-Baptiste Poline; Ben D Fulcher; Alex Fornito; Bratislav Misic
Journal:  Elife       Date:  2021-11-16       Impact factor: 8.140

Review 6.  Machine learning for medical imaging: methodological failures and recommendations for the future.

Authors:  Gaël Varoquaux; Veronika Cheplygina
Journal:  NPJ Digit Med       Date:  2022-04-12

7.  Neuroscout, a unified platform for generalizable and reproducible fMRI research.

Authors:  Alejandro de la Vega; Roberta Rocca; Ross W Blair; Christopher J Markiewicz; Jeff Mentch; James D Kent; Peer Herholz; Satrajit S Ghosh; Russell A Poldrack; Tal Yarkoni
Journal:  Elife       Date:  2022-08-30       Impact factor: 8.713

8.  Decreased amygdala volume in adults after premature birth.

Authors:  Dennis M Hedderich; Christian Sorg; Benita Schmitz-Koep; Juliana Zimmermann; Aurore Menegaux; Rachel Nuttall; Josef G Bäuml; Sebastian C Schneider; Marcel Daamen; Henning Boecker; Claus Zimmer; Dieter Wolke; Peter Bartmann
Journal:  Sci Rep       Date:  2021-03-08       Impact factor: 4.379
  8 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.