Literature DB >> 19648445

What is the probability of replicating a statistically significant effect?

Jeff Miller1.   

Abstract

If an initial experiment produces a statistically significant effect, what is the probability that this effect will be replicated in a follow-up experiment? I argue that this seemingly fundamental question can be interpreted in two very different ways and that its answer is, in practice, virtually unknowable under either interpretation. Although the data from an initial experiment can be used to estimate one type of replication probability, this estimate will rarely be precise enough to be of any use. The other type of replication probability is also unknowable, because it depends on unknown aspects of the research context. Thus, although it would be nice to know the probability of replicating a significant effect, researchers must accept the fact that they generally cannot determine this information, whichever type of replication probability they seek.

Mesh:

Year:  2009        PMID: 19648445     DOI: 10.3758/PBR.16.4.617

Source DB:  PubMed          Journal:  Psychon Bull Rev        ISSN: 1069-9384


  22 in total

1.  Null hypothesis significance testing: a review of an old and continuing controversy.

Authors:  R S Nickerson
Journal:  Psychol Methods       Date:  2000-06

2.  A model-averaging approach to replication: the case of p rep.

Authors:  Geoffrey J Iverson; Eric-Jan Wagenmakers; Michael D Lee
Journal:  Psychol Methods       Date:  2010-06

3.  Likelihood ratios: a simple and flexible statistic for empirical psychologists.

Authors:  Scott Glover; Peter Dixon
Journal:  Psychon Bull Rev       Date:  2004-10

4.  Probability of replication revisited: comment on "An alternative to Null-Hypothesis Significance Tests.".

Authors:  Gheorghe Doros; Andrew B Geier
Journal:  Psychol Sci       Date:  2005-12

5.  Understanding the average probability of replication: comment on Killeen (2005).

Authors:  Geoff Cumming
Journal:  Psychol Sci       Date:  2005-12

6.  A practical solution to the pervasive problems of p values.

Authors:  Eric-Jan Wagenmakers
Journal:  Psychon Bull Rev       Date:  2007-10

7.  A power primer.

Authors:  J Cohen
Journal:  Psychol Bull       Date:  1992-07       Impact factor: 17.737

8.  Two-sided confidence intervals for the single proportion: comparison of seven methods.

Authors:  R G Newcombe
Journal:  Stat Med       Date:  1998-04-30       Impact factor: 2.373

9.  Replication and p Intervals: p Values Predict the Future Only Vaguely, but Confidence Intervals Do Much Better.

Authors:  Geoff Cumming
Journal:  Perspect Psychol Sci       Date:  2008-07

Review 10.  The efficacy of psychological, educational, and behavioral treatment. Confirmation from meta-analysis.

Authors:  M W Lipsey; D B Wilson
Journal:  Am Psychol       Date:  1993-12
View more
  17 in total

1.  Replication is not coincidence: reply to Iverson, Lee, and Wagenmakers (2009).

Authors:  Bruno Lecoutre; Peter R Killeen
Journal:  Psychon Bull Rev       Date:  2010-04

2.  Revised standards for statistical evidence.

Authors:  Valen E Johnson
Journal:  Proc Natl Acad Sci U S A       Date:  2013-11-11       Impact factor: 11.205

3.  Accounting for Differential Item Functioning Using Bayesian Approximate Measurement Invariance.

Authors:  Georgios D Sideridis; Ioannis Tsaousis; Abeer A Alamri
Journal:  Educ Psychol Meas       Date:  2019-12-04       Impact factor: 2.821

Review 4.  Addressing the theory crisis in psychology.

Authors:  Klaus Oberauer; Stephan Lewandowsky
Journal:  Psychon Bull Rev       Date:  2019-10

5.  Using Classroom Data to Teach Students about Data Cleaning and Testing Assumptions.

Authors:  Kevin Cummiskey; Shonda Kuiper; Rodney Sturdivant
Journal:  Front Psychol       Date:  2012-09-25

6.  Quantifying the Time Course of Visual Object Processing Using ERPs: It's Time to Up the Game.

Authors:  Guillaume A Rousselet; Cyril R Pernet
Journal:  Front Psychol       Date:  2011-05-23

7.  Modeling Single-Trial ERP Reveals Modulation of Bottom-Up Face Visual Processing by Top-Down Task Constraints (in Some Subjects).

Authors:  Guillaume A Rousselet; Carl M Gaspar; Kacper P Wieczorek; Cyril R Pernet
Journal:  Front Psychol       Date:  2011-06-23

8.  Improving standards in brain-behavior correlation analyses.

Authors:  Guillaume A Rousselet; Cyril R Pernet
Journal:  Front Hum Neurosci       Date:  2012-05-03       Impact factor: 3.169

9.  Brain network informed subject community detection in early-onset schizophrenia.

Authors:  Zhi Yang; Yong Xu; Ting Xu; Colin W Hoy; Daniel A Handwerker; Gang Chen; Georg Northoff; Xi-Nian Zuo; Peter A Bandettini
Journal:  Sci Rep       Date:  2014-07-03       Impact factor: 4.379

Review 10.  Fifty psychological and psychiatric terms to avoid: a list of inaccurate, misleading, misused, ambiguous, and logically confused words and phrases.

Authors:  Scott O Lilienfeld; Katheryn C Sauvigné; Steven Jay Lynn; Robin L Cautin; Robert D Latzman; Irwin D Waldman
Journal:  Front Psychol       Date:  2015-08-03
View more

北京卡尤迪生物科技股份有限公司 © 2022-2023.