Richard Hooper1. 1. National Heart & Lung Institute, Imperial College London, UK. richard.hooper2@imperial.ac.uk
Abstract
OBJECTIVE: It is often repeated that a low P-value provides more persuasive evidence for a genuine effect if the power of the test is high. However, this is based on an argument which ignores the precise P-value in favor of simply observing whether P is less than some cut-off, and which oversimplifies the possible effect sizes. In a non-Bayesian framework, there are good reasons to think that power does not affect the evidence of a given P-value. Here I illustrate the relationship between pre-study power and the Bayesian interpretation of a P-value in realistic situations. STUDY DESIGN AND SETTING: A Bayesian calculation, using a conventional prior distribution for the effect size and a normal approximation to the sampling distribution of the sample estimate, where the datum is the precise P-value. RESULTS: Over the range of pre-study powers typical in published research, the Bayesian interpretation of a given P-value varies little with power. CONCLUSION: A Bayesian analysis with reasonable assumptions produces results remarkably in line with a more simple, non-Bayesian intuition-that the evidence against the null hypothesis provided by a precise P-value should not depend on power.
OBJECTIVE: It is often repeated that a low P-value provides more persuasive evidence for a genuine effect if the power of the test is high. However, this is based on an argument which ignores the precise P-value in favor of simply observing whether P is less than some cut-off, and which oversimplifies the possible effect sizes. In a non-Bayesian framework, there are good reasons to think that power does not affect the evidence of a given P-value. Here I illustrate the relationship between pre-study power and the Bayesian interpretation of a P-value in realistic situations. STUDY DESIGN AND SETTING: A Bayesian calculation, using a conventional prior distribution for the effect size and a normal approximation to the sampling distribution of the sample estimate, where the datum is the precise P-value. RESULTS: Over the range of pre-study powers typical in published research, the Bayesian interpretation of a given P-value varies little with power. CONCLUSION: A Bayesian analysis with reasonable assumptions produces results remarkably in line with a more simple, non-Bayesian intuition-that the evidence against the null hypothesis provided by a precise P-value should not depend on power.
Authors: Timothy Iveson; Kathleen A Boyd; Rachel S Kerr; Jose Robles-Zurita; Mark P Saunders; Andrew H Briggs; Jim Cassidy; Niels Henrik Hollander; Josep Tabernero; Andrew Haydon; Bengt Glimelius; Andrea Harkin; Karen Allan; John McQueen; Sarah Pearson; Ashita Waterston; Louise Medley; Charles Wilson; Richard Ellis; Sharadah Essapen; Amandeep S Dhadda; Mark Harrison; Stephen Falk; Sherif Raouf; Charlotte Rees; Rene K Olesen; David Propper; John Bridgewater; Ashraf Azzabi; David Farrugia; Andrew Webb; David Cunningham; Tamas Hickish; Andrew Weaver; Simon Gollins; Harpreet Wasan; James Paul Journal: Health Technol Assess Date: 2019-12 Impact factor: 4.014
Authors: Timothy J Iveson; Rachel S Kerr; Mark P Saunders; Jim Cassidy; Niels Henrik Hollander; Josep Tabernero; Andrew Haydon; Bengt Glimelius; Andrea Harkin; Karen Allan; John McQueen; Claire Scudder; Kathleen Anne Boyd; Andrew Briggs; Ashita Waterston; Louise Medley; Charles Wilson; Richard Ellis; Sharadah Essapen; Amandeep S Dhadda; Mark Harrison; Stephen Falk; Sherif Raouf; Charlotte Rees; Rene K Olesen; David Propper; John Bridgewater; Ashraf Azzabi; David Farrugia; Andrew Webb; David Cunningham; Tamas Hickish; Andrew Weaver; Simon Gollins; Harpreet S Wasan; James Paul Journal: Lancet Oncol Date: 2018-04 Impact factor: 41.316