N Fox1, N Mathers. 1. Institute of General Practice and Primary Care, School of Health and Related Research, University of Sheffield, UK.
Abstract
BACKGROUND: Statistical power is a measure of the extent to which a study is capable of discerning differences or associations which exist within the population under investigation, and is of critical importance whenever a hypothesis is tested by statistics. Conventionally, studies should reach a power level of 0.8, such that four times out of five a false null hypothesis will be rejected by a study. Statistical power may most easily be increased by increasing sample size. OBJECTIVE: We aimed to assess the level of statistical power of general practice research. METHODS: A total of 1422 statistical tests in 85 quantitative original papers in the British Journal of General Practice were analysed for statistical power. RESULTS: The median power of tests analysed was 0.71, representing a slightly greater than two-thirds likelihood of rejecting false null hypotheses. Of 85 studies, 37 (44%) attained power of 0.8 or more. Ten studies had power of more than 0.99 suggesting 'over-powering'. Twenty-one of the papers surveyed (25%) had a likelihood of gaining significant results poorer than that obtained by tossing a coin when a null hypothesis is false. CONCLUSION: While achieving higher power than studies in similar surveys of other disciplines, the power of general practice research falls short of the 0.8 convention. Adequate power is essential so that effects which exist are not missed. Recommendations are made concerning power calculations prior to the start of research and reporting of results in journal articles.
BACKGROUND: Statistical power is a measure of the extent to which a study is capable of discerning differences or associations which exist within the population under investigation, and is of critical importance whenever a hypothesis is tested by statistics. Conventionally, studies should reach a power level of 0.8, such that four times out of five a false null hypothesis will be rejected by a study. Statistical power may most easily be increased by increasing sample size. OBJECTIVE: We aimed to assess the level of statistical power of general practice research. METHODS: A total of 1422 statistical tests in 85 quantitative original papers in the British Journal of General Practice were analysed for statistical power. RESULTS: The median power of tests analysed was 0.71, representing a slightly greater than two-thirds likelihood of rejecting false null hypotheses. Of 85 studies, 37 (44%) attained power of 0.8 or more. Ten studies had power of more than 0.99 suggesting 'over-powering'. Twenty-one of the papers surveyed (25%) had a likelihood of gaining significant results poorer than that obtained by tossing a coin when a null hypothesis is false. CONCLUSION: While achieving higher power than studies in similar surveys of other disciplines, the power of general practice research falls short of the 0.8 convention. Adequate power is essential so that effects which exist are not missed. Recommendations are made concerning power calculations prior to the start of research and reporting of results in journal articles.