Victoria Vickerstaff1, Gareth Ambler2, Michael King3, Irwin Nazareth4, Rumana Z Omar5. 1. Division of Psychiatry, University College London, 6th Floor, Maple House, 149 Tottenham Court Road, London W1T 7NF, UK; Department of Statistical Science, University College London, Gower Street, London WC1E 6BT, UK; The Research Department of Primary Care and Population Health, University College London, Rowland Hill Street, London NW3 2PF, UK. Electronic address: v.vickerstaff@ucl.ac.uk. 2. Department of Statistical Science, University College London, Gower Street, London WC1E 6BT, UK. Electronic address: g.ambler@ucl.ac.uk. 3. Division of Psychiatry, University College London, 6th Floor, Maple House, 149 Tottenham Court Road, London W1T 7NF, UK. Electronic address: michael.king@ucl.ac.uk. 4. The Research Department of Primary Care and Population Health, University College London, Rowland Hill Street, London NW3 2PF, UK. Electronic address: i.nazareth@ucl.ac.uk. 5. Department of Statistical Science, University College London, Gower Street, London WC1E 6BT, UK. Electronic address: r.omar@ucl.ac.uk.
Abstract
OBJECTIVES: To review how multiple primary outcomes are currently considered in the analysis of randomised controlled trials. We briefly describe the methods available to safeguard the inferences and to raise awareness of the potential problems caused by multiple outcomes. METHODS/ DESIGN: We reviewed randomised controlled trials (RCTs) in neurology and psychiatry disease areas, as these frequently analyse multiple outcomes. We reviewed all published RCTs from July 2011 to June 2014 inclusive in the following high impact journals: The New England Journal of Medicine, The Lancet, The American Journal of Psychiatry, JAMA Psychiatry, The Lancet Neurology and Neurology. We examined the information presented in the abstract and the methods used for sample size calculation and statistical analysis. We recorded the number of primary outcomes, the methods used to account for multiple primary outcomes, the number of outcomes discussed in the abstract and the number of outcomes used in the sample size calculation. RESULTS: Of the 209 RCTs that we identified, 60 (29%) analysed multiple primary outcomes. Of these, 45 (75%) did not adjust for multiplicity in their analyses. Had multiplicity been addressed, some of the trial conclusions would have changed. Of the 15 (25%) trials which accounted for multiplicity, Bonferroni's correction was the most commonly used method. CONCLUSIONS: Our review shows that trials with multiple primary outcomes are common. However, appropriate steps are not usually taken in most of the analyses to safeguard the inferences against multiplicity. Authors should state their chosen primary outcomes clearly and justify their methods of analysis.
OBJECTIVES: To review how multiple primary outcomes are currently considered in the analysis of randomised controlled trials. We briefly describe the methods available to safeguard the inferences and to raise awareness of the potential problems caused by multiple outcomes. METHODS/ DESIGN: We reviewed randomised controlled trials (RCTs) in neurology and psychiatry disease areas, as these frequently analyse multiple outcomes. We reviewed all published RCTs from July 2011 to June 2014 inclusive in the following high impact journals: The New England Journal of Medicine, The Lancet, The American Journal of Psychiatry, JAMA Psychiatry, The Lancet Neurology and Neurology. We examined the information presented in the abstract and the methods used for sample size calculation and statistical analysis. We recorded the number of primary outcomes, the methods used to account for multiple primary outcomes, the number of outcomes discussed in the abstract and the number of outcomes used in the sample size calculation. RESULTS: Of the 209 RCTs that we identified, 60 (29%) analysed multiple primary outcomes. Of these, 45 (75%) did not adjust for multiplicity in their analyses. Had multiplicity been addressed, some of the trial conclusions would have changed. Of the 15 (25%) trials which accounted for multiplicity, Bonferroni's correction was the most commonly used method. CONCLUSIONS: Our review shows that trials with multiple primary outcomes are common. However, appropriate steps are not usually taken in most of the analyses to safeguard the inferences against multiplicity. Authors should state their chosen primary outcomes clearly and justify their methods of analysis.
Authors: Pascale Nevins; Shelley Vanderhout; Kelly Carroll; Stuart G Nicholls; Seana N Semchishen; Jamie C Brehaut; Dean A Fergusson; Bruno Giraudeau; Monica Taljaard Journal: J Clin Epidemiol Date: 2021-12-08 Impact factor: 7.407
Authors: Franziska Holtdirk; Anja Mehnert; Mario Weiss; Johannes Mayer; Björn Meyer; Peter Bröde; Maren Claus; Carsten Watzl Journal: PLoS One Date: 2021-05-07 Impact factor: 3.240
Authors: Nicholas D Myers; Isaac Prilleltensky; Seungmin Lee; Samantha Dietz; Ora Prilleltensky; Adam McMahon; Karin A Pfeiffer; Morgan E Ellithorpe; Ahnalee M Brincks Journal: BMC Public Health Date: 2019-06-13 Impact factor: 3.295
Authors: Miguel Sampayo-Cordero; Bernat Miguel-Huguet; Almudena Pardo-Mateos; Andrea Malfettone; José Pérez-García; Antonio Llombart-Cussac; Javier Cortés; Marc Moltó-Abad; Cecilia Muñoz-Delgado; Marta Pérez-Quintana; Jordi Pérez-López Journal: Orphanet J Rare Dis Date: 2019-10-21 Impact factor: 4.123
Authors: Nicholas D Myers; Isaac Prilleltensky; Adam McMahon; Ahnalee M Brincks; Seungmin Lee; Ora Prilleltensky; Karin A Pfeiffer; André G Bateman Journal: Prev Sci Date: 2021-06-25