A L Seidler1,2, K E Hunter3,4, D Espinoza3,4, S Mihrshahi4,5, L M Askie3,4. 1. NHMRC Clinical Trials Centre, University of Sydney, Sydney, Australia. lene.seidler@ctc.usyd.edu.au. 2. NHMRC Centre of Research Excellence in the Early Prevention of Obesity in Childhood, Prevention Research Collaboration, Sydney School of Public Health, University of Sydney, Camperdown, NSW, Australia. lene.seidler@ctc.usyd.edu.au. 3. NHMRC Clinical Trials Centre, University of Sydney, Sydney, Australia. 4. NHMRC Centre of Research Excellence in the Early Prevention of Obesity in Childhood, Prevention Research Collaboration, Sydney School of Public Health, University of Sydney, Camperdown, NSW, Australia. 5. Department of Health Systems and Populations, Faculty of Medicine, Health and Human Sciences, Macquarie University, Sydney, NSW, 2109, Australia.
Abstract
BACKGROUND: For prospective meta-analyses (PMAs), eligible studies are identified, and the PMA hypotheses, selection criteria, and analysis methods are pre-specified before the results of any of the studies are known. This reduces publication bias and selective outcome reporting and provides a unique opportunity for outcome standardisation/harmonisation. We conducted a world-first PMA of four trials investigating interventions to prevent early childhood obesity. The aims of this study were to quantitatively analyse the effects of prospective planning on variations across trials, outcome harmonisation, and the power to detect intervention effects, and to derive recommendations for future PMA. METHODS: We examined intervention design, participant characteristics, and outcomes collected across the four trials included in the EPOCH PMA using their registration records, protocol publications, and variable lists. The outcomes that trials planned to collect prior to inclusion in the PMA were compared to the outcomes that trials collected after PMA inclusion. We analysed the proportion of matching outcome definitions across trials, the number of outcomes per trial, and how collaboration increased the statistical power to detect intervention effects. RESULTS: The included trials varied in intervention design and participants, this improved external validity and the ability to perform subgroup analyses for the meta-analysis. While individual trials had limited power to detect the main intervention effect (BMI z-score), synthesising data substantially increased statistical power. Prospective planning led to an increase in the number of collected outcome categories (e.g. weight, child's diet, sleep), and greater outcome harmonisation. Prior to PMA inclusion, only 18% of outcome categories were included in all trials. After PMA inclusion, this increased to 91% of outcome categories. However, while trials mostly collected the same outcome categories after PMA inclusion, some inconsistencies in how the outcomes were measured remained (such as measuring physical activity by hours of outside play versus using an activity monitor). CONCLUSION: Prospective planning led to greater outcome harmonisation and greater power to detect intervention effects, while maintaining acceptable variation in trial designs and populations, which improved external validity. Recommendations for future PMA include more detailed harmonisation of outcome measures and careful pre-specification of analyses to avoid research waste by unnecessary over-collection of data.
BACKGROUND: For prospective meta-analyses (PMAs), eligible studies are identified, and the PMA hypotheses, selection criteria, and analysis methods are pre-specified before the results of any of the studies are known. This reduces publication bias and selective outcome reporting and provides a unique opportunity for outcome standardisation/harmonisation. We conducted a world-first PMA of four trials investigating interventions to prevent early childhood obesity. The aims of this study were to quantitatively analyse the effects of prospective planning on variations across trials, outcome harmonisation, and the power to detect intervention effects, and to derive recommendations for future PMA. METHODS: We examined intervention design, participant characteristics, and outcomes collected across the four trials included in the EPOCH PMA using their registration records, protocol publications, and variable lists. The outcomes that trials planned to collect prior to inclusion in the PMA were compared to the outcomes that trials collected after PMA inclusion. We analysed the proportion of matching outcome definitions across trials, the number of outcomes per trial, and how collaboration increased the statistical power to detect intervention effects. RESULTS: The included trials varied in intervention design and participants, this improved external validity and the ability to perform subgroup analyses for the meta-analysis. While individual trials had limited power to detect the main intervention effect (BMI z-score), synthesising data substantially increased statistical power. Prospective planning led to an increase in the number of collected outcome categories (e.g. weight, child's diet, sleep), and greater outcome harmonisation. Prior to PMA inclusion, only 18% of outcome categories were included in all trials. After PMA inclusion, this increased to 91% of outcome categories. However, while trials mostly collected the same outcome categories after PMA inclusion, some inconsistencies in how the outcomes were measured remained (such as measuring physical activity by hours of outside play versus using an activity monitor). CONCLUSION: Prospective planning led to greater outcome harmonisation and greater power to detect intervention effects, while maintaining acceptable variation in trial designs and populations, which improved external validity. Recommendations for future PMA include more detailed harmonisation of outcome measures and careful pre-specification of analyses to avoid research waste by unnecessary over-collection of data.
Authors: Catherine De Angelis; Jeffrey M Drazen; Frank A Frizelle; Charlotte Haug; John Hoey; Richard Horton; Sheldon Kotzin; Christine Laine; Ana Marusic; A John P M Overbeke; Torben V Schroeder; Hal C Sox; Martin B Van Der Weyden Journal: N Engl J Med Date: 2004-09-08 Impact factor: 91.245
Authors: Paula R Williamson; Douglas G Altman; Heather Bagley; Karen L Barnes; Jane M Blazeby; Sara T Brookes; Mike Clarke; Elizabeth Gargon; Sarah Gorst; Nicola Harman; Jamie J Kirkham; Angus McNair; Cecilia A C Prinsen; Jochen Schmitt; Caroline B Terwee; Bridget Young Journal: Trials Date: 2017-06-20 Impact factor: 2.279
Authors: Barry J Taylor; Anne-Louise M Heath; Barbara C Galland; Andrew R Gray; Julie A Lawrence; Rachel M Sayers; Kelly Dale; Kirsten J Coppell; Rachael W Taylor Journal: BMC Public Health Date: 2011-12-19 Impact factor: 3.295
Authors: Lynne A Daniels; Anthea Magarey; Diana Battistutta; Jan M Nicholson; Ann Farrell; Geoffrey Davidson; Geoffrey Cleghorn Journal: BMC Public Health Date: 2009-10-14 Impact factor: 3.295
Authors: Anna Lene Seidler; Lelia Duley; Anup C Katheria; Catalina De Paco Matallana; Eugene Dempsey; Heike Rabe; John Kattwinkel; Judith Mercer; Justin Josephsen; Karen Fairchild; Ola Andersson; Shigeharu Hosono; Venkataseshan Sundaram; Vikram Datta; Walid El-Naggar; William Tarnow-Mordi; Thomas Debray; Stuart B Hooper; Martin Kluckow; Graeme Polglase; Peter G Davis; Alan Montgomery; Kylie E Hunter; Angie Barba; John Simes; Lisa Askie Journal: BMJ Open Date: 2020-03-29 Impact factor: 2.692
Authors: Anna Lene Seidler; Kylie E Hunter; Saskia Cheyne; Jesse A Berlin; Davina Ghersi; Lisa M Askie Journal: Cochrane Database Syst Rev Date: 2020-10-30
Authors: Anna Lene Seidler; Kylie E Hunter; Louise Baur; David Espinoza; Rachael W Taylor; Li Ming Wen; Kylie D Hesketh; Karen Campbell; Lynne Daniels; Seema Mihrshahi; Chris Rissel; Barry Taylor; Lisa M Askie Journal: Pediatr Obes Date: 2022-04-08 Impact factor: 3.910
Authors: Seema Mihrshahi; Danielle Jawad; Louise Richards; Kylie E Hunter; Mahalakshmi Ekambareshwar; Anna Lene Seidler; Louise A Baur Journal: Int J Environ Res Public Health Date: 2021-03-02 Impact factor: 3.390
Authors: Kylie E Hunter; Brittany J Johnson; Lisa Askie; Rebecca K Golley; Louise A Baur; Ian C Marschner; Rachael W Taylor; Luke Wolfenden; Charles T Wood; Seema Mihrshahi; Alison J Hayes; Chris Rissel; Kristy P Robledo; Denise A O'Connor; David Espinoza; Lukas P Staub; Paul Chadwick; Sarah Taki; Angie Barba; Sol Libesman; Mason Aberoumand; Wendy A Smith; Michelle Sue-See; Kylie D Hesketh; Jessica L Thomson; Maria Bryant; Ian M Paul; Vera Verbestel; Cathleen Odar Stough; Li Ming Wen; Junilla K Larsen; Sharleen L O'Reilly; Heather M Wasser; Jennifer S Savage; Ken K Ong; Sarah-Jeanne Salvy; Mary Jo Messito; Rachel S Gross; Levie T Karssen; Finn E Rasmussen; Karen Campbell; Ana Maria Linares; Nina Cecilie Øverby; Cristina Palacios; Kaumudi J Joshipura; Carolina González Acero; Rajalakshmi Lakshman; Amanda L Thompson; Claudio Maffeis; Emily Oken; Ata Ghaderi; Maribel Campos Rivera; Ana B Pérez-Expósito; Jinan C Banna; Kayla de la Haye; Michael Goran; Margrethe Røed; Stephanie Anzman-Frasca; Barry J Taylor; Anna Lene Seidler Journal: BMJ Open Date: 2022-01-20 Impact factor: 2.692