Dena Zeraatkar1,2, Arrti Bhasin1, Rita E Morassut3, Isabella Churchill1, Arnav Gupta4, Daeria O Lawson1, Anna Miroshnychenko1, Emily Sirotich1, Komal Aryal1, David Mikhail5, Tauseef A Khan6,7, Vanessa Ha8, John L Sievenpiper6,7, Steven E Hanna1, Joseph Beyene1, Russell J de Souza1,7,9. 1. Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada. 2. Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA. 3. Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada. 4. Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada. 5. Faculty of Science, McMaster University, Hamilton, Ontario, Canada. 6. Department of Nutritional Sciences, Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada. 7. 3D Knowledge Synthesis and Clinical Trials Unit, Clinical Nutrition and Risk Factor Modification Centre, Division of Endocrinology & Metabolism, St. Michael's Hospital, Toronto, Ontario, Canada. 8. School of Medicine, Queen's University, Kingston, Ontario, Canada. 9. Population Health Research Institute, McMaster University, Hamilton, Ontario, Canada.
Abstract
BACKGROUND: Dietary recommendations and policies should be guided by rigorous systematic reviews. Reviews that are of poor methodological quality may be ineffective or misleading. Most of the evidence in nutrition comes from nonrandomized studies of nutritional exposures (usually referred to as nutritional epidemiology studies), but to date methodological evaluations of the quality of systematic reviews of such studies have been sparse and inconsistent. OBJECTIVES: We aimed to investigate the quality of recently published systematic reviews and meta-analyses of nutritional epidemiology studies and to propose guidance addressing major limitations. METHODS: We searched MEDLINE (January 2018-August 2019), EMBASE (January 2018-August 2019), and the Cochrane Database of Systematic Reviews (January 2018-February 2019) for systematic reviews of nutritional epidemiology studies. We included a random sample of 150 reviews. RESULTS: Most reviews were published by authors from Asia (n = 49; 32.7%) or Europe (n = 43; 28.7%) and investigated foods or beverages (n = 60; 40.0%) and cancer morbidity and mortality (n = 54; 36%). Reviews often had important limitations: less than one-quarter (n = 30; 20.0%) reported preregistration of a protocol and almost one-third (n = 42; 28.0%) did not report a replicable search strategy. Suboptimal practices and errors in the synthesis of results were common: one-quarter of meta-analyses (n = 30; 26.1%) selected the meta-analytic model based on statistical indicators of heterogeneity and almost half of meta-analyses (n = 50; 43.5%) did not consider dose-response associations even when it was appropriate to do so. Only 16 (10.7%) reviews used an established system to evaluate the certainty of evidence. CONCLUSIONS: Systematic reviews of nutritional epidemiology studies often have serious limitations. Authors can improve future reviews by involving statisticians, methodologists, and researchers with substantive knowledge in the specific area of nutrition being studied and using a rigorous and transparent system to evaluate the certainty of evidence.
BACKGROUND: Dietary recommendations and policies should be guided by rigorous systematic reviews. Reviews that are of poor methodological quality may be ineffective or misleading. Most of the evidence in nutrition comes from nonrandomized studies of nutritional exposures (usually referred to as nutritional epidemiology studies), but to date methodological evaluations of the quality of systematic reviews of such studies have been sparse and inconsistent. OBJECTIVES: We aimed to investigate the quality of recently published systematic reviews and meta-analyses of nutritional epidemiology studies and to propose guidance addressing major limitations. METHODS: We searched MEDLINE (January 2018-August 2019), EMBASE (January 2018-August 2019), and the Cochrane Database of Systematic Reviews (January 2018-February 2019) for systematic reviews of nutritional epidemiology studies. We included a random sample of 150 reviews. RESULTS: Most reviews were published by authors from Asia (n = 49; 32.7%) or Europe (n = 43; 28.7%) and investigated foods or beverages (n = 60; 40.0%) and cancer morbidity and mortality (n = 54; 36%). Reviews often had important limitations: less than one-quarter (n = 30; 20.0%) reported preregistration of a protocol and almost one-third (n = 42; 28.0%) did not report a replicable search strategy. Suboptimal practices and errors in the synthesis of results were common: one-quarter of meta-analyses (n = 30; 26.1%) selected the meta-analytic model based on statistical indicators of heterogeneity and almost half of meta-analyses (n = 50; 43.5%) did not consider dose-response associations even when it was appropriate to do so. Only 16 (10.7%) reviews used an established system to evaluate the certainty of evidence. CONCLUSIONS: Systematic reviews of nutritional epidemiology studies often have serious limitations. Authors can improve future reviews by involving statisticians, methodologists, and researchers with substantive knowledge in the specific area of nutrition being studied and using a rigorous and transparent system to evaluate the certainty of evidence.
Authors: Howard Balshem; Mark Helfand; Holger J Schünemann; Andrew D Oxman; Regina Kunz; Jan Brozek; Gunn E Vist; Yngve Falck-Ytter; Joerg Meerpohl; Susan Norris; Gordon H Guyatt Journal: J Clin Epidemiol Date: 2011-01-05 Impact factor: 6.437
Authors: James R Hébert; Edward A Frongillo; Swann A Adams; Gabrielle M Turner-McGrievy; Thomas G Hurley; Donald R Miller; Ira S Ockene Journal: Adv Nutr Date: 2016-05-16 Impact factor: 8.701
Authors: Matthew J Page; Larissa Shamseer; Douglas G Altman; Jennifer Tetzlaff; Margaret Sampson; Andrea C Tricco; Ferrán Catalá-López; Lun Li; Emma K Reid; Rafael Sarkis-Onofre; David Moher Journal: PLoS Med Date: 2016-05-24 Impact factor: 11.069
Authors: Paul Garner; Sally Hopewell; Jackie Chandler; Harriet MacLehose; Holger J Schünemann; Elie A Akl; Joseph Beyene; Stephanie Chang; Rachel Churchill; Karin Dearness; Gordon Guyatt; Carol Lefebvre; Beth Liles; Rachel Marshall; Laura Martínez García; Chris Mavergames; Mona Nasser; Amir Qaseem; Margaret Sampson; Karla Soares-Weiser; Yemisi Takwoingi; Lehana Thabane; Marialena Trivella; Peter Tugwell; Emma Welsh; Ed C Wilson; Holger J Schünemann Journal: BMJ Date: 2016-07-20
Authors: Shannon E Kelly; Linda S Greene-Finestone; Elizabeth A Yetley; Karima Benkhedda; Stephen P J Brooks; George A Wells; Amanda J MacFarlane Journal: Am J Clin Nutr Date: 2022-01-11 Impact factor: 7.045
Authors: Dena Zeraatkar; Alana Kohut; Arrti Bhasin; Rita E Morassut; Isabella Churchill; Arnav Gupta; Daeria Lawson; Anna Miroshnychenko; Emily Sirotich; Komal Aryal; Maria Azab; Joseph Beyene; Russell J de Souza Journal: BMJ Nutr Prev Health Date: 2021-12-07
Authors: Dawid Storman; Magdalena Koperny; Joanna Zając; Maciej Polak; Paulina Weglarz; Justyna Bochenek-Cibor; Mateusz J Swierz; Wojciech Staskiewicz; Magdalena Gorecka; Anna Skuza; Adam A Wach; Klaudia Kaluzinska; Małgorzata M Bała Journal: Int J Environ Res Public Health Date: 2022-01-03 Impact factor: 3.390