Patrice Sutton1, Nicholas Chartres2, Swati D G Rayasam3, Natalyn Daniels4, Juleen Lam5, Eman Maghrbi6, Tracey J Woodruff7. 1. UCSF Program on Reproductive Health and the Environment, Department of Obstetrics, Gynecology & Reproductive Sciences, Box 0132, 490 Illinois Street, Floor 10, San Francisco, CA 94143, United States. Electronic address: patrice.sutton@ucsf.edu. 2. UCSF Program on Reproductive Health and the Environment, Department of Obstetrics, Gynecology & Reproductive Sciences, Box 0132, 490 Illinois Street, Floor 10, San Francisco, CA 94143, United States. Electronic address: nicholas.chartres@ucsf.edu. 3. UCSF Program on Reproductive Health and the Environment, Department of Obstetrics, Gynecology & Reproductive Sciences, Box 0132, 490 Illinois Street, Floor 10, San Francisco, CA 94143, United States. Electronic address: swati.rayasam@ucsf.edu. 4. UCSF Program on Reproductive Health and the Environment, Department of Obstetrics, Gynecology & Reproductive Sciences, Box 0132, 490 Illinois Street, Floor 10, San Francisco, CA 94143, United States. 5. Department of Health Sciences, California State University East Bay, SF 533, 25800 Carlos Bee Blvd, Hayward, CA 94542, United States. Electronic address: juleen.lam@csueastbay.edu. 6. Department of Health Sciences, California State University East Bay, SF 533, 25800 Carlos Bee Blvd, Hayward, CA 94542, United States. Electronic address: emaghrbi@horizon.csueastbay.edu. 7. UCSF Program on Reproductive Health and the Environment, Department of Obstetrics, Gynecology & Reproductive Sciences, Box 0132, 490 Illinois Street, Floor 10, San Francisco, CA 94143, United States. Electronic address: tracey.woodruff@ucsf.edu.
Abstract
BACKGROUND: Synthesizing environmental health science is crucial to taking action to protect public health. Procedures for evidence evaluation and integration are transitioning from "expert-based narrative" to "systematic" review methods. However, little is known about the methodology being utilized for either type of review. OBJECTIVES: To appraise the methodological strengths and weaknesses of a sample of "expert-based narrative" and "systematic" reviews in environmental health. METHODS: We conducted a comprehensive search of multiple databases and identified relevant reviews using pre-specified eligibility criteria. We applied a modified version of the Literature Review Appraisal Toolkit (LRAT) to three environmental health topics that assessed the utility, validity and transparency of reviews. RESULTS: We identified 29 reviews published between 2003 and 2019, of which 13 (45%) were self-identified as systematic reviews. Across every LRAT domain, systematic reviews received a higher percentage of "satisfactory" ratings compared to non-systematic reviews. In eight of these domains, there was a statistically significant difference observed between the two types of reviews and "satisfactory" ratings. Non-systematic reviews performed poorly with the majority receiving an "unsatisfactory" or "unclear" rating in 11 of the 12 domains. Systematic reviews performed poorly in six of the 12 domains; 10 (77%) did not state the reviews objectives or develop a protocol; eight (62%) did not state the roles and contribution of the authors, or evaluate the internal validity of the included evidence consistently using a valid method; and only seven (54%) stated a pre-defined definition of the evidence bar on which their conclusions were based, or had an author disclosure of interest statement. DISCUSSION: Systematic reviews produced more useful, valid, and transparent conclusions compared to non-systematic reviews, but poorly conducted systematic reviews were prevalent. Ongoing development and implementation of empirically based systematic review methods are required in environmental health to ensure transparent and timely decision making to protect the public's health.
BACKGROUND: Synthesizing environmental health science is crucial to taking action to protect public health. Procedures for evidence evaluation and integration are transitioning from "expert-based narrative" to "systematic" review methods. However, little is known about the methodology being utilized for either type of review. OBJECTIVES: To appraise the methodological strengths and weaknesses of a sample of "expert-based narrative" and "systematic" reviews in environmental health. METHODS: We conducted a comprehensive search of multiple databases and identified relevant reviews using pre-specified eligibility criteria. We applied a modified version of the Literature Review Appraisal Toolkit (LRAT) to three environmental health topics that assessed the utility, validity and transparency of reviews. RESULTS: We identified 29 reviews published between 2003 and 2019, of which 13 (45%) were self-identified as systematic reviews. Across every LRAT domain, systematic reviews received a higher percentage of "satisfactory" ratings compared to non-systematic reviews. In eight of these domains, there was a statistically significant difference observed between the two types of reviews and "satisfactory" ratings. Non-systematic reviews performed poorly with the majority receiving an "unsatisfactory" or "unclear" rating in 11 of the 12 domains. Systematic reviews performed poorly in six of the 12 domains; 10 (77%) did not state the reviews objectives or develop a protocol; eight (62%) did not state the roles and contribution of the authors, or evaluate the internal validity of the included evidence consistently using a valid method; and only seven (54%) stated a pre-defined definition of the evidence bar on which their conclusions were based, or had an author disclosure of interest statement. DISCUSSION: Systematic reviews produced more useful, valid, and transparent conclusions compared to non-systematic reviews, but poorly conducted systematic reviews were prevalent. Ongoing development and implementation of empirically based systematic review methods are required in environmental health to ensure transparent and timely decision making to protect the public's health.
Authors: Hanna M Vesterinen; Paula I Johnson; Dylan S Atchley; Patrice Sutton; Juleen Lam; Marya G Zlatnik; Saunak Sen; Tracey J Woodruff Journal: J Matern Fetal Neonatal Med Date: 2015-09-04
Authors: Rebecca L Morgan; Kristina A Thayer; Lisa Bero; Nigel Bruce; Yngve Falck-Ytter; Davina Ghersi; Gordon Guyatt; Carlijn Hooijmans; Miranda Langendam; Daniele Mandrioli; Reem A Mustafa; Eva A Rehfuess; Andrew A Rooney; Beverley Shea; Ellen K Silbergeld; Patrice Sutton; Mary S Wolfe; Tracey J Woodruff; Jos H Verbeek; Alison C Holloway; Nancy Santesso; Holger J Schünemann Journal: Environ Int Date: 2016-01-27 Impact factor: 9.621
Authors: Beverley J Shea; Jeremy M Grimshaw; George A Wells; Maarten Boers; Neil Andersson; Candyce Hamel; Ashley C Porter; Peter Tugwell; David Moher; Lex M Bouter Journal: BMC Med Res Methodol Date: 2007-02-15 Impact factor: 4.615
Authors: Paul Whaley; Bas J Blaauboer; Jan Brozek; Elaine A Cohen Hubal; Kaitlyn Hair; Sam Kacew; Thomas B Knudsen; Carol F Kwiatkowski; David T Mellor; Andrew F Olshan; Matthew J Page; Andrew A Rooney; Elizabeth G Radke; Larissa Shamseer; Katya Tsaioun; Peter Tugwell; Daniele Wikoff; Tracey J Woodruff Journal: ALTEX Date: 2021-06-22 Impact factor: 6.250