Elizabeth G Radke1, Barbara Glenn2, Audrey Galizia2, Amanda Persad2, Rebecca Nachman2, Thomas Bateson2, J Michael Wright2, Ana Navas-Acien3, Whitney D Arroyave4, Robin C Puett5, Emily W Harville6, Anna Z Pollack7, Jane S Burns8, Courtney D Lynch9, Sharon K Sagiv10, Cheryl Stein11, Glinda S Cooper12. 1. U.S. Environmental Protection Agency, National Center for Environmental Assessment, United States. Electronic address: radke-farabaugh.elizabeth@epa.gov. 2. U.S. Environmental Protection Agency, National Center for Environmental Assessment, United States. 3. Department of Environmental Health Sciences, Columbia University Mailman School of Public Health, United States. 4. Integrated Laboratory Systems, United States. 5. Department of Epidemiology and Biostatistics, University of Maryland School of Public Health, United States. 6. Department of Epidemiology, Tulane University School of Public Health and Tropical Medicine, United States. 7. Department of Global and Community Health, College of Health and Human Services, George Mason University, United States. 8. Department of Environmental Health, Harvard T. H. Chan School of Public Health, United States. 9. Department of Obstetrics and Gynecology, The Ohio State University College of Medicine, United States. 10. Division of Epidemiology, University of California Berkeley, United States. 11. Department of Child and Adolescent Psychiatry, Hassenfeld Children's Hospital at NYU Langone, United States. 12. U.S. Environmental Protection Agency, National Center for Environmental Assessment, United States; The Innocence Project, United States.
Abstract
INTRODUCTION AND OBJECTIVE: Systematic review tools that provide guidance on evaluating epidemiology studies are receiving increasing attention and support because their application facilitates improved quality of the review, consistency across reviewers, and transparency for readers. The U.S. Environmental Protection Agency's Integrated Risk Information System (IRIS) Program has developed an approach for systematic review of evidence of health effects from chemical exposures that includes structured approaches for literature search and screening, study evaluation, data extraction, and evidence synthesis and integration. This approach recognizes the need for developing outcome-specific criteria for study evaluation. Because studies are assessed at the outcome level, a study could be considered high quality for one investigated outcome, and low quality for another, due to differences in the outcome measures, analytic strategies, how relevant a certain bias is to the outcome, and how the exposure measure relates to the outcome. The objective of this paper is to illustrate the need for outcome-specific criteria in study evaluation or risk of bias evaluation, describe the process we used to develop the criteria, and summarize the resulting criteria. METHODS: We used a process of expert consultation to develop several sets of outcome-specific criteria to guide study reviewers, improve consistency, and ensure consideration of critical issues specific to the outcomes. The criteria were developed using the following domains: outcome assessment, exposure measurement (specifically timing of exposure in relation to outcome; other exposure measurement issues would be addressed in exposure-specific criteria), participant selection, confounding, analysis, and sensitivity (the study's ability to detect a true effect or hazard). RESULTS: We discuss the application of this process to pregnancy-related outcomes (preterm birth, spontaneous abortion), other reproductive-related outcomes (male reproductive hormones, sperm parameters, time to pregnancy, pubertal development), chronic disease (diabetes, insulin resistance), and acute or episodic conditions (asthma, allergies), and provide examples of the criteria developed. For each outcome the most influential methodological considerations are highlighted including biological sample collection and quality control, sensitivity and specificity of ascertainment tools, optimal timing for recruitment into the study (e.g., preconception, specific trimesters), the etiologically relevant window for exposure assessments, and important potential confounders. CONCLUSIONS: Outcome-specific criteria are an important part of a systematic review and will facilitate study evaluations by epidemiologists with experience in evaluating studies using systematic review methods who may not have extensive discipline-specific experience in the outcomes being reviewed.
INTRODUCTION AND OBJECTIVE: Systematic review tools that provide guidance on evaluating epidemiology studies are receiving increasing attention and support because their application facilitates improved quality of the review, consistency across reviewers, and transparency for readers. The U.S. Environmental Protection Agency's Integrated Risk Information System (IRIS) Program has developed an approach for systematic review of evidence of health effects from chemical exposures that includes structured approaches for literature search and screening, study evaluation, data extraction, and evidence synthesis and integration. This approach recognizes the need for developing outcome-specific criteria for study evaluation. Because studies are assessed at the outcome level, a study could be considered high quality for one investigated outcome, and low quality for another, due to differences in the outcome measures, analytic strategies, how relevant a certain bias is to the outcome, and how the exposure measure relates to the outcome. The objective of this paper is to illustrate the need for outcome-specific criteria in study evaluation or risk of bias evaluation, describe the process we used to develop the criteria, and summarize the resulting criteria. METHODS: We used a process of expert consultation to develop several sets of outcome-specific criteria to guide study reviewers, improve consistency, and ensure consideration of critical issues specific to the outcomes. The criteria were developed using the following domains: outcome assessment, exposure measurement (specifically timing of exposure in relation to outcome; other exposure measurement issues would be addressed in exposure-specific criteria), participant selection, confounding, analysis, and sensitivity (the study's ability to detect a true effect or hazard). RESULTS: We discuss the application of this process to pregnancy-related outcomes (preterm birth, spontaneous abortion), other reproductive-related outcomes (male reproductive hormones, sperm parameters, time to pregnancy, pubertal development), chronic disease (diabetes, insulin resistance), and acute or episodic conditions (asthma, allergies), and provide examples of the criteria developed. For each outcome the most influential methodological considerations are highlighted including biological sample collection and quality control, sensitivity and specificity of ascertainment tools, optimal timing for recruitment into the study (e.g., preconception, specific trimesters), the etiologically relevant window for exposure assessments, and important potential confounders. CONCLUSIONS: Outcome-specific criteria are an important part of a systematic review and will facilitate study evaluations by epidemiologists with experience in evaluating studies using systematic review methods who may not have extensive discipline-specific experience in the outcomes being reviewed.
Authors: Laura M Carlson; Michelle Angrish; Avanti V Shirke; Elizabeth G Radke; Brittany Schulz; Andrew Kraft; Richard Judson; Grace Patlewicz; Robyn Blain; Cynthia Lin; Nicole Vetter; Courtney Lemeris; Pamela Hartman; Heidi Hubbard; Xabier Arzuaga; Allen Davis; Laura V Dishaw; Ingrid L Druwe; Hillary Hollinger; Ryan Jones; J Phillip Kaiser; Lucina Lizarraga; Pamela D Noyes; Michele Taylor; Andrew J Shapiro; Antony J Williams; Kristina A Thayer Journal: Environ Health Perspect Date: 2022-05-17 Impact factor: 11.035
Authors: A Boscutti; G Delvecchio; A Pigoni; G Cereda; V Ciappolino; M Bellani; P Fusar-Poli; P Brambilla Journal: Brain Behav Immun Health Date: 2021-05-18
Authors: Erin E Yost; Audrey Galizia; Dustin F Kapraun; Amanda S Persad; Suryanarayana V Vulimiri; Michelle Angrish; Janice S Lee; Ingrid L Druwe Journal: Environ Health Perspect Date: 2021-07-12 Impact factor: 9.031
Authors: Whitney D Arroyave; Suril S Mehta; Neela Guha; Pam Schwingl; Kyla W Taylor; Barbara Glenn; Elizabeth G Radke; Nadia Vilahur; Tania Carreón; Rebecca M Nachman; Ruth M Lunn Journal: J Expo Sci Environ Epidemiol Date: 2020-05-15 Impact factor: 5.563
Authors: Alessandro Rigolon; Matthew H E M Browning; Olivia McAnirlin; Hyunseo Violet Yoon Journal: Int J Environ Res Public Health Date: 2021-03-04 Impact factor: 3.390