Nancy D Berkman1, Kathleen N Lohr2, Mohammed T Ansari3, Ethan M Balk4, Robert Kane5, Marian McDonagh6, Sally C Morton7, Meera Viswanathan2, Eric B Bass8, Mary Butler5, Gerald Gartlehner9, Lisa Hartling10, Melissa McPheeters11, Laura C Morgan2, James Reston12, Priyanka Sista13, Evelyn Whitlock14, Stephanie Chang15. 1. Division of Social Policy, Health & Economics Research, RTI International (Research Triangle Institute), PO Box 12194, 3040 Cornwallis Road, Research Triangle Park, NC 27709, USA. Electronic address: berkman@rti.org. 2. Division of Social Policy, Health & Economics Research, RTI International (Research Triangle Institute), PO Box 12194, 3040 Cornwallis Road, Research Triangle Park, NC 27709, USA. 3. Clinical Epidemiology Program, Ottawa Hospital Research Institute, Box 201B, Ottawa Hospital - General campus, 501 Smyth Road, Ottawa, ON K1H 8L6, Canada. 4. Center for Evidence-Based Medicine, Brown University, School of Public Health, Box G-S121-8, Providence, RI 02912, USA. 5. University of Minnesota, School of Public Health, 420 Delaware St SE, MMC 197, Minneapolis, MN 55455, USA. 6. Pacific Northwest Evidence-based Practice Center, Oregon Health Science University, 3181 SW Sam Jackson Park Road, Portland, OR 97239, USA. 7. Department of Biostatistics, Graduate School of Public Health, University of Pittsburgh, 130 DeSoto Street, Pittsburg, PA 15261, USA. 8. Department of Medicine, and Department of Health Policy and Management, Johns Hopkins University, 624 North Broadway, Room 680A, Baltimore, MD, 21205, USA. 9. Division of Social Policy, Health & Economics Research, RTI International (Research Triangle Institute), PO Box 12194, 3040 Cornwallis Road, Research Triangle Park, NC 27709, USA; Department for Clinical Epidemiology and Evidence-based Medicine, Danube University, Dr. Karl Dorrek Strasse 30, 3500 Krems, Austria. 10. Department of Pediatrics, University of Alberta, ECHA 4-472, 11405-87 Avenue, Edmonton, Alberta, T6G 1C9, Canada. 11. Department of Health Policy, Vanderbilt University, Medical Center, 6th Floor, 2525 West End Avenue, Nashville, TN 37203, USA. 12. Evidence-Based Practice Center and Health Technology Assessment Group, ECRI Institute Headquarters, 5200 Butler Pike, Plymouth Meeting, PA 19462, USA. 13. University of North Carolina School of Medicine, 1001 Bondurant Hall, CB# 9535, Chapel Hill, NC 27599, USA. 14. Center for Health Research, Kaiser Permanente, Northwest, 3800 N. Interstate Avenue, Portland, OR 97227, USA. 15. Center for Evidence and Practice Improvement, Agency for Healthcare Research and Quality, 540 Gaither Road, Rockville, MD 20850, USA.
Abstract
OBJECTIVES: To revise 2010 guidance on grading the strength of evidence (SOE) of the effectiveness of drugs, devices, and other preventive and therapeutic interventions in systematic reviews produced by the Evidence-based Practice Center (EPC) program, established by the US Agency for Healthcare Research and Quality (AHRQ). STUDY DESIGN AND SETTING: A cross-EPC working group reviewed authoritative systems for grading SOE [primarily the approach from the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group] and conducted extensive discussions with GRADE and other experts. RESULTS: Updated guidance continues to be conceptually similar to GRADE. Reviewers are to evaluate SOE separately for each major treatment comparison for each major outcome. We added reporting bias as a required domain and retained study limitations (risk of bias), consistency, directness, and precision (and three optional domains). Additional guidance covers scoring consistency, precision, and reporting bias, grading bodies of evidence with randomized controlled trials and observational studies, evaluating single study bodies of evidence, using studies with high risk of bias, and presenting findings with greater clarity and transparency. SOE is graded high, moderate, low, or insufficient, reflecting reviewers' confidence in the findings for a specific treatment comparison and outcome. CONCLUSION: No single approach for grading SOE suits all reviews, but a more consistent and transparent approach to reporting summary information will make reviews more useful to the broad range of audiences that AHRQ's work aims to reach. EPC working groups will consider ongoing challenges and modify guidance as needed, on issues such as combining trials and observational studies in bodies of evidence, weighting domains, and combining qualitative and quantitative syntheses.
OBJECTIVES: To revise 2010 guidance on grading the strength of evidence (SOE) of the effectiveness of drugs, devices, and other preventive and therapeutic interventions in systematic reviews produced by the Evidence-based Practice Center (EPC) program, established by the US Agency for Healthcare Research and Quality (AHRQ). STUDY DESIGN AND SETTING: A cross-EPC working group reviewed authoritative systems for grading SOE [primarily the approach from the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group] and conducted extensive discussions with GRADE and other experts. RESULTS: Updated guidance continues to be conceptually similar to GRADE. Reviewers are to evaluate SOE separately for each major treatment comparison for each major outcome. We added reporting bias as a required domain and retained study limitations (risk of bias), consistency, directness, and precision (and three optional domains). Additional guidance covers scoring consistency, precision, and reporting bias, grading bodies of evidence with randomized controlled trials and observational studies, evaluating single study bodies of evidence, using studies with high risk of bias, and presenting findings with greater clarity and transparency. SOE is graded high, moderate, low, or insufficient, reflecting reviewers' confidence in the findings for a specific treatment comparison and outcome. CONCLUSION: No single approach for grading SOE suits all reviews, but a more consistent and transparent approach to reporting summary information will make reviews more useful to the broad range of audiences that AHRQ's work aims to reach. EPC working groups will consider ongoing challenges and modify guidance as needed, on issues such as combining trials and observational studies in bodies of evidence, weighting domains, and combining qualitative and quantitative syntheses.
Keywords:
Clinical practice guidelines; Evidence-based practice; Health care delivery; Health policy; Methods; Minimally important differences; Optimal information size; Strength of evidence; Systematic reviews
Authors: Akshay Sharma; Sherif M Badawy; Elizabeth M Suelzer; Hemant S Murthy; Pinki Prasad; Hesham Eissa; Paul A Carpenter; Mehdi Hamadani; Myriam Labopin; Hélène Schoemans; André Tichelli; Rachel Phelan; Betty K Hamilton; David Buchbinder; Annie Im; Rebecca Hunter; Ruta Brazauskas; Linda J Burns Journal: Bone Marrow Transplant Date: 2021-01-29 Impact factor: 5.483
Authors: Elizabeth A Yetley; Amanda J MacFarlane; Linda S Greene-Finestone; Cutberto Garza; Jamy D Ard; Stephanie A Atkinson; Dennis M Bier; Alicia L Carriquiry; William R Harlan; Dale Hattis; Janet C King; Daniel Krewski; Deborah L O'Connor; Ross L Prentice; Joseph V Rodricks; George A Wells Journal: Am J Clin Nutr Date: 2016-12-07 Impact factor: 7.045
Authors: Nila Sathe; Sivakumar Chinnadurai; Melissa McPheeters; David O Francis Journal: Otolaryngol Head Neck Surg Date: 2017-01-17 Impact factor: 3.497
Authors: Sivakumar Chinnadurai; Atia K Jordan; Nila A Sathe; Christopher Fonnesbeck; Melissa L McPheeters; David O Francis Journal: Pediatrics Date: 2017-01-17 Impact factor: 7.124
Authors: David O Francis; Christopher Fonnesbeck; Nila Sathe; Melissa McPheeters; Shanthi Krishnaswami; Sivakumar Chinnadurai Journal: Otolaryngol Head Neck Surg Date: 2017-01-17 Impact factor: 3.497
Authors: Anna Morad; Nila A Sathe; David O Francis; Melissa L McPheeters; Sivakumar Chinnadurai Journal: Pediatrics Date: 2017-01-17 Impact factor: 7.124
Authors: M Alabousi; A Alabousi; T A McGrath; K D Cobey; B Budhram; R A Frank; F Nguyen; J P Salameh; A Dehmoobad Sharifabadi; M D F McInnes Journal: Eur Radiol Date: 2018-07-26 Impact factor: 5.315
Authors: Kimberly A Brownley; Nancy D Berkman; Christine M Peat; Kathleen N Lohr; Katherine E Cullen; Carla M Bann; Cynthia M Bulik Journal: Ann Intern Med Date: 2016-06-28 Impact factor: 25.391