Yuan Zhang1, Pablo Alonso-Coello2, Gordon H Guyatt1, Juan José Yepes-Nuñez1, Elie A Akl3, Glen Hazlewood4, Hector Pardo-Hernandez5, Itziar Etxeandia-Ikobaltzeta1, Amir Qaseem6, John W Williams7, Peter Tugwell8, Signe Flottorp9, Yaping Chang1, Yuqing Zhang1, Reem A Mustafa10, María Ximena Rojas11, Holger J Schünemann12. 1. Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada. 2. Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada; Centro Cochrane Iberoamericano, Instituto de Investigacion Biomedica (IIB Sant Pau-CIBERESP), Sant Antoni Maria Claret 167, 08025 Barcelona, Spain. Electronic address: PAlonso@santpau.cat. 3. Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada; Department of Internal Medicine, Faculty of Medicine, American University of Beirut, Beirut, Lebanon. 4. Department of Medicine and Department of Community Health Sciences, University of Calgary, Calgary, Alberta, Canada. 5. Centro Cochrane Iberoamericano, Instituto de Investigacion Biomedica (IIB Sant Pau-CIBERESP), Sant Antoni Maria Claret 167, 08025 Barcelona, Spain. 6. Department of Clinical Policy, American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106, USA. 7. Center of Innovation for Health Services Research in Primary Care, Durham Veterans Affairs Medical Center and Duke University, Durham, NC 27701, USA. 8. Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada. 9. Division for Health Services, Norwegian Institute of Public Health, Oslo, Norway; Department of Health Management and Health Economics, Institute of Health and Society, University of Oslo, Oslo, Norway. 10. Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada; Department of Internal Medicine, Division of Nephrology and Hypertension, University of Kansas Medical Center, Kansas City, KS, USA. 11. Department of Clinical Epidemiology and Biostatistics, Pontificia Universidad Javeriana, Bogotá, Colombia. 12. Department of Health Research Methods, Evidence, and Impact & McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 4K1, Canada; Department of Medicine, McMaster University, Hamilton, Ontario, Canada. Electronic address: schuneh@mcmaster.ca.
Abstract
OBJECTIVES: The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) working group defines patient values and preferences as the relative importance patients place on the main health outcomes. We provide GRADE guidance for assessing the risk of bias and indirectness domains for certainty of evidence about the relative importance of outcomes. STUDY DESIGN AND SETTING: We applied the GRADE domains to rate the certainty of evidence in the importance of outcomes to several systematic reviews, iteratively reviewed draft guidance and consulted GRADE members and other stakeholders for feedback. RESULTS: This is the first of two articles. A body of evidence addressing the importance of outcomes starts at "high certainty"; concerns with risk of bias, indirectness, inconsistency, imprecision, and publication bias lead to downgrading to moderate, low, or very low certainty. We propose subdomains of risk of bias as selection of the study population, missing data, the type of measurement instrument, and confounding; we have developed items for each subdomain. The population, intervention, comparison, and outcome elements associated with the evidence determine the degree of indirectness. CONCLUSION: This article provides guidance and examples for rating the risk of bias and indirectness for a body of evidence summarizing the importance of outcomes.
OBJECTIVES: The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) working group defines patient values and preferences as the relative importance patients place on the main health outcomes. We provide GRADE guidance for assessing the risk of bias and indirectness domains for certainty of evidence about the relative importance of outcomes. STUDY DESIGN AND SETTING: We applied the GRADE domains to rate the certainty of evidence in the importance of outcomes to several systematic reviews, iteratively reviewed draft guidance and consulted GRADE members and other stakeholders for feedback. RESULTS: This is the first of two articles. A body of evidence addressing the importance of outcomes starts at "high certainty"; concerns with risk of bias, indirectness, inconsistency, imprecision, and publication bias lead to downgrading to moderate, low, or very low certainty. We propose subdomains of risk of bias as selection of the study population, missing data, the type of measurement instrument, and confounding; we have developed items for each subdomain. The population, intervention, comparison, and outcome elements associated with the evidence determine the degree of indirectness. CONCLUSION: This article provides guidance and examples for rating the risk of bias and indirectness for a body of evidence summarizing the importance of outcomes.
Authors: Shahnaz Sultan; Rebecca L Morgan; M Hassan Murad; Yngve Falck-Ytter; Philipp Dahm; Holger J Schünemann; Reem A Mustafa Journal: J Gen Intern Med Date: 2019-11-14 Impact factor: 5.128
Authors: Ariel Izcovich; Adam Cuker; Robert Kunkle; Ignacio Neumann; Julie Panepinto; Menaka Pai; Matthew Seftel; Matthew C Cheung; Richard Lottenberg; Michael Byrne; Robert Plovnick; Deirdra Terrell; Jennifer L Holter-Chakrabarty; Benjamin Djulbegovic; Lisa K Hicks; Wojtek Wiercioch; Robby Nieuwlaat; Holger J Schünemann Journal: Blood Adv Date: 2020-05-12
Authors: M Hassan Murad; Robert I Liem; Eddy S Lang; Elie A Akl; Joerg J Meerpohl; Michael R DeBaun; John F Tisdale; Amanda M Brandow; Sophie M Lanzkron; Stella T Chou; Starr Webb; Reem A Mustafa Journal: Blood Adv Date: 2019-12-10
Authors: Jan L Brozek; Carlos Canelo-Aybar; Elie A Akl; James M Bowen; John Bucher; Weihsueh A Chiu; Mark Cronin; Benjamin Djulbegovic; Maicon Falavigna; Gordon H Guyatt; Ami A Gordon; Michele Hilton Boon; Raymond C W Hutubessy; Manuela A Joore; Vittal Katikireddi; Judy LaKind; Miranda Langendam; Veena Manja; Kristen Magnuson; Alexander G Mathioudakis; Joerg Meerpohl; Dominik Mertz; Roman Mezencev; Rebecca Morgan; Gian Paolo Morgano; Reem Mustafa; Martin O'Flaherty; Grace Patlewicz; John J Riva; Margarita Posso; Andrew Rooney; Paul M Schlosser; Lisa Schwartz; Ian Shemilt; Jean-Eric Tarride; Kristina A Thayer; Katya Tsaioun; Luke Vale; John Wambaugh; Jessica Wignall; Ashley Williams; Feng Xie; Yuan Zhang; Holger J Schünemann Journal: J Clin Epidemiol Date: 2020-09-24 Impact factor: 6.437
Authors: José Gerardo González-González; Alejandro Díaz González-Colmenero; Juan Manuel Millán-Alanís; Lyubov Lytvyn; Ricardo Cesar Solis; Reem A Mustafa; Suetonia C Palmer; Sheyu Li; Qiukui Hao; Neri Alejandro Alvarez-Villalobos; Per Olav Vandvik; René Rodríguez-Gutiérrez Journal: BMJ Open Date: 2021-07-09 Impact factor: 2.692
Authors: Ryan O'Reilly; Sayako Yokoyama; Justin Boyle; Jeffrey C Kwong; Allison McGeer; Teresa To; Beate Sander Journal: Qual Life Res Date: 2021-07-17 Impact factor: 4.147