Yuan Zhang1, Pablo Alonso Coello2, Gordon H Guyatt1, Juan Jose Yepes-Nuñez1, Elie A Akl3, Glen Hazlewood4, Hector Pardo-Hernandez5, Itziar Etxeandia-Ikobaltzeta1, Amir Qaseem6, John W Williams7, Peter Tugwell8, Signe Flottorp9, Yaping Chang1, Yuqing Zhang1, Reem A Mustafa10, María Ximena Rojas11, Feng Xie12, Holger J Schünemann13. 1. Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada. 2. Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada; Iberoamerican Cochrane Centre, CIBERESP-IIB Sant Pau, Barcelona, Spain. 3. Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada; Department of Internal Medicine, Faculty of Medicine, American University of Beirut, Beirut, Lebanon. 4. Department of Medicine and Department of Community Health Sciences, University of Calgary, Calgary, Canada. 5. Iberoamerican Cochrane Centre, CIBERESP-IIB Sant Pau, Barcelona, Spain. 6. American College of Physicians, Philadelphia, PA, USA. 7. Center of Innovation for Health Services Research in Primary Care at the Durham Veterans Affairs Medical Center and Duke University, Durham, NC 27701, USA. 8. Department of Medicine, University of Ottawa, Ottawa, Canada. 9. Norwegian Institute of Public Health, Oslo, Norway; Institute of Health and Society, University of Oslo, Oslo, Norway. 10. Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada; Division of Nephrology and Hypertension, Department of Internal Medicine, University of Kansas Medical Center, Kansas City, KS, USA. 11. Department of Clinical Epidemiology and Biostatistics, Pontificia Universidad Javeriana, Bogotá, Colombia. 12. Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada; Program for Health Economics and Outcome Measures (PHENOM), Hamilton, Canada. 13. Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada; Department of Medicine, McMaster University, Hamilton, Canada. Electronic address: schuneh@mcmaster.ca.
Abstract
OBJECTIVE: To provide Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) guidance for assessing inconsistency, imprecision, and other domains for the certainty of evidence about the relative importance of outcomes. STUDY DESIGN AND SETTING: We applied the GRADE domains to rate the certainty of evidence in the importance of outcomes to several systematic reviews, iteratively reviewed draft guidance, and consulted GRADE members and other stakeholders for feedback. RESULTS: We describe the rationale for considering the remaining GRADE domains when rating the certainty in a body of evidence for the relative importance of outcomes. As meta-analyses are not common in this context, inconsistency and imprecision assessments are challenging. Furthermore, confusion exists about inconsistency, imprecision, and true variability in the relative importance of outcomes. To clarify this issue, we suggest that the true variability is neither equivalent to inconsistency nor imprecision. Specifically, inconsistency arises from population, intervention, comparison and outcome and methodological elements that should be explored and, if possible, explained. The width of the confidence interval and sample size inform judgments about imprecision. We also provide suggestions on how to detect publication bias and discuss the domains to rate up the certainty. CONCLUSION: We provide guidance and examples for rating inconsistency, imprecision, and other domains for a body of evidence describing the relative importance of outcomes.
OBJECTIVE: To provide Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) guidance for assessing inconsistency, imprecision, and other domains for the certainty of evidence about the relative importance of outcomes. STUDY DESIGN AND SETTING: We applied the GRADE domains to rate the certainty of evidence in the importance of outcomes to several systematic reviews, iteratively reviewed draft guidance, and consulted GRADE members and other stakeholders for feedback. RESULTS: We describe the rationale for considering the remaining GRADE domains when rating the certainty in a body of evidence for the relative importance of outcomes. As meta-analyses are not common in this context, inconsistency and imprecision assessments are challenging. Furthermore, confusion exists about inconsistency, imprecision, and true variability in the relative importance of outcomes. To clarify this issue, we suggest that the true variability is neither equivalent to inconsistency nor imprecision. Specifically, inconsistency arises from population, intervention, comparison and outcome and methodological elements that should be explored and, if possible, explained. The width of the confidence interval and sample size inform judgments about imprecision. We also provide suggestions on how to detect publication bias and discuss the domains to rate up the certainty. CONCLUSION: We provide guidance and examples for rating inconsistency, imprecision, and other domains for a body of evidence describing the relative importance of outcomes.
Authors: Shahnaz Sultan; Rebecca L Morgan; M Hassan Murad; Yngve Falck-Ytter; Philipp Dahm; Holger J Schünemann; Reem A Mustafa Journal: J Gen Intern Med Date: 2019-11-14 Impact factor: 5.128
Authors: Ariel Izcovich; Adam Cuker; Robert Kunkle; Ignacio Neumann; Julie Panepinto; Menaka Pai; Matthew Seftel; Matthew C Cheung; Richard Lottenberg; Michael Byrne; Robert Plovnick; Deirdra Terrell; Jennifer L Holter-Chakrabarty; Benjamin Djulbegovic; Lisa K Hicks; Wojtek Wiercioch; Robby Nieuwlaat; Holger J Schünemann Journal: Blood Adv Date: 2020-05-12
Authors: Jan L Brozek; Carlos Canelo-Aybar; Elie A Akl; James M Bowen; John Bucher; Weihsueh A Chiu; Mark Cronin; Benjamin Djulbegovic; Maicon Falavigna; Gordon H Guyatt; Ami A Gordon; Michele Hilton Boon; Raymond C W Hutubessy; Manuela A Joore; Vittal Katikireddi; Judy LaKind; Miranda Langendam; Veena Manja; Kristen Magnuson; Alexander G Mathioudakis; Joerg Meerpohl; Dominik Mertz; Roman Mezencev; Rebecca Morgan; Gian Paolo Morgano; Reem Mustafa; Martin O'Flaherty; Grace Patlewicz; John J Riva; Margarita Posso; Andrew Rooney; Paul M Schlosser; Lisa Schwartz; Ian Shemilt; Jean-Eric Tarride; Kristina A Thayer; Katya Tsaioun; Luke Vale; John Wambaugh; Jessica Wignall; Ashley Williams; Feng Xie; Yuan Zhang; Holger J Schünemann Journal: J Clin Epidemiol Date: 2020-09-24 Impact factor: 6.437
Authors: Richard Macey; Tanya Walsh; Philip Riley; Anne-Marie Glenny; Helen V Worthington; Lucy O'Malley; Janet E Clarkson; David Ricketts Journal: Cochrane Database Syst Rev Date: 2021-06-14