Thomas Piggott1, Rebecca L Morgan1, Carlos A Cuello-Garcia2, Nancy Santesso1, Reem A Mustafa3, Joerg J Meerpohl4, Holger J Schünemann5. 1. Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada. 2. Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Pediatrics, McMaster University, Hamilton, Ontario, Canada. 3. Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Medicine, University of Kansas Health System, Kansas City, Kansas, USA. 4. Institute for Evidence in Medicine, Medical Center and Faculty of Medicine, University of Freiburg, Freiburg, Germany. 5. Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Medicine, McMaster University, Hamilton, Ontario, Canada. Electronic address: schuneh@mcmaster.ca.
Abstract
OBJECTIVES: The Grading of Recommendations Assessment, Development, and Evaluations (GRADE) system for assessing certainty in a body of evidence currently uses two levels, serious and very serious, for downgrading on a single domain. In the context of newer risk of bias instruments, such as Risk of Bias in Non-Randomized Studies I (ROBINS-I), evidence generated by nonrandomized studies may justify rating down by more than two levels on a single domain. Given the importance users of GRADE assign to terminology, our objective was to assess what term GRADE stakeholders would prefer for rating down certainty by three levels. STUDY DESIGN AND SETTING: We conducted a purposefully sampled online survey of GRADE stakeholders to assess possible terms including "critically serious," "extremely serious," "most serious," and "very, very serious" and conducted a descriptive and thematic analysis of responses. We then facilitated a GRADE working group workshop to generate consensus. RESULTS: A total of 225 respondents ranked and rated "extremely serious" highest, closely followed by "critically serious." Respondents felt that "extremely serious" was "more understandable" and "easiest to interpret". GRADE working group members described that the terms "extremely serious" appeared clearer and easier to translate in other languages. CONCLUSION: Based on this stakeholder-driven study, "extremely serious" is the preferred term to rate down certainty of evidence by three levels in the GRADE approach.
OBJECTIVES: The Grading of Recommendations Assessment, Development, and Evaluations (GRADE) system for assessing certainty in a body of evidence currently uses two levels, serious and very serious, for downgrading on a single domain. In the context of newer risk of bias instruments, such as Risk of Bias in Non-Randomized Studies I (ROBINS-I), evidence generated by nonrandomized studies may justify rating down by more than two levels on a single domain. Given the importance users of GRADE assign to terminology, our objective was to assess what term GRADE stakeholders would prefer for rating down certainty by three levels. STUDY DESIGN AND SETTING: We conducted a purposefully sampled online survey of GRADE stakeholders to assess possible terms including "critically serious," "extremely serious," "most serious," and "very, very serious" and conducted a descriptive and thematic analysis of responses. We then facilitated a GRADE working group workshop to generate consensus. RESULTS: A total of 225 respondents ranked and rated "extremely serious" highest, closely followed by "critically serious." Respondents felt that "extremely serious" was "more understandable" and "easiest to interpret". GRADE working group members described that the terms "extremely serious" appeared clearer and easier to translate in other languages. CONCLUSION: Based on this stakeholder-driven study, "extremely serious" is the preferred term to rate down certainty of evidence by three levels in the GRADE approach.