Holger J Schünemann1, Carlos Cuello2, Elie A Akl3, Reem A Mustafa4, Jörg J Meerpohl5, Kris Thayer6, Rebecca L Morgan2, Gerald Gartlehner7, Regina Kunz8, S Vittal Katikireddi9, Jonathan Sterne10, Julian Pt Higgins10, Gordon Guyatt11. 1. Department of Health Research Methods, Evidence, and Impact and McGRADE Center, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S4K1, Canada; Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S4K1, Canada. Electronic address: schuneh@mcmaster.ca. 2. Department of Health Research Methods, Evidence, and Impact and McGRADE Center, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S4K1, Canada. 3. Department of Health Research Methods, Evidence, and Impact and McGRADE Center, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S4K1, Canada; AUB GRADE Center, Clinical Research Institute, American University of Beirut, PO Box 11-0236, Riad El Solh, Beirut, 1107 2020, Lebanon. 4. Department of Health Research Methods, Evidence, and Impact and McGRADE Center, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S4K1, Canada; Department of Medicine, University of Kansas Medical Center, 3901 Rainbow Blvd, MS3002, Kansas City, KS, 66160, USA. 5. Cochrane Germany, Medical Center University of Freiburg, Breisacher Strasse 153, Freiburg, 79110, Germany. 6. Integrated Risk Information System (IRIS) Division, National Center for Environmental Assessment, Environmental Protection Agency, 1200 Pennsylvania Avenue, N.W.Washington, DC 20460, USA. 7. Department for Evidence-Based Medicine and Clinical Epidemiology, Danube University Krems, Dr Karl Dorrek Straße 30, Krems, 3500, Austria. 8. Basel Institute of Clinical Epidemiology, University Hospital Basel, Hebelstrasse 10, Basel, 4031, Switzerland. 9. MRC/CSO Social and Public Health Sciences Unit, University of Glasgow, Top Floor, 200 Renfield Street, Glasgow, G2 3QB, Scotland. 10. Population Health Sciences, Bristol Medical School, University of Bristol, Canynge Hall, 39 Whatley Road, Bristol, BS8 2PS, UK. 11. Department of Health Research Methods, Evidence, and Impact and McGRADE Center, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S4K1, Canada; Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S4K1, Canada.
Abstract
OBJECTIVE: To provide guidance on how systematic review authors, guideline developers, and health technology assessment practitioners should approach the use of the risk of bias in nonrandomized studies of interventions (ROBINS-I) tool as a part of GRADE's certainty rating process. STUDY DESIGN AND SETTING: The study design and setting comprised iterative discussions, testing in systematic reviews, and presentation at GRADE working group meetings with feedback from the GRADE working group. RESULTS: We describe where to start the initial assessment of a body of evidence with the use of ROBINS-I and where one would anticipate the final rating would end up. The GRADE accounted for issues that mitigate concerns about confounding and selection bias by introducing the upgrading domains: large effects, dose-effect relations, and when plausible residual confounders or other biases increase certainty. They will need to be considered in an assessment of a body of evidence when using ROBINS-I. CONCLUSIONS: The use of ROBINS-I in GRADE assessments may allow for a better comparison of evidence from randomized controlled trials (RCTs) and nonrandomized studies (NRSs) because they are placed on a common metric for risk of bias. Challenges remain, including appropriate presentation of evidence from RCTs and NRSs for decision-making and how to optimally integrate RCTs and NRSs in an evidence assessment.
OBJECTIVE: To provide guidance on how systematic review authors, guideline developers, and health technology assessment practitioners should approach the use of the risk of bias in nonrandomized studies of interventions (ROBINS-I) tool as a part of GRADE's certainty rating process. STUDY DESIGN AND SETTING: The study design and setting comprised iterative discussions, testing in systematic reviews, and presentation at GRADE working group meetings with feedback from the GRADE working group. RESULTS: We describe where to start the initial assessment of a body of evidence with the use of ROBINS-I and where one would anticipate the final rating would end up. The GRADE accounted for issues that mitigate concerns about confounding and selection bias by introducing the upgrading domains: large effects, dose-effect relations, and when plausible residual confounders or other biases increase certainty. They will need to be considered in an assessment of a body of evidence when using ROBINS-I. CONCLUSIONS: The use of ROBINS-I in GRADE assessments may allow for a better comparison of evidence from randomized controlled trials (RCTs) and nonrandomized studies (NRSs) because they are placed on a common metric for risk of bias. Challenges remain, including appropriate presentation of evidence from RCTs and NRSs for decision-making and how to optimally integrate RCTs and NRSs in an evidence assessment.
Authors: P J Devereaux; Peter T L Choi; Christina Lacchetti; Bruce Weaver; Holger J Schünemann; Ted Haines; John N Lavis; Brydon J B Grant; David R S Haslam; Mohit Bhandari; Terrence Sullivan; Deborah J Cook; Stephen D Walter; Maureen Meade; Humaira Khan; Neera Bhatnagar; Gordon H Guyatt Journal: CMAJ Date: 2002-05-28 Impact factor: 8.262
Authors: David Atkins; Dana Best; Peter A Briss; Martin Eccles; Yngve Falck-Ytter; Signe Flottorp; Gordon H Guyatt; Robin T Harbour; Margaret C Haugh; David Henry; Suzanne Hill; Roman Jaeschke; Gillian Leng; Alessandro Liberati; Nicola Magrini; James Mason; Philippa Middleton; Jacek Mrukowicz; Dianne O'Connell; Andrew D Oxman; Bob Phillips; Holger J Schünemann; Tessa Tan-Torres Edejer; Helena Varonen; Gunn E Vist; John W Williams; Stephanie Zaza Journal: BMJ Date: 2004-06-19
Authors: Gordon Guyatt; Andrew D Oxman; Elie A Akl; Regina Kunz; Gunn Vist; Jan Brozek; Susan Norris; Yngve Falck-Ytter; Paul Glasziou; Hans DeBeer; Roman Jaeschke; David Rind; Joerg Meerpohl; Philipp Dahm; Holger J Schünemann Journal: J Clin Epidemiol Date: 2010-12-31 Impact factor: 6.437
Authors: Gordon H Guyatt; Andrew D Oxman; Holger J Schünemann; Peter Tugwell; Andre Knottnerus Journal: J Clin Epidemiol Date: 2010-12-24 Impact factor: 6.437
Authors: Holger J Schünemann; Andrew D Oxman; Jan Brozek; Paul Glasziou; Patrick Bossuyt; Stephanie Chang; Paola Muti; Roman Jaeschke; Gordon H Guyatt Journal: ACP J Club Date: 2008-12-16
Authors: Holger J Schünemann; A Holger J Schünemann; Andrew D Oxman; Jan Brozek; Paul Glasziou; Roman Jaeschke; Gunn E Vist; John W Williams; Regina Kunz; Jonathan Craig; Victor M Montori; Patrick Bossuyt; Gordon H Guyatt Journal: BMJ Date: 2008-05-17
Authors: Gordon H Guyatt; Andrew D Oxman; Shahnaz Sultan; Paul Glasziou; Elie A Akl; Pablo Alonso-Coello; David Atkins; Regina Kunz; Jan Brozek; Victor Montori; Roman Jaeschke; David Rind; Philipp Dahm; Joerg Meerpohl; Gunn Vist; Elise Berliner; Susan Norris; Yngve Falck-Ytter; M Hassan Murad; Holger J Schünemann Journal: J Clin Epidemiol Date: 2011-07-30 Impact factor: 6.437
Authors: Holger J Schünemann; Suzanne R Hill; Meetali Kakad; Gunn E Vist; Richard Bellamy; Lauren Stockman; Torbjørn Fosen Wisløff; Chris Del Mar; Frederick Hayden; Timothy M Uyeki; Jeremy Farrar; Yazdan Yazdanpanah; Howard Zucker; John Beigel; Tawee Chotpitayasunondh; Tran Tinh Hien; Bülent Ozbay; Norio Sugaya; Andrew D Oxman Journal: PLoS Med Date: 2007-05 Impact factor: 11.069
Authors: Shahnaz Sultan; Rebecca L Morgan; M Hassan Murad; Yngve Falck-Ytter; Philipp Dahm; Holger J Schünemann; Reem A Mustafa Journal: J Gen Intern Med Date: 2019-11-14 Impact factor: 5.128
Authors: Kim Edmunds; Haitham Tuffaha; Daniel A Galvão; Paul Scuffham; Robert U Newton Journal: Support Care Cancer Date: 2020-01-07 Impact factor: 3.603
Authors: Juan José Yepes-Nuñez; Anita Rajasekhar; Maryam Rahman; Philipp Dahm; David R Anderson; Luis Enrique Colunga-Lozano; Stephanie Ross; Meha Bhatt; Kelly Estrada Orozco; Federico Popoff; Matthew Ventresca; Angela M Barbara; Sara Balduzzi; Housne Begum; Arnav Agarwal; Wojtek Wiercioch; Robby Nieuwlaat; Gian Paolo Morgano; Holger J Schünemann Journal: Blood Adv Date: 2020-06-23
Authors: Sue E Brennan; Steve McDonald; Matthew J Page; Jane Reid; Stephanie Ward; Andrew B Forbes; Joanne E McKenzie Journal: Syst Rev Date: 2020-02-13
Authors: Dena Zeraatkar; Arrti Bhasin; Rita E Morassut; Isabella Churchill; Arnav Gupta; Daeria O Lawson; Anna Miroshnychenko; Emily Sirotich; Komal Aryal; David Mikhail; Tauseef A Khan; Vanessa Ha; John L Sievenpiper; Steven E Hanna; Joseph Beyene; Russell J de Souza Journal: Am J Clin Nutr Date: 2021-06-01 Impact factor: 7.045