Shelly-Anne Li1, Paul E Alexander2, Tea Reljic3, Adam Cuker4, Robby Nieuwlaat2, Wojtek Wiercioch2, Gordon Guyatt2, Holger J Schünemann2, Benjamin Djulbegovic5. 1. Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, 155 College Street, Toronto, Ontario M5T 1P8, Canada. Electronic address: shellyanne.li@mail.utoronto.ca. 2. Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street W, Hamilton, Ontario L8S 4K1, Canada. 3. Comparative Effectiveness Research Program, Morsani College of Medicine, University of South Florida, 3515 E. Fletcher Avenue, MDT 1212, Tampa, FL 33612, USA. 4. Department of Medicine and Department of Pathology & Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, 3400 Civic Center Boulevard, Philadelphia, PA 19104, USA. 5. Department of Supportive Care Medicine, Department of Hematology, City of Hope, 1500 East Duarte Road, Duarte, CA 91010, USA.
Abstract
OBJECTIVES: It is unclear how guidelines panelists discuss and consider factors (criteria) that are formally and not formally included in the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach. To describe the use of decision criteria, we explored how panelists adhered to GRADE criteria and sought to identify any emerging non-GRADE criteria when the panelists used the Evidence to Decision (EtD) framework as part of GRADE application. STUDY DESIGN AND SETTING: We used conventional and summative qualitative analyses to identify themes emerging from face-to-face, panel meeting discussions. Forty-eight members from 12 countries participated in the development of five guidelines for the management of venous thromboembolism by the American Society of Hematology. RESULTS: Ten themes corresponded to the GRADE approach and represented all panel discussions. Over half (53%) of the total panel discussions concerned the use of research evidence. When evidence was considered sufficient and clear, the decision-making process proved rapid. CONCLUSION: The GRADE EtD framework provides structure to guidelines panel meetings, and ensures that the panelists consider all established formal GRADE criteria as they decide on the recommendation text, strength, and direction (for or against an intervention). This is the first study assessing the use of GRADE's EtD framework during real-time guidelines development using panel discussions. Given the widespread use of GRADE, this study provides important information for practice recommendations generated when guidelines panels explicitly follow, in a transparent and systematic manner, the structured GRADE EtD framework. By recognizing the extent to which panels discuss and consider GRADE and other (non-GRADE) criteria for producing guideline recommendations, we are one step closer to understanding the decision-making process in panels that use a structured framework such as the GRADE EtD framework.
OBJECTIVES: It is unclear how guidelines panelists discuss and consider factors (criteria) that are formally and not formally included in the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach. To describe the use of decision criteria, we explored how panelists adhered to GRADE criteria and sought to identify any emerging non-GRADE criteria when the panelists used the Evidence to Decision (EtD) framework as part of GRADE application. STUDY DESIGN AND SETTING: We used conventional and summative qualitative analyses to identify themes emerging from face-to-face, panel meeting discussions. Forty-eight members from 12 countries participated in the development of five guidelines for the management of venous thromboembolism by the American Society of Hematology. RESULTS: Ten themes corresponded to the GRADE approach and represented all panel discussions. Over half (53%) of the total panel discussions concerned the use of research evidence. When evidence was considered sufficient and clear, the decision-making process proved rapid. CONCLUSION: The GRADE EtD framework provides structure to guidelines panel meetings, and ensures that the panelists consider all established formal GRADE criteria as they decide on the recommendation text, strength, and direction (for or against an intervention). This is the first study assessing the use of GRADE's EtD framework during real-time guidelines development using panel discussions. Given the widespread use of GRADE, this study provides important information for practice recommendations generated when guidelines panels explicitly follow, in a transparent and systematic manner, the structured GRADE EtD framework. By recognizing the extent to which panels discuss and consider GRADE and other (non-GRADE) criteria for producing guideline recommendations, we are one step closer to understanding the decision-making process in panels that use a structured framework such as the GRADE EtD framework.
Authors: Amir Qaseem; Frode Forland; Fergus Macbeth; Günter Ollenschläger; Sue Phillips; Philip van der Wees Journal: Ann Intern Med Date: 2012-04-03 Impact factor: 25.391
Authors: Philipp Dahm; Andrew D Oxman; Benjamin Djulbegovic; Gordon H Guyatt; M Hassan Murad; Laura Amato; Elena Parmelli; Marina Davoli; Rebecca L Morgan; Reem A Mustafa; Shahnaz Sultan; Yngve Falck-Ytter; Elie A Akl; Holger J Schünemann Journal: J Clin Epidemiol Date: 2017-04-01 Impact factor: 6.437
Authors: Jeffrey C Andrews; Holger J Schünemann; Andrew D Oxman; Kevin Pottie; Joerg J Meerpohl; Pablo Alonso Coello; David Rind; Victor M Montori; Juan Pablo Brito; Susan Norris; Mahmoud Elbarbary; Piet Post; Mona Nasser; Vijay Shukla; Roman Jaeschke; Jan Brozek; Ben Djulbegovic; Gordon Guyatt Journal: J Clin Epidemiol Date: 2013-04-06 Impact factor: 6.437
Authors: Mia Djulbegovic; Jason Beckstead; Shira Elqayam; Tea Reljic; Ambuj Kumar; Charles Paidas; Benjamin Djulbegovic Journal: PLoS One Date: 2015-08-04 Impact factor: 3.240
Authors: Giovanni Ostuzzi; Davide Papola; Chiara Gastaldon; Georgios Schoretsanitis; Federico Bertolini; Francesco Amaddeo; Alessandro Cuomo; Robin Emsley; Andrea Fagiolini; Giuseppe Imperadore; Taishiro Kishimoto; Giulia Michencigh; Michela Nosé; Marianna Purgato; Serdar Dursun; Brendon Stubbs; David Taylor; Graham Thornicroft; Philip B Ward; Christoph Hiemke; Christoph U Correll; Corrado Barbui Journal: BMC Med Date: 2020-07-15 Impact factor: 8.775
Authors: Hans Van Remoortel; Kari Aranko; Markus M Mueller; Emmy De Buck; Dana Devine; Gilles Folléa; Patrick Meybohm; Pierre Tiberghien; Erica M Wood; Philippe Vandekerckhove; Erhard Seifried Journal: Vox Sang Date: 2019-11-10 Impact factor: 2.144
Authors: Sebastian Hoffmann; Elisa Aiassa; Michelle Angrish; Claire Beausoleil; Frederic Y Bois; Laura Ciccolallo; Peter S Craig; Rob B M De Vries; Jean Lou C M Dorne; Ingrid L Druwe; Stephen W Edwards; Chantra Eskes; Marios Georgiadis; Thomas Hartung; Aude Kienzler; Elisabeth A Kristjansson; Juleen Lam; Laura Martino; Bette Meek; Rebecca L Morgan; Irene Munoz-Guajardo; Pamela D Noyes; Elena Parmelli; Aldert Piersma; Andrew Rooney; Emily Sena; Kristie Sullivan; José Tarazona; Andrea Terron; Kris Thayer; Jan Turner; Jos Verbeek; Didier Verloo; Mathieu Vinken; Sean Watford; Paul Whaley; Daniele Wikoff; Kate Willett; Katya Tsaioun Journal: ALTEX Date: 2022-03-01 Impact factor: 6.250
Authors: Nawar Diar Bakerly; Kirill Nikitin; Neil G Snowise; Glenn Cardwell; Daryl Freeman; Ravijyot Saggu; Anthony De Soyza Journal: BMJ Open Respir Res Date: 2022-09
Authors: Linda Nici; Manoj J Mammen; Edward Charbek; Paul E Alexander; David H Au; Cynthia M Boyd; Gerard J Criner; Gavin C Donaldson; Michael Dreher; Vincent S Fan; Andrea S Gershon; MeiLan K Han; Jerry A Krishnan; Fernando J Martinez; Paula M Meek; Michael Morgan; Michael I Polkey; Milo A Puhan; Mohsen Sadatsafavi; Don D Sin; George R Washko; Jadwiga A Wedzicha; Shawn D Aaron Journal: Am J Respir Crit Care Med Date: 2020-05-01 Impact factor: 21.405
Authors: Bram Rochwerg; Sharon Einav; Dipayan Chaudhuri; Jordi Mancebo; Tommaso Mauri; Yigal Helviz; Ewan C Goligher; Samir Jaber; Jean-Damien Ricard; Nuttapol Rittayamai; Oriol Roca; Massimo Antonelli; Salvatore Maurizio Maggiore; Alexandre Demoule; Carol L Hodgson; Alain Mercat; M Elizabeth Wilcox; David Granton; Dominic Wang; Elie Azoulay; Lamia Ouanes-Besbes; Gilda Cinnella; Michela Rauseo; Carlos Carvalho; Armand Dessap-Mekontso; John Fraser; Jean-Pierre Frat; Charles Gomersall; Giacomo Grasselli; Gonzalo Hernandez; Sameer Jog; Antonio Pesenti; Elisabeth D Riviello; Arthur S Slutsky; Renee D Stapleton; Daniel Talmor; Arnaud W Thille; Laurent Brochard; Karen E A Burns Journal: Intensive Care Med Date: 2020-11-17 Impact factor: 17.440