Paul E Alexander1, Michael R Gionfriddo2, Shelly-Anne Li3, Lisa Bero4, Rebecca J Stoltzfus5, Ignacio Neumann6, Juan P Brito7, Benjamin Djulbegovic8, Victor M Montori9, Susan L Norris10, Holger J Schünemann11, Lehana Thabane12, Gordon H Guyatt13. 1. Health Research Methods (HRM), Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 3Z5, Canada. Electronic address: elias98_99@yahoo.com. 2. Mayo Graduate School, Mayo Clinic, Plummer 3-35, 200 First Street SW, Rochester, MN 55905, USA; Knowledge and Evaluation Research Unit, Mayo Clinic, Plummer 3-35, 200 First Street SW, Rochester, MN 55905, USA. 3. Health Research Methods (HRM), Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 3Z5, Canada. 4. Charles Perkins Centre, The University of Sydney, Camperdown, New South Wales 2006, Australia. 5. Division of Nutritional Sciences, Cornell University, 120 Savage Hall, Ithaca, NY 14853, USA. 6. Health Research Methods (HRM), Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8N 3Z5, Canada; Department of Internal Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile. 7. Knowledge and Evaluation Research Unit, Division of Endocrinology, Diabetes, Metabolism, and Nutrition, Mayo Clinic, Plummer 3-35, 200 First Street SW, Rochester, MN 55905, USA. 8. H Lee Moffitt Cancer Center, University of South Florida, FL, USA. 9. Knowledge and Evaluation Research Unit, Division of Endocrinology and Diabetes, Mayo Clinic, Plummer 4-402, 200 First Street SW, Rochester, MN 55905, USA; Knowledge and Evaluation Research Unit, Health Care Policy and Research, Mayo Clinic, Plummer 4-402, 200 First Street SW, Rochester, MN 55905, USA. 10. World Health Organization, Av. Appia 20, CH-1211, Geneva 27, Switzerland. 11. Department of Clinical Epidemiology & Biostatistics, McMaster University Health Sciences Centre, Room 2C16, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada. 12. Biostatistics Unit, Centre for Evaluation of Medicine, Department of Clinical Epidemiology & Biostatistics, PHRI, Hamilton Health Sciences, McMaster University, St. Joseph's Healthcare Hamilton, 3rd Floor, Martha Wing, Room H-325, 50 Charlton Avenue East, Hamilton, Ontario L8N 4A6, Canada. 13. Department of Clinical Epidemiology & Biostatistics, McMaster University Health Sciences Centre, Room 2C12, 1200 Main Street West, Hamilton, Ontario L8S 4K1, Canada.
Abstract
OBJECTIVE: Many strong recommendations issued by the World Health Organization (WHO) are based on low- or very low-quality (low certainty) evidence (discordant recommendations). Many such discordant recommendations are inconsistent with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance. We sought to understand why WHO makes discordant recommendations inconsistent with GRADE guidance. STUDY DESIGN AND SETTING: We interviewed panel members involved in guidelines approved by WHO (2007-2012) that included discordant recommendations. Interviews, recorded and transcribed, focused on use of GRADE including the reasoning underlying, and factors contributing to, discordant recommendations. RESULTS: Four themes emerged: strengths of GRADE, challenges and barriers to GRADE, strategies to improve GRADE application, and explanations for discordant recommendations. Reasons for discordant recommendations included skepticism about the value of making conditional recommendations; political considerations; high certainty in benefits (sometimes warranted, sometimes not) despite assessing evidence as low certainty; and concerns that conditional recommendations will be ignored. CONCLUSION: WHO panelists make discordant recommendations inconsistent with GRADE guidance for reasons that include limitations in their understanding of GRADE. Ensuring optimal application of GRADE at WHO and elsewhere likely requires selecting panelists who have a commitment to GRADE principles, additional training of panelists, and formal processes to maximize adherence to GRADE principles.
OBJECTIVE: Many strong recommendations issued by the World Health Organization (WHO) are based on low- or very low-quality (low certainty) evidence (discordant recommendations). Many such discordant recommendations are inconsistent with the Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance. We sought to understand why WHO makes discordant recommendations inconsistent with GRADE guidance. STUDY DESIGN AND SETTING: We interviewed panel members involved in guidelines approved by WHO (2007-2012) that included discordant recommendations. Interviews, recorded and transcribed, focused on use of GRADE including the reasoning underlying, and factors contributing to, discordant recommendations. RESULTS: Four themes emerged: strengths of GRADE, challenges and barriers to GRADE, strategies to improve GRADE application, and explanations for discordant recommendations. Reasons for discordant recommendations included skepticism about the value of making conditional recommendations; political considerations; high certainty in benefits (sometimes warranted, sometimes not) despite assessing evidence as low certainty; and concerns that conditional recommendations will be ignored. CONCLUSION: WHO panelists make discordant recommendations inconsistent with GRADE guidance for reasons that include limitations in their understanding of GRADE. Ensuring optimal application of GRADE at WHO and elsewhere likely requires selecting panelists who have a commitment to GRADE principles, additional training of panelists, and formal processes to maximize adherence to GRADE principles.
Authors: Scott L Weiss; Mark J Peters; Waleed Alhazzani; Michael S D Agus; Heidi R Flori; David P Inwald; Simon Nadel; Luregn J Schlapbach; Robert C Tasker; Andrew C Argent; Joe Brierley; Joseph Carcillo; Enitan D Carrol; Christopher L Carroll; Ira M Cheifetz; Karen Choong; Jeffry J Cies; Andrea T Cruz; Daniele De Luca; Akash Deep; Saul N Faust; Claudio Flauzino De Oliveira; Mark W Hall; Paul Ishimine; Etienne Javouhey; Koen F M Joosten; Poonam Joshi; Oliver Karam; Martin C J Kneyber; Joris Lemson; Graeme MacLaren; Nilesh M Mehta; Morten Hylander Møller; Christopher J L Newth; Trung C Nguyen; Akira Nishisaki; Mark E Nunnally; Margaret M Parker; Raina M Paul; Adrienne G Randolph; Suchitra Ranjit; Lewis H Romer; Halden F Scott; Lyvonne N Tume; Judy T Verger; Eric A Williams; Joshua Wolf; Hector R Wong; Jerry J Zimmerman; Niranjan Kissoon; Pierre Tissieres Journal: Intensive Care Med Date: 2020-02 Impact factor: 17.440
Authors: Scott L Weiss; Mark J Peters; Waleed Alhazzani; Michael S D Agus; Heidi R Flori; David P Inwald; Simon Nadel; Luregn J Schlapbach; Robert C Tasker; Andrew C Argent; Joe Brierley; Joseph Carcillo; Enitan D Carrol; Christopher L Carroll; Ira M Cheifetz; Karen Choong; Jeffry J Cies; Andrea T Cruz; Daniele De Luca; Akash Deep; Saul N Faust; Claudio Flauzino De Oliveira; Mark W Hall; Paul Ishimine; Etienne Javouhey; Koen F M Joosten; Poonam Joshi; Oliver Karam; Martin C J Kneyber; Joris Lemson; Graeme MacLaren; Nilesh M Mehta; Morten Hylander Møller; Christopher J L Newth; Trung C Nguyen; Akira Nishisaki; Mark E Nunnally; Margaret M Parker; Raina M Paul; Adrienne G Randolph; Suchitra Ranjit; Lewis H Romer; Halden F Scott; Lyvonne N Tume; Judy T Verger; Eric A Williams; Joshua Wolf; Hector R Wong; Jerry J Zimmerman; Niranjan Kissoon; Pierre Tissieres Journal: Intensive Care Med Date: 2020-02 Impact factor: 17.440
Authors: Jennifer A Frontera; John J Lewin; Alejandro A Rabinstein; Imo P Aisiku; Anne W Alexandrov; Aaron M Cook; Gregory J del Zoppo; Monisha A Kumar; Ellinor I B Peerschke; Michael F Stiefel; Jeanne S Teitelbaum; Katja E Wartenberg; Cindy L Zerfoss Journal: Neurocrit Care Date: 2016-02 Impact factor: 3.210
Authors: Robert I Liem; Sophie Lanzkron; Thomas D Coates; Laura DeCastro; Ankit A Desai; Kenneth I Ataga; Robyn T Cohen; Johnson Haynes; Ifeyinwa Osunkwo; Jeffrey D Lebensburger; James P Lash; Theodore Wun; Madeleine Verhovsek; Elodie Ontala; Rae Blaylark; Fares Alahdab; Abdulrahman Katabi; Reem A Mustafa Journal: Blood Adv Date: 2019-12-10
Authors: Paula D James; Nathan T Connell; Barbara Ameer; Jorge Di Paola; Jeroen Eikenboom; Nicolas Giraud; Sandra Haberichter; Vicki Jacobs-Pratt; Barbara Konkle; Claire McLintock; Simon McRae; Robert R Montgomery; James S O'Donnell; Nikole Scappe; Robert Sidonio; Veronica H Flood; Nedaa Husainat; Mohamad A Kalot; Reem A Mustafa Journal: Blood Adv Date: 2021-01-12
Authors: Liana Fraenkel; Joan M Bathon; Bryant R England; E William St Clair; Thurayya Arayssi; Kristine Carandang; Kevin D Deane; Mark Genovese; Kent Kwas Huston; Gail Kerr; Joel Kremer; Mary C Nakamura; Linda A Russell; Jasvinder A Singh; Benjamin J Smith; Jeffrey A Sparks; Shilpa Venkatachalam; Michael E Weinblatt; Mounir Al-Gibbawi; Joshua F Baker; Kamil E Barbour; Jennifer L Barton; Laura Cappelli; Fatimah Chamseddine; Michael George; Sindhu R Johnson; Lara Kahale; Basil S Karam; Assem M Khamis; Iris Navarro-Millán; Reza Mirza; Pascale Schwab; Namrata Singh; Marat Turgunbaev; Amy S Turner; Sally Yaacoub; Elie A Akl Journal: Arthritis Care Res (Hoboken) Date: 2021-06-08 Impact factor: 5.178
Authors: Maria Cecilia Dedios; Alexo Esperato; Luz Maria De-Regil; Juan Pablo Peña-Rosas; Susan L Norris Journal: Implement Sci Date: 2017-03-21 Impact factor: 7.327
Authors: Ein Soon Shin; Ji Eun Jang; Ji Yun Yeon; Da Sol Kim; Sung Goo Chang; Yoon Seong Lee Journal: J Korean Med Sci Date: 2018-02-19 Impact factor: 2.153
Authors: Thomas Agoritsas; Arnaud Merglen; Anja Fog Heen; Annette Kristiansen; Ignacio Neumann; Juan P Brito; Romina Brignardello-Petersen; Paul E Alexander; David M Rind; Per O Vandvik; Gordon H Guyatt Journal: BMJ Open Date: 2017-11-16 Impact factor: 2.692