Paul E Alexander1, Juan P Brito2, Ignacio Neumann3, Michael R Gionfriddo4, Lisa Bero5, Benjamin Djulbegovic6, Rebecca Stoltzfus7, Victor M Montori8, Susan L Norris9, Holger J Schünemann10, Gordon H Guyatt11. 1. Health Research Methods (HRM), Department of Clinical Epidemiology and Biostatistics, McMaster University, Health Sciences Building (HSB), 1280 Main Street West, Hamilton, Ontario L8N 3Z5, Canada. Electronic address: elias98_99@yahoo.com. 2. Knowledge and Evaluation Research Unit, Divisions of Endocrinology, Diabetes, Metabolism and Nutrition, Mayo Clinic, Plummer 3-35, 200 First Street SW, Rochester, MN 55905, USA. 3. Health Research Methods (HRM), Department of Clinical Epidemiology and Biostatistics, McMaster University, Health Sciences Building (HSB), 1280 Main Street West, Hamilton, Ontario L8N 3Z5, Canada; Department of Internal Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile. 4. Mayo Graduate School, Mayo Clinic, Knowledge and Evaluation Research Unit, Mayo Clinic, Plummer 3-35, 200 First Street SW, Rochester, MN 55905, USA. 5. Charles Perkins Centre, The University of Sydney, Camperdown, NSW 2006, Australia. 6. H Lee Moffitt Cancer Center, University of South Florida, FL, USA. 7. Global Health Program, Program in International Nutrition, Division of Nutritional Sciences, 120 Savage Hall, Cornell University, Ithaca, NY 14853, USA. 8. Healthcare Delivery Research Program, Knowledge and Evaluation Research Unit, Divisions of Endocrinology and Diabetes and Health Care and Policy Research, Mayo Clinic, Plummer 3-35, 200 First Street SW, Rochester, MN 55905, USA. 9. Guidelines Review Committee Secretariat World Health Association (WHO), Geneva, Switzerland. 10. Department of Clinical Epidemiology & Biostatistics, McMaster University Health Sciences Centre, Room 2C16, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada. 11. Department of Clinical Epidemiology & Biostatistics, McMaster University Health Sciences Centre, 1200 Main Street West, Room 2C12, Hamilton, Ontario L8S 4K1, Canada.
Abstract
OBJECTIVES: In 2007 the World Health Organization (WHO) adopted the GRADE system for development of public health guidelines. Previously we found that many strong recommendations issued by WHO are based on evidence for which there is only low or very low confidence in the estimates of effect (discordant recommendations). GRADE guidance indicates that such discordant recommendations are rarely appropriate but suggests five paradigmatic situations in which discordant recommendations may be warranted. We sought to provide insight into the many discordant recommendations in WHO guidelines. STUDY DESIGN AND SETTING: We examined all guidelines that used the GRADE method and were approved by the WHO Guideline Review Committee between 2007 and 2012. Teams of reviewers independently abstracted data from eligible guidelines and classified recommendations either into one of the five paradigms for appropriately-formulated discordant recommendations or into three additional categories in which discordant recommendations were inconsistent with GRADE guidance: 1) the evidence warranted moderate or high confidence (a misclassification of evidence) rather than low or very low confidence; 2) good practice statements; or 3) uncertainty in the estimates of effect would best lead to a conditional (weak) recommendation. RESULTS: The 33 eligible guidelines included 160 discordant recommendations, of which 98 (61.3%) addressed drug interventions and 132 (82.5%) provided some rationale (though not entirely explicit at times) for the strong recommendation. Of 160 discordant recommendations, 25 (15.6%) were judged consistent with one of the five paradigms for appropriate recommendations; 33 (21%) were based on evidence warranting moderate or high confidence in the estimates of effect; 29 (18%) were good practice statements; and 73 (46%) warranted a conditional, rather than a strong recommendation. CONCLUSION: WHO discordant recommendations are often inconsistent with GRADE guidance, possibly threatening the integrity of the process. Further training in GRADE methods for WHO guideline development group members may be necessary, along with further research on what motivates the formulation of such recommendations.
OBJECTIVES: In 2007 the World Health Organization (WHO) adopted the GRADE system for development of public health guidelines. Previously we found that many strong recommendations issued by WHO are based on evidence for which there is only low or very low confidence in the estimates of effect (discordant recommendations). GRADE guidance indicates that such discordant recommendations are rarely appropriate but suggests five paradigmatic situations in which discordant recommendations may be warranted. We sought to provide insight into the many discordant recommendations in WHO guidelines. STUDY DESIGN AND SETTING: We examined all guidelines that used the GRADE method and were approved by the WHO Guideline Review Committee between 2007 and 2012. Teams of reviewers independently abstracted data from eligible guidelines and classified recommendations either into one of the five paradigms for appropriately-formulated discordant recommendations or into three additional categories in which discordant recommendations were inconsistent with GRADE guidance: 1) the evidence warranted moderate or high confidence (a misclassification of evidence) rather than low or very low confidence; 2) good practice statements; or 3) uncertainty in the estimates of effect would best lead to a conditional (weak) recommendation. RESULTS: The 33 eligible guidelines included 160 discordant recommendations, of which 98 (61.3%) addressed drug interventions and 132 (82.5%) provided some rationale (though not entirely explicit at times) for the strong recommendation. Of 160 discordant recommendations, 25 (15.6%) were judged consistent with one of the five paradigms for appropriate recommendations; 33 (21%) were based on evidence warranting moderate or high confidence in the estimates of effect; 29 (18%) were good practice statements; and 73 (46%) warranted a conditional, rather than a strong recommendation. CONCLUSION: WHO discordant recommendations are often inconsistent with GRADE guidance, possibly threatening the integrity of the process. Further training in GRADE methods for WHO guideline development group members may be necessary, along with further research on what motivates the formulation of such recommendations.
Keywords:
Confidence in effect estimates; Discordant recommendations; GRADE; Low study quality; Public health guidelines; Strong recommendations; World Health Organization
Authors: Joanna Palasz; Linda D'Antona; Sarah Farrell; Mohamed A Elborady; Laurence D Watkins; Ahmed K Toma Journal: Neurosurg Rev Date: 2021-08-26 Impact factor: 3.042
Authors: Herbert I Fried; Barnett R Nathan; A Shaun Rowe; Joseph M Zabramski; Norberto Andaluz; Adarsh Bhimraj; Mary McKenna Guanci; David B Seder; Jeffrey M Singh Journal: Neurocrit Care Date: 2016-02 Impact factor: 3.210
Authors: Maria Cecilia Dedios; Alexo Esperato; Luz Maria De-Regil; Juan Pablo Peña-Rosas; Susan L Norris Journal: Implement Sci Date: 2017-03-21 Impact factor: 7.327
Authors: Ein Soon Shin; Ji Eun Jang; Ji Yun Yeon; Da Sol Kim; Sung Goo Chang; Yoon Seong Lee Journal: J Korean Med Sci Date: 2018-02-19 Impact factor: 2.153
Authors: Thomas Agoritsas; Arnaud Merglen; Anja Fog Heen; Annette Kristiansen; Ignacio Neumann; Juan P Brito; Romina Brignardello-Petersen; Paul E Alexander; David M Rind; Per O Vandvik; Gordon H Guyatt Journal: BMJ Open Date: 2017-11-16 Impact factor: 2.692
Authors: M Rehn; P K Hyldmo; V Magnusson; J Kurola; P Kongstad; L Rognås; L K Juvet; M Sandberg Journal: Acta Anaesthesiol Scand Date: 2016-06-03 Impact factor: 2.105