Thomas Piggott1, Jan Brozek1, Artur Nowak2, Helena Dietl2, Bart Dietl2, Zuleika Saz-Parkinson3, Fuad Mirzayev4, Licé González-Angulo4, Juan José Yepes-Nuñez5, Reem A Mustafa6, Grigorios I Leontiadis7, Romina Brignardello-Petersen1, Pablo Alonso-Coello8, Holger J Schünemann9. 1. Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada. 2. Evidence Prime, Hamilton, Canada and Krakow, Poland. 3. European Commission, Joint Research Centre, Ispra, Italy. 4. Global TB Program, World Health Organization, Geneva, Switzerland. 5. Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada; School of Medicine, Universidad de Los Andes, Bogotá, Colombia. 6. Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada; Department of Medicine, University of Kansas Health System, Kansas City, USA. 7. Department of Medicine, McMaster University, Hamilton, Canada. 8. Iberoamerican Cochrane Center-Servicio de Epidemiología Clínica y Salud Pública, Biomedical Research Institute (IIB-Sant Pau), Barcelona, Spain; CIBER of Epidemiology and Public Health (CIBERESP), Barcelona, Spain. 9. Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada; Department of Medicine, McMaster University, Hamilton, Canada; Institut für Evidence in Medicine, Medical Center & Faculty of Medicine, University of Freiburg, Freiburg, Germany. Electronic address: schuneh@mcmaster.ca.
Abstract
BACKGROUND AND OBJECTIVE: Guideline development groups or other health care decision makers frequently encounter situations that require a simultaneous comparison of multiple interventions. This sometimes becomes apparent either when they identify questions of interest, before they formulate recommendations, or it may surface only when recommendations have already been formulated based on pairwise comparisons. METHODS: Using examples from the World Health Organization, the European Commission, and a professional society, we developed a flexible approach to developing recommendations when a multiple-intervention comparison (MC) is needed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) evidence to decision (EtD) frameworks. We iteratively refined this approach through user testing and then included a module in GRADE's official software GRADEpro to test the approach in two real and one theoretical guideline recommendations. RESULTS: We found the approach feasible and that all EtD criteria should be considered in an MC approach. We judged that guideline development groups and other decision makers will benefit from the availability of a network meta-analyses (NMA) of intervention effects to support decisions; however, NMA supports only one of many criteria, that is, the balance of health benefits and harms, and is therefore helpful, but not essential to the approach we propose. When similar but not identical comparators are used to address MC, challenges may arise with intransitivity and the relative rankings of interventions. CONCLUSION: We successfully applied the MC approach and software module in generating recommendations across different scenarios and identified challenges. The MC approach allows guideline groups and other decision makers to transparently and critically assess multiple options for a given health question. Application of the approach by others may lead to refinement and allow for better understanding of its impact in developing recommendations and making choices.
BACKGROUND AND OBJECTIVE: Guideline development groups or other health care decision makers frequently encounter situations that require a simultaneous comparison of multiple interventions. This sometimes becomes apparent either when they identify questions of interest, before they formulate recommendations, or it may surface only when recommendations have already been formulated based on pairwise comparisons. METHODS: Using examples from the World Health Organization, the European Commission, and a professional society, we developed a flexible approach to developing recommendations when a multiple-intervention comparison (MC) is needed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) evidence to decision (EtD) frameworks. We iteratively refined this approach through user testing and then included a module in GRADE's official software GRADEpro to test the approach in two real and one theoretical guideline recommendations. RESULTS: We found the approach feasible and that all EtD criteria should be considered in an MC approach. We judged that guideline development groups and other decision makers will benefit from the availability of a network meta-analyses (NMA) of intervention effects to support decisions; however, NMA supports only one of many criteria, that is, the balance of health benefits and harms, and is therefore helpful, but not essential to the approach we propose. When similar but not identical comparators are used to address MC, challenges may arise with intransitivity and the relative rankings of interventions. CONCLUSION: We successfully applied the MC approach and software module in generating recommendations across different scenarios and identified challenges. The MC approach allows guideline groups and other decision makers to transparently and critically assess multiple options for a given health question. Application of the approach by others may lead to refinement and allow for better understanding of its impact in developing recommendations and making choices.