Romina Brignardello-Petersen1, Bradley C Johnston2, Alejandro R Jadad3, George Tomlinson4. 1. Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street W, Hamilton, Ontario L8S 4L8, Canada; Evidence-Based Dentistry Unit, Faculty of Dentistry, University of Chile, Sergio Livingstone 943, Independencia, Santiago, Chile. Electronic address: brignarr@mcmaster.ca. 2. Institute of Health Policy, Management and Evaluation, University of Toronto, 155 College Street, Toronto, Ontario M5T 3M6, Canada; Child Health Evaluative Sciences, Hospital for Sick Children Research Institute, Peter Gilgan Centre for Research and Learning, 686 Bay Street, Rm 11.9859 West, Toronto, Ontario M5G 0A4, Canada; Department of Anesthesia and Pain Medicine, Hospital for Sick Children, 555 University Avenue, Toronto, Ontario M5G 1X8, Canada. 3. Institute of Health Policy, Management and Evaluation, University of Toronto, 155 College Street, Toronto, Ontario M5T 3M6, Canada; Institute for Global Health Equity and Innovation, Dalla Lana School of Public Health, University of Toronto, 155 College Street, Toronto, Ontario M5T 3M6, Canada; Centre for global eHealth Innovation, R Fraser Elliot Building, 190 Elizabeth Street, Toronto, Ontario M5G 2C4, Canada. 4. Institute of Health Policy, Management and Evaluation, University of Toronto, 155 College Street, Toronto, Ontario M5T 3M6, Canada; Department of Medicine, University Health Network and Mt Sinai Hospital, Toronto, Eaton North, 10th floor, room 235, 200 Elizabeth Street, Toronto, Ontario M5G 2C4, Canada.
Abstract
OBJECTIVES: To evaluate how the rank probabilities obtained from network meta-analysis (NMA) change with the use of increasingly stringent criteria for the relative effect comparing two treatments which ranks one treatment better than the other. STUDY DESIGN AND SETTING: Systematic survey and reanalysis of published data. We included all systematic reviews (SRs) with NMA from the field of cardiovascular medicine that had trial-level data available, published in Medline up to February 2015. We reran all the NMAs and determined the probabilities of each treatment being the best. For the best treatment, we examined the effect on these probabilities of varying, what we call the decision threshold, the relative effect required to declare two treatments different. RESULTS: We included 14 SRs, having a median of 20 randomized trials and 9 treatments. The best treatments had probabilities of being best that ranged from 38% to 85.3%. The effect of changing the decision thresholds on the probability of a treatment being best varied substantially across reviews, with relatively little decrease (∼20 percentage points) in some settings but a decline to near 0% in others. CONCLUSION: Rank probabilities can be fragile to increases in the decision threshold used to claim that one treatment is more effective than another. Including these thresholds into the calculation of rankings may aid their interpretation and use in clinical practice.
OBJECTIVES: To evaluate how the rank probabilities obtained from network meta-analysis (NMA) change with the use of increasingly stringent criteria for the relative effect comparing two treatments which ranks one treatment better than the other. STUDY DESIGN AND SETTING: Systematic survey and reanalysis of published data. We included all systematic reviews (SRs) with NMA from the field of cardiovascular medicine that had trial-level data available, published in Medline up to February 2015. We reran all the NMAs and determined the probabilities of each treatment being the best. For the best treatment, we examined the effect on these probabilities of varying, what we call the decision threshold, the relative effect required to declare two treatments different. RESULTS: We included 14 SRs, having a median of 20 randomized trials and 9 treatments. The best treatments had probabilities of being best that ranged from 38% to 85.3%. The effect of changing the decision thresholds on the probability of a treatment being best varied substantially across reviews, with relatively little decrease (∼20 percentage points) in some settings but a decline to near 0% in others. CONCLUSION: Rank probabilities can be fragile to increases in the decision threshold used to claim that one treatment is more effective than another. Including these thresholds into the calculation of rankings may aid their interpretation and use in clinical practice.
Authors: Mona Hersi; Gregory Traversy; Brett D Thombs; Andrew Beck; Becky Skidmore; Stéphane Groulx; Eddy Lang; Donna L Reynolds; Brenda Wilson; Steven L Bernstein; Peter Selby; Stephanie Johnson-Obaseki; Douglas Manuel; Smita Pakhale; Justin Presseau; Susan Courage; Brian Hutton; Beverley J Shea; Vivian Welch; Matt Morrow; Julian Little; Adrienne Stevens Journal: Syst Rev Date: 2019-01-19