Alexander Mensch1, Tanja Beck2, Daniele Civello2, Christopher Kunigkeit2, Nicole Lachmann2, Stephanie Stock2, Afschin Gandjour3, Dirk Müller4. 1. Janssen-Cilag GmbH, Johnson & Johnson Platz 1, 41470, Neuss, Germany. 2. Institute for Health Economics and Clinical Epidemiology, The University Hospital of Cologne (AöR), Gleueler Straße 176-178, Cologne, Germany. 3. Frankfurt School of Finance and Management, Sonnemannstr. 9-11, 60314, Frankfurt am Main, Germany. 4. Institute for Health Economics and Clinical Epidemiology, The University Hospital of Cologne (AöR), Gleueler Straße 176-178, Cologne, Germany. dirk.mueller@uk-koeln.de.
Abstract
BACKGROUND: Concerns have been raised about the use of clinical data in cost-effectiveness models. The aim of this analysis was to evaluate the appropriate use of data on clinical effectiveness in cost-effectiveness modeling studies that were published between 2001 and 2015. METHODS: Assessors rated 72 modeling studies obtained from three therapeutic areas by applying criteria defined by the Grading of Recommendations Assessment, Development and Evaluation group for assessing the quality of clinical evidence: selection of clinical data (publication bias), imprecision, indirectness, inconsistency (i.e., heterogeneity), and study limitations (risk of bias). For all parameters included in the analyses, potential changes over time were assessed. RESULTS: Although three out of four modeling studies relied on randomized controlled trials, more than 60% of the modeling studies were based on clinical data with a high or unclear risk of bias, in more than 80%, a risk of publication bias was found, and in about 30%, evidence was based on indirect clinical evidence, having significantly increased over the years. Study limitations were inadequately described in more than one third of the studies. However, less than 10% of clinical studies showed inconsistency or imprecision in study results. CONCLUSION: Despite the fact that the majority of economic evaluations are based on precise and consistent randomized controlled trials, their results are often affected by limitations arising from methodological shortcomings in the underlying data on clinical efficacy. Modelers and assessors should be more aware of aspects surrounding the quality of clinical evidence as considered by the Grading of Recommendations Assessment, Development and Evaluation group.
BACKGROUND: Concerns have been raised about the use of clinical data in cost-effectiveness models. The aim of this analysis was to evaluate the appropriate use of data on clinical effectiveness in cost-effectiveness modeling studies that were published between 2001 and 2015. METHODS: Assessors rated 72 modeling studies obtained from three therapeutic areas by applying criteria defined by the Grading of Recommendations Assessment, Development and Evaluation group for assessing the quality of clinical evidence: selection of clinical data (publication bias), imprecision, indirectness, inconsistency (i.e., heterogeneity), and study limitations (risk of bias). For all parameters included in the analyses, potential changes over time were assessed. RESULTS: Although three out of four modeling studies relied on randomized controlled trials, more than 60% of the modeling studies were based on clinical data with a high or unclear risk of bias, in more than 80%, a risk of publication bias was found, and in about 30%, evidence was based on indirect clinical evidence, having significantly increased over the years. Study limitations were inadequately described in more than one third of the studies. However, less than 10% of clinical studies showed inconsistency or imprecision in study results. CONCLUSION: Despite the fact that the majority of economic evaluations are based on precise and consistent randomized controlled trials, their results are often affected by limitations arising from methodological shortcomings in the underlying data on clinical efficacy. Modelers and assessors should be more aware of aspects surrounding the quality of clinical evidence as considered by the Grading of Recommendations Assessment, Development and Evaluation group.
Authors: Z Philips; L Ginnelly; M Sculpher; K Claxton; S Golder; R Riemsma; N Woolacoot; J Glanville Journal: Health Technol Assess Date: 2004-09 Impact factor: 4.014
Authors: Gordon H Guyatt; Andrew D Oxman; Holger J Schünemann; Peter Tugwell; Andre Knottnerus Journal: J Clin Epidemiol Date: 2010-12-24 Impact factor: 6.437
Authors: Edward J Mills; John P A Ioannidis; Kristian Thorlund; Holger J Schünemann; Milo A Puhan; Gordon H Guyatt Journal: JAMA Date: 2012-09-26 Impact factor: 56.272
Authors: Don Husereau; Michael Drummond; Stavros Petrou; Chris Carswell; David Moher; Dan Greenberg; Federico Augustovski; Andrew H Briggs; Josephine Mauskopf; Elizabeth Loder Journal: BMJ Date: 2013-03-25
Authors: Beverley J Shea; Jeremy M Grimshaw; George A Wells; Maarten Boers; Neil Andersson; Candyce Hamel; Ashley C Porter; Peter Tugwell; David Moher; Lex M Bouter Journal: BMC Med Res Methodol Date: 2007-02-15 Impact factor: 4.615