Evidence used in model-based economic evaluations for evaluating pharmacogenetic and pharmacogenomic tests: a systematic review protocol.

INTRODUCTION: Decision models can be used to conduct economic evaluations of new pharmacogenetic and pharmacogenomic tests to ensure they offer value for money to healthcare systems. These models require a great deal of evidence, yet research suggests the evidence used is diverse and of uncertain quality. By conducting a systematic review, we aim to investigate the test-related evidence used to inform decision models developed for the economic evaluation of genetic tests. METHODS AND ANALYSIS: We will search electronic databases including MEDLINE, EMBASE and NHS EEDs to identify model-based economic evaluations of pharmacogenetic and pharmacogenomic tests. The search will not be limited by language or date. Title and abstract screening will be conducted independently by 2 reviewers, with screening of full texts and data extraction conducted by 1 reviewer, and checked by another. Characteristics of the decision problem, the decision model and the test evidence used to inform the model will be extracted. Specifically, we will identify the reported evidence sources for the test-related evidence used, describe the study design and how the evidence was identified. A checklist developed specifically for decision analytic models will be used to critically appraise the models described in these studies. Variations in the test evidence used in the decision models will be explored across the included studies, and we will identify gaps in the evidence in terms of both quantity and quality. DISSEMINATION: The findings of this work will be disseminated via a peer-reviewed journal publication and at national and international conferences. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

The systematic review will extensively search decision models for evaluating pharmacogenetic and pharmacogenomic tests.

Focusing on the test-related evidence used allows a thorough investigation into the quantity and quality of such evidence to inform these models.

Obtaining the level of detail required to answer the systematic review questions may be limited by the amount of information reported in the included articles.

Introduction

Information from genetic and genomic tests is increasingly being used to inform patient management decisions in healthcare systems.1 Examples include the identification of individuals likely to respond to treatment (eg, treatment with cetuximab in individuals with K-RAS wild-type colorectal cancer2), likely to have adverse treatment responses (eg, HLA-B*15:02 testing to predict Stevens-Johnson syndrome and toxic epidermal necrolysis when receiving carbamazopine)3, or to inform treatment dose (eg, thiopurine S-methyltransferase testing prior to treatment with azathioprine)4. Such tests are commonly called pharmacogenetic or pharmacogenomic tests and are referred to collectively as pharmacogenomic tests hereafter. Economic evaluation of these tests is required to ensure that these interventions are providing value for money. Test-treatment randomised controlled trials capturing the health outcomes arising from the actions taken as a consequence of test results can be complex, time-consuming, costly5 and have a strong potential for bias,6 so are rare.7 8 Decision analytic models are therefore advocated as the most systematic and transparent method for economic evaluation.9 10 Decision analytic modelling allows the costs and benefits of strategies involving genomic testing to inform treatment response, and permits subsequent patient management decisions to be compared with standard approaches, providing insights into the trade-offs associated with the use of these strategies. However, evidence suggests that relevant aspects of pharmacogenomic testing are not necessarily being captured in economic evaluations,11 and there is a lack of standardisation in methods12 and outcomes used.13 A recent methodological review14 highlighted additional issues in developing decision models for genomic testing strategies in general, including poor-quality effectiveness evidence and uncertainty concerning the appropriate analytical perspective, what resource and cost data to include, and how to measure outcomes and effectiveness.

A key issue with model-based economic evaluations of pharmacogenomic tests is that they generally contain many more parameters than decision models for economic evaluations of treatments. In addition to modelling the analytical and clinical validity and cost of the genomic testing, other aspects of testing that may be important in these evaluations include:

The strength of relationship between the genetic information and clinical outcomes: the results of genomic tests do not themselves lead to improved outcomes. Links need to be made between the genomic test results, the treatment options available, the likely treatment response and the clinical outcomes for individuals.

Estimates of the uptake of genomic testing by patients and clinicians: even if genomic testing has greater analytical and clinical validity than current practice, if individuals are less likely to agree to the genomic testing, it will have little clinical utility and may result in fewer clinical benefits compared with current practice.

Consequences of false-positive and false-negative test results: depending on the context, the consequences of incorrect test results may have a large impact on the findings of the economic evaluation, for example, severe health impacts of experiencing an adverse drug reaction.

Costs of sample collection: the costs associated with collecting the samples required for genomic testing should be accounted for.

Costs of genetic counselling: it may be the case that additional resources are associated with a genomic testing strategy, so that details of the testing and the results can be communicated to, and understood by, those eligible for genomic testing.

Test failures and/or repeated testing: it is possible that tests may not provide usable results and additional samples may need to be collected and/or tests repeated. Accounting for the costs of obtaining additional samples and/or the time impacts of any failures and repeat testing may be important in an economic evaluation.

Given these considerations, it is not always the case that analytical and clinical validity drive the economic evaluation of pharmacogenomic testing: the clinical utility of new strategies must also be considered.5 15 It is therefore important that the evidence base to inform pharmacogenomic test parameters in decision models consists of the most relevant and unbiased evidence possible. However, research suggests that for many model-based economic evaluations in health technology assessment, this evidence base is often diverse and of uncertain quality, and that sufficient information is rarely provided on how evidence has been identified.16 Although reviews of model-based economic evaluations of pharmacogenomics tests have been conducted,11 12 they have not specifically evaluated the evidence base informing the decision models. In this review, we will systematically investigate the use of test-related evidence in economic evaluations of pharmacogenomic tests to inform treatment response and subsequent patient management decisions. Test-related evidence includes evidence on the analytical and clinical validity of the test, its clinical utility including the relation between genetic information and clinical outcomes, consequences of incorrect test results, test failures and repeats, costs of the test, sample collection and genetic counselling. We will also comment on the quality and quantity of this evidence. Understanding the current state of evidence used in decision models for pharmacogenomic tests will help identify what evidence is lacking and highlight areas where the collection of better quality evidence would be useful for future evaluations. This systematic review protocol has been reported according to the PRISMA-P reporting guidelines.17

The aim of this systematic review is to answer the following questions (1) What test-related evidence is being included in model-based economic evaluations of pharmacogenomic tests? (2) How is this evidence being identified? (3) What is the quality of this evidence? and (4) What is the general quality of these model-based economic evaluation?

Methods and analysis

Population: There will be no restrictions placed on the populations in which pharmacogenomic testing strategies are evaluated. For instance, individuals may be newly diagnosed with a condition and yet untreated, or may have received a number of previous treatments before being considered for pharmacogenomic testing.

Intervention: Any pharmacogenomic test used for predicting treatment response will be included. This will include targeted genetic tests as well as genomic tests, and may include next-generation sequencing.

Study design: Economic evaluations of pharmacogenomic tests using decision modelling will be sought regardless of the type of modelling used. Given that there are no restrictions on the outcomes used (see below), this could include cost-effectiveness, cost-utility, cost-benefit, cost-minimisation and cost-consequence analyses.

Measurement of outcomes: There will be no restrictions on the measurement of outcomes. The systematic review will capture all reported model outcomes, which may include quality-adjusted life years (QALYs) from cost-utility analyses, cases detected from cost-effectiveness analyses, net monetary benefits from cost-benefit analyses, as well as other outcomes.

Search strategy: The search strategy will take the following form: (terms for genetic tests) AND (a bespoke methodological search filter to locate studies which use decision analytic models).

The search strategy, informed by the Centre for Reviews and Dissemination guidance18 will be run in the following bibliographic databases:

MEDLINE and MEDLINE in PROCESS (via OVID) 1946 to March 2015;

EMBASE (via OVID) 1974 to March 2015 March;

NHS EEDs via (The Cochrane Library, Wiley interface) 1994 to March 2015;

Econlit (via EBSCO Host) 1886 to March 2015; and

Web of Science (via ISI) 1900 to March 2015.

As NHS EED is no longer updated, we will be searching this resource as an archive. The HEED database closed in 2015 and it is no longer possible to search it, or access the archive. The annotated search strategy is provided in the online supplementary material. Reports produced by health technology assessment agencies will also be searched to identify relevant model-based economic evaluations that may not have been published. In particular, the online records of the National Institute for Health and Care Excellence in England, the Pharmaceutical Benefit Scheme in Australia and the Canadian Agency for Drugs and Technologies in Health will be searched.

Search limit: Where possible, the search will be limited to human-only population groups. The search will not be limited by language or date. Owing to the level of information required from each article in this review, only studies reporting full details of the decision model will be included. Therefore, conference abstracts will be excluded at the screening stage.

Search recording: EndNote V.7.3 (Thompson Reuters).

Study selection: There will be two stages to the screening. Following de-duplication, title and abstract screening to identify model-based economic evaluations of pharmacogenomic tests will be completed by two reviewers using defined inclusion and exclusion criteria (see table 1). Pilot screening of 100 hits has shown a very high level of agreement between these two reviewers (κ statistic of 0.93). Screening of full-text articles will be completed by one reviewer (but in discussion with a second researcher should there be uncertainties regarding the inclusion of an article).

Table 1

Inclusion and exclusion criteria

	Included	Excluded
Study type	Model-based economic evaluations, including cost-effectiveness analyses, cost-utility analyses, cost-benefit analyses, cost-consequence analyses, cost-minimisation analyses	Any non-model-based economic evaluationAny decision model not including measurement of costs
Population	Any	–
Disease/condition	Any	–
Purpose of testing	Genetic or genomic testing to predict treatment response	Any genomic or genetic testing used for screening, diagnosis, prognosis or prediction of current or future disease status

Data extraction: A data extraction form will be developed and piloted. Details to be collected will include:

Characteristics of the decision problem, such as disease/condition, gene(s), setting, perspective, purpose of the test (eg, to predict a treatment response, aid dose setting, predict adverse drug reactions), type of test (eg, fluorescence in situ hybridisation testing, Sanger sequencing, microarray testing, whole genome sequencing).

Characteristics of the decision model, such as the model structure (eg, decision tree plus Markov model), discount rate, time horizon, outcome measures used (eg, QALYs, cases detected), whether probabilistic analyses were done.

Which aspects of the pharmacogenomic testing strategy reflect clinical utility/benefit above current practice (eg, improved clinical validity, less invasive testing). We will use the checklist developed by Ferrante di Ruffano et al15 to help identify the clinical utility of the new pharmacogenomic test(s).

Characteristics of the test evidence used to inform the model, including those stated in the introduction. The evidence source used, its study design, how the evidence was identified (eg, by a systematic review, not reported), whether an assessment of the quality of the evidence was reported to have been done. The evidence hierarchy used by Cooper et al16 will be used to help assess these characteristics.

Whether sensitivity analyses have captured uncertainty in the genomic test evidence.

Whether authors have reported the use of good practice guidelines to conduct their analyses and/or report their model and results, such as the Modelling Good Practice Guidelines19 or the Consolidated Health Economic Evaluations Reporting Standards (CHEERS) statement.20

The first 20% of included articles will have data extracted by one reviewer and checked by another. If there are any disagreements or inaccuracies in the data extraction, these will be discussed. Once these disagreements or inaccuracies have been addressed, one reviewer will extract data from the remaining included articles, in discussion with another reviewer in the case of uncertainties.

Study quality: A modified version of the Philips et al21 checklist for the quality of economic evaluations will be piloted before use. A copy of the checklist is given in the appendix but may change after piloting. The Phillips checklist is a suggested list of items for critical appraisal of decision analytic models in health technology assessment and will reflect a number of decision model characteristics that will be extracted.

Data synthesis: Characteristics of the decision models will be tabulated and summarised, drawing together similarities and highlighting differences in approach and/or quality. Variations in the test evidence used in the decision models will be explored, and we will identify gaps in the evidence in terms of both quantity and quality.

Reporting: The systematic review will be reported in line with the PRISMA reporting guidelines.22

Discussion: The systematic review will help to characterise the state of decision models evaluating pharmacogenomic testing strategies. It will focus primarily on the evidence used in the decision models to inform the pharmacogenomic testing aspects of the evaluation; however, it is acknowledged that the detail required may be limited by the extent of reporting in included articles (any evidence of this effect will also be noted). Understanding the extent to which genetic test evidence is incorporated into decision models, with particular attention paid to the identification of this evidence, its type and quality, will highlight evidence gaps and areas where better quality evidence is needed.

Ethics and dissemination

As this is secondary research, ethical approval is not required. Disseminating this work to developers of genetic and genomic tests will be important to highlight current evidence gaps as future research priorities. The findings of this work will also be very relevant to researchers undertaking decision modelling to help consider the type of test-related evidence that might be included in future models, and also provide insight on how to identify such evidence. Dissemination will be undertaken via a peer-reviewed journal publication and at national and international conferences.