A Practical Guide to Modeling and Conducting a Cost-Effectiveness Analysis of Companion Biomarker Tests for Targeted Therapies Using R: Tutorial Paper.

Despite the increasing number of potential biomarkers identified in laboratories and reported in much literature, the adoption of biomarkers routinely available in clinical practice to inform treatment decisions is very limited. Reimbursement decisions for new health technologies are often informed by economic evaluations; however, economic evaluations of diagnostics/testing technologies, such as companion biomarker tests, are far less frequently reported than drugs. Furthermore, few countries provide the health economic evaluation methods guide specific to co-dependent technologies such as companion diagnostics or precision medicines. Therefore, this paper aims to guide the process of the development of cost-effectiveness models of cancer biomarkers for targeted therapies, focusing on companion diagnostics. This tutorial paper provides practical guidance on how to conduct economic evaluations of cancer biomarkers and how to model the characteristics of the biomarker tests as part of the value for money of corresponding targeted therapies. This paper presents a brief introduction to the methods and data requirements, a step-by-step guide to constructing a health economic model of companion cancer biomarkers, and a discussion of issues that arise in their application to healthcare decision making. This practical guidance is provided in R, and worked examples are provided in this paper with R codes in the accompanying electronic supplementary material.

Key Points for Decision makers

Introduction

The optimization of treatment strategies has become possible based on the information provided by biomarkers before the administration of treatments, especially in oncology. This advance has raised expectations over biomarker-guided therapies in cancer that may improve patient outcomes while helping to achieve efficient resource allocation in healthcare [1-5]. However, the number of biomarkers successfully entering clinical routine practices is very low compared with the number of biomarkers published [6, 7]. One argues that the lack of consensus in methodological approaches and data requirements in economic evaluations of biomarkers might be one of the limiting factors on why there are as yet only a small number of biomarker tests routinely provided in clinical practice [8-10]. Furthermore, few countries provide a guide to the methods of health economic evaluation specifically for co-dependent technologies; only two countries (Australia and Scotland) provide some high-level guidance on modeling the characteristics of companion testing technologies as part of assessing the value for money of co-dependent technologies such as biomarker-guided therapies or companion diagnostics [11, 12]. It reflects the current reality that reimbursement agencies in many countries do not keep pace with the rapidly evolving health technologies such as ‘omics’-based therapies with the integration of companion biomarkers.

This tutorial paper aims to guide the process of the development of cost-effectiveness models of companion cancer biomarkers for targeted therapies (specifically companion diagnostics, classifying patients into responders and non-responders for a specified therapeutic agent in treating patients with cancer). Companion biomarker tests (interchangeably, companion diagnostics) guide the safe and effective use of therapeutics with its approved label restricting drug access [13]. Although model conceptualization is the first key step in developing an appropriate model, it is beyond the scope of this tutorial paper. This paper is intended for those who chose a state-transition model as their appropriate model, based on their decision problems to be represented in the model. For those who are not yet clear what model types are appropriate for their decision problems, there is a useful paper providing a series of consensus-based best practices for the process of model conceptualization [14]. For example, when the decision problem requires modeling the effect of patient interaction (e.g. the treatment effect on disease spread), this core model is not applicable. As explained by Roberts et al. [14], this state-transition model is appropriate in instances where the disease is broken into distinct health states, as in cancer.

Users can adapt this core model to assess their biomarker-guided therapies by making local adaptations in data requirements and methodological approaches from the perspective of their specific payers and country settings. The example used in this tutorial paper has three health states, progression-free survival (PFS), progressive disease (PD), and dead. This analysis is performed for a hypothetical cohort of cancer patients who are not eligible for tumor excision surgery.

We chose to use R (The R Foundation for Statistical Computing, Vienna, Austria) in building this practical model because of the advantages of using R (or script-based programming) for the development of economic models for health technology assessment (HTA), although these are only beginning to be recognized [15]. R is easily reproducible and flexible compared with Microsoft Excel® (Microsoft Corporation, Redmond, WA, USA).

Model Background and Description

Overall, several elements need to be defined in order to construct the health economic model for health technologies. The decision problem of this tutorial is to assess the cost effectiveness of testing patients with a companion biomarker test and treating them according to their biomarker status, in comparison with comparator strategies such as treat all patients with the biomarker-guided therapy or treat all patients with usual treatment regardless of biomarker status without testing. The study design is a model-based cost-effectiveness analysis using a hypothetical cohort of patients, and the study outcome to be calculated is incremental cost-effectiveness ratio (ICER; cost per life-year [LY] and cost per quality-adjusted life-year [QALY] gained). The reference case applied in this worked example of the core model is summarized in Table 1. This core model is developed based on the findings and practical lessons acquired from previous studies on companion diagnostics or co-dependent technologies [8, 10, 16, 17]. Systematic reviews found that the characteristics of companion biomarker tests were not consistently incorporated in economic evaluations of biomarker-guided therapies and the structure of comparative analyses of the strategy arms was so varied that it may lead to a different conclusion in terms of cost effectiveness [8, 10, 17]. The process of building a health economic model involves defining the structure of the model and data inputs. Its detailed descriptions are provided in the following subsections.

Table 1

Summary of the reference case used in this guide

Element	Reference case
Intervention strategy	Test–treat patients according to biomarker status, using companion diagnostics for targeted therapies
Choice of treatment alternative (comparator strategies)	The comparator strategy that the new biomarker-guided therapy will most likely replace. Thus, in this core model, two comparator strategies were employed: (1) Treat all patients with biomarker-guided therapy regardless of biomarker status; (2) Treat all patients with usual treatment regardless of biomarker status
Health state	Three health states: progression-free survival (PFS), progressed disease (PD) and dead
Viewpoint of the analysis	Health system perspective
Time horizon	Lifetime
Analysis model	Cost-utility analysis
Health outcome	Quality-adjusted life-year
Method for the measurement and valuation of health effects	Generic measures of health instruments
Discounting rate	3.5%
Uncertainty	Probabilistic sensitivity analysis; with an option of deterministic sensitivity analysis

Strategy Arms to be Compared and Assessed

The intervention strategy of this core model is biomarker-guided therapies guided by companion diagnostics that patients are tested prior to the administration of targeted therapies according to their biomarker status (Table 1). In terms of comparator strategy that this intervention strategy is being compared against, standard of care (SOC) is an appropriate comparator strategy that reflects the most relevant alternative intervention(s) used in clinical practice [18, 19]. However, previous literature reviews [8, 10] found that the existing literature of economic evaluations demonstrates that the choice of comparator strategies and the comparison structure is not consistently applied in economic evaluations of biomarker-guided therapies. For example, SOC (e.g. usual therapy without biomarker testing) was not chosen as a comparator strategy in some evaluations. Instead, they compared biomarker-guided therapy with biomarker testing versus without the testing, while no SOC was considered in the evaluations. Therefore, based on the study findings from previous studies [8, 10, 16], we found that assessing the biomarker-guided therapy against two comparator strategies was the most suitable structure for strategy comparison. Therefore, we constructed three strategies as default comparisons in this core model: (1) patients being tested with a companion biomarker and treated with the corresponding targeted therapy according to their biomarker testing result (hereinafter referred to as the test–treat strategy, i.e. ‘TT arm’); (2) patients not tested and treated with SOC (hereinafter referred to as the usual care strategy, i.e. ‘all-UC arm’); and (3) patients not tested and treated with the corresponding targeted therapy (hereinafter referred to as the targeted care strategy, i.e. ‘all-TC arm’). These arms can be compared with one another depending on the decision problem, and the users are expected to adapt this core model for their specific research question. This construct of comparative strategy arms is also line with what has been suggested by previous studies [17, 20]. The detailed schematic of the comparative structure of strategy arms is depicted in Fig. 1.

Fig. 1

Model schematic. ‘M’ indicates a move into the Model in Fig. 2. PFS progression-free survival, PD progressed disease, M Markov model

Fig. 2

Health transition diagram. PFS progression-free survival, PD progressed disease

Model Structure

A discrete-time Markov cohort model is constructed to record the transition between health states experienced by a hypothetical cohort of patients eligible to be treated either with targeted care (biomarker-guided therapy) or usual care (non-guided therapy) in oncology treatments. Health-related quality of life (HRQoL) weights and a cost pertinent to each of these health states are assigned. The model has three mutually exclusive health states: PFS, PD, and dead. As depicted in Fig. 2, the arrows indicate the flow of individual patients in every model cycle. Transition from PD to PFS is assumed to be impossible. The transition probability can be calculated using the formula suggested by Briggs et al. [21]. Given that health states are mutually exclusive, the transition probabilities sum to one. A Markov model of disease progression is presented in Fig. 2. The detailed model schematic of decision tree linking to the health state transitions is provided in Fig. 1, with ‘M’ indicating a move into the Markov model. Once patients are allocated to their respective decision branch, they enter a Markov model based on their assigned transition probabilities. Patients assigned to ‘treat-all’ strategies (either with new therapy or with usual therapy) will enter the Markov model without being biomarker tested, and move to respective health states (PFS, PD, dead) assigned by the given transition probabilities. On the other hand, patients assigned to the ‘test–treat’ strategy arm will be either provided new therapy or usual therapy according to biomarker status, and will then enter a Markov model and be assigned to a respective health state followed by transition probabilities. A lifetime horizon is applied.

Data Requirements and Model Inputs

Model inputs are detailed in Table 2. These data inputs are just exemplary figures to guide the process of developing an economic model for biomarker-guided therapies, developed based on the previous study findings [8, 10, 16]. For example, we have incorporated all data inputs relevant to the key characteristics of companion biomarker tests that are often ignored in existing economic evaluations of biomarker-guided therapies such as clinical utility, patient preferences (disutility or utility of biomarker testing), frequency/prevalence of biomarker status, and diagnostic accuracy, including false negative and false positive. A third-party payer perspective (e.g. the National Health Service) is employed in developing the model, and thus any non-medical costs (e.g. lost productivity costs) are beyond the scope of this core modeling practice. Health state costs are defined per model cycle, including drug costs and biomarker testing costs (Table 2). HRQoL (e.g. EQ-5D) data inputs are also provided in Table 2. In practice, HRQoL data are often obtained along with clinical trials or by separate literature reviews (e.g. systematic literature review and/or meta-analysis); however, for the development of this practical guide, dummy estimates on utility values are used in this core model, although these data are obtained by clinical trials or separate literature reviews in practice. It is recommended that all relevant data from difference sources (e.g. clinical trials) are systematically synthesized and incorporated in economic evaluations [22, 23]. Biomarker-related parameters such as biomarker testing disutility value, performance accuracy (sensitivity and specificity), and biomarker prevalence are also shown in Table 2. Companion diagnostic technology for cancer patients usually require collecting a biosample for analysis, and this gives rise to the existence of process utility (such as reassurance or information) [24-26]. Patients might express different preferences by being informed of the process of interventions or diagnostics; for example, patients may prefer diagnostic A because it is more convenient or informative to them than diagnostic B, even though there is no definite difference in health outcomes. Brennan and Dixon supported the existence of process utility and found different approaches being used to detect and measure it [27]. Given the existence of process utility, in this core model, testing disutility was used under the assumption that undergoing biomarker testing might cause some discomfort to patients. However, if this is not the case (e.g. testing bringing not discomfort but convenience to patients), the utility value of testing should be considered when adapting this core model. In addition, transition probabilities, drug efficacy, and discounting rate are also provided. All-cause mortality was not considered in this core model, however it should be considered when adapting this core model for local adaptations of country-specific settings. In other words, modellers are advised to incorporate country-specific epidemiological data, such as all-cause mortality, into the core model for their local adaptations if applicable.

Table 2

Parameter values for the model development

Variable name coded in R	Value	Description
Costs
cPFS	500	State cost of one cycle in the progression-free disease state
cPD	3000	State cost of one cycle in the progressive disease state
cDrug	1000	State cost of drug for one cycle
cTest	100	State cost of biomarker testing for one cycle
cDead	0	State cost of one cycle in the death
Quality-of-life adjustments
uPFS.UC	0.75	Quality-of-life weight for one cycle in PFS for patients treated with usual care
uPD.UC	0.65	Quality-of-life weight for one cycle in PD for patients treated with usual care
uPFS.TC	0.80	Quality-of-life weight for one cycle in PFS for patients treated with targeted care
uPD.TC	0.70	Quality-of-life weight for one cycle in PD for patients treated with targeted care
Biomarker-related parameters
disutility.Test	0.05	Disutility value of testing a biomarker status
pBiomarker	0.74	Biomarker prevalence/frequency
tp	0.285	Biomarker testing accuracy (true positive)
fp	0.245	Biomarker testing accuracy (false positive)
tn	0.015	Biomarker testing accuracy (true negative)
fn	0.455	Biomarker testing accuracy (false negative)
Transition probabilities
pPFS2PD	0.2	Probability of entering the PD state
pPD2D	0.25	Probability of dying from PD
pPFS2D	0.05	Probability of dying from PFS
pPD2PFS	0	Recovery from PD to PFS is not permitted in the model
Other parameters
eff	0.25	Targeted drug reduces the likelihood of being progressed by 25% Relative risk of disease progression from using the drug Targeted drug is discontinued upon progression
rDiscount	0.035	Discount rate for outcomes and costs 3.5%

PFS progression-free survival, PD progressed disease

Uncertainty Analysis

Uncertainty analysis is a standard practice in modeling studies to assess the uncertainties around parameters and assumptions used in the model. Both deterministic sensitivity analyses (DSA) and probabilistic sensitivity analyses (PSA) are performed in this practical model in order to assess the impact of parameter uncertainty on the cost-effectiveness results. DSA is performed to test the sensitivity of the results of cost effectiveness to specific parameter values. As for PSA, all parameters are simultaneously tested for uncertainty while randomly sampling the parameter values from a priori-defined probability distributions [28].

Step-by-Step Guide

Figure 3 is an overall picture of the steps involved to perform cost-effectiveness analysis of companion biomarkers for targeted therapies in R. Note that it is a general guidance and thus some specific adjustments might be required depending on the country-specific clinical settings or country-specific HTA requirements. It should also be decision problem specific. An explanation of each step is provided. More detailed R codes are provided in electronic supplementary material (ESM) 1–7. R codes can be self-explanatory, with some notes written in italics with the # symbol, which can be useful when the codes are copied and pasted in R; however, a basic understanding of R is required in order to follow this guide. This modeling guide is not intended for complete R beginners. As a basic note, the <- symbol is to assign values in R.

Fig. 3

Algorithm steps in performing cost-effectiveness analysis for cancer biomarkers for targeted therapies in R. UC usual care, TC targeted care

Step 1: Create Transition Probability Matrices

This step is to prepare the transition probability matrix per strategy arm. Before this step, it is necessary to first decide which model is suitable, such as a Markov or semi-Markov model, etc., as shown in Fig. 1. The core model presented here is constructed based on a state-transition model. In order to construct the probability matrices, parameter values exampled in Table 2 need to be assigned to R first. It can then create the transition matrix of each health state per strategy arm. Refer to ESM 1 for the entire R code for Step 1.

Step 2: Create Cost and Utility Transition Matrices

Similarly, the transition matrices for cost and utility values can be prepared using the Matrix command in R. This step is similar to Step 1 in a sense that model inputs are defined and vectored into the R model. The detailed R code for Step 2 is provided in ESM 2.

Step 3: Building a Markov Model for the ‘All Usual Care’ Arm

Based on the transition matrices set up in Step 1, a Markov trace with a hypothetical cohort of patients, summing up over time, needs to be constructed. In this stage, all different scenarios of treatment pathways by different strategy and testing results should be constructed, as depicted in Fig. 1 and Fig. 2. Thus, biomarker-related data need to be defined and vectored into the R model including biomarker testing accuracy and biomarker prevalence. Overall, 1000 cycles were assigned to capture the lifetime horizon of all patients entered in the model, with one cycle being equivalent to 1 month in this core model. In other words, 1000 cycles are equivalent to 83.33 years, which is long enough to simulate the model in a lifetime horizon. Depending on the progression of the disease of interest, the cycle can be shortened or lengthened. These model settings can be easily altered according to local adaptation requirements. The R code for this Step 3 is detailed in ESM 3.

Step 4: Adapting the Model for the ‘All Targeted Care’ and ‘Test–Treat’ Arms

This stage is relatively simple. The R code used for the all-UC arm in Step 3 can be easily modified and adapted for both the all-TC and TT arms. This feature is one of the advantages of using a script-based program when building health economic models. It can be easily transformed and adapted for other strategy arms with a relatively small amount of time and effort dedicated. The cohort trace of patients in the all-TC arm needs to be separated into two branches depending on their actual biomarker status, because all patients will be treated with biomarker-guided therapy; however, some patients might not be biomarker-positive and thus the targeted therapy will not be effective for these patients. The cohort trace for patients in the TT arm needs to be separately constructed for patients who truly tested positive, falsely tested positive, truly tested negative, or falsely tested negative. Cohort simulation commences with a hypothetical cohort of patients (in this core model, it is set at 1000, which means a hypothetical cohort of 1000 patients started the model). These patients then move or stay in the possible health state according to the transition probabilities defined by different treatment scenarios of the strategy arms. The simulation tracks the cohort from one cycle to the next following the transition probabilities. Refer to ESM 4 for the detailed R code for Step 4.

Step 5: Computing Epidemiological Outcomes

Epidemiological outcomes of different health states can be computed and plotted in a graph using the R code written in ESM 5. Respective cohort traces per strategy arm can be plotted as survival curves. For the all-TC and TT arms, the cohort transition matrices need to be merged before plotting the survival curves. Overall survival (OS) probability can be separately computed and plotted in the OS curve according to different strategy arms. Life expectancy can be calculated by summing the probability of OS over time.

Step 6. Estimating the Base-Case Cost Effectiveness

We are now ready to perform the analysis and estimate the expected values and cost effectiveness. In R, the expected values of each strategy can be calculated by processing the multiplication of the Markov trace produced in Steps 3 and 4 and the transition matrices of the cost and utility inputs produced in Step 2. The R code for Step 6 is provided in ESM 6, with self-explanatory comments shown in italics. The example base-case ICER calculated for this exercise is also provided in ESM 6.

Step 7: Performing Sensitivity Analyses

Uncertainty analysis is a standard practice in modeling studies to assess the uncertainties around parameters and assumptions used in the model. ESM 7 explains how to perform PSA using R as an integral part of the uncertainty analysis of cancer biomarkers for targeted therapies in addition to the detailed R code. In R, the model can be run by the function defined by the modeler (R is known to be extremely flexible in this regard) and we define all parameters in the run_model function. DSA modeling is similar to that of PSA (refer to the supplementary R code for DSA in ESM 7).

In addition to this parameter uncertainty, an analysis of structural uncertainty can be performed by adapting this core model. For example, the structure of health states can be readily altered considering the natural course of disease progression of interest for local adaptation.

Discussion

This paper introduced a core model that can be adaptable for users’ analysis of their specific datasets and requirements in assessing the value of cancer biomarkers. This guide demonstrated the model structure of strategy comparisons and data requirements relevant to the characteristics of companion biomarker testing that require incorporation into the health economic modeling of biomarker-targeted therapies. Users can vector in their specific data inputs instead of dummy variables used in the worked example (Steps 1, 2 and 3). In addition, those wanting to make any structural changes, such as health states or strategy arms, can readily replicate the process of creating the transition matrices by modifying (adding/removing) the matrices of transition probability, cost, and utility values in Steps 1–2. Nevertheless, we expect the user to have a prior understanding of, and experience with, R coding in order to understand this tutorial and adapt this core model for their specific local model. For any beginners using R, there are useful references to familiarize themselves first [29, 30]. For those who need a more generic understanding on health economic modeling in R that is non-specific to certain health technologies, such as companion biomarker tests as focused on in this tutorial, a tutorial paper is available that they might like to read first [31].

As found in a previous study [10], many existing economic evaluations of biomarker-guided therapies have ignored several key characteristics of companion biomarkers when assessing the value for money of biomarker-guided therapies. They found that the most frequently ignored areas were patient preferences, clinical utility, and prevalence of biomarker status, while the costs of biomarker testing were considered. For example, it can be very difficult to generate data inputs on the clinical value of companion biomarker tests if the biomarker test is developed in silos without being embedded in the clinical trials of its corresponding drug. This tutorial informs readers on what key data inputs relevant to companion biomarker tests are required to be incorporated and tested for uncertainty in economic evaluations of the guided therapies.

Furthermore, given that a single clinical study is unlikely to include all relevant scenarios of the TT, all-UC and all-TT arms, data synthesis from multiple sources might be more challenging for co-dependent technologies (i.e. biomarker-guided therapies) than traditional health technologies. Nevertheless, it is widely known that all relevant data from different sources of clinical trials should be systematically synthesized and incorporated in economic evaluations. When direct evidence (i.e. head-to-head trials) is lacking, network meta-analysis or indirect treatment comparison methods are useful to synthesize the estimates of clinical trials. These methods of data synthesis for economic evaluations are beyond the scope of this tutorial and have been described extensively elsewhere, including uncertainty analyses around the impact of synthesis-based estimates on cost effectiveness [18, 22, 23, 32, 33].

This paper has also provided a step-by-step guide to carrying out cost-effectiveness analysis for biomarker-guided therapies in the state-transition modeling framework, and has provided R codes in vectoring data inputs, running the simulations, performing survival analyses, calculating base-case mean LYs/QALYs and performing sensitivity analyses. The user can adapt this core model to develop their own local model applied to their specific cancer biomarker testing technology and specific jurisdiction of reimbursement decision making. Alternatively, test developers can assess the potential value for money of their candidate cancer biomarker tests at an early stage of development by incorporating the pertinent model inputs with necessary adaptations and modifications to this core model. For example, the user can adapt the structure of health states, the strategy arms to be compared against one another, transition probabilities, biomarker-specific characteristics, and cost and utility values, etc. However, for those who need to reconstruct time-to-event data from published Kaplan–Meier survival curves as part of building health economic models in R, two useful tutorial papers providing algorithms are available for the user to use [34, 35].

There are a couple of limitations that readers might wish to take into consideration when adapting this model for their local models. First, this core model is constructed based on a state-transition model and therefore, those wishing to build a partitioned survival model (PSM) may require more time to adapt. However, PSM does not require as many of the modeling techniques as used in the state-transition model. Second, the user is required to have some understanding of the concepts of economic evaluations and HTA, as well as knowledge of programing in R. Therefore, there is still a programing language barrier for test developers to adapt or apply this core model to their data and requirements if the user is not familiar with cost-effectiveness analysis and R coding. It requires some intermediate level of R programing/coding and a conceptual understanding of economic evaluations of health technologies. Third, guiding on how to validate a model was not covered by this guide because this study intends to provide a step-by-step guide on how to build a model of co-dependent technologies rather than providing a guide to the validation of a specific model. Furthermore, this core model is built using ‘exemplary’ data inputs (not real dataset) and thus, as Eddy et al. suggested in the model validation [36], the concept of validity should apply to particular applications, not to the model itself. Therefore, modellers wishing to adapt this core model to their local settings with a specific dataset (e.g. ‘real’ data inputs from clinical trials) and assumptions applied to their specific decision problems, the process of model validation should be accompanied in their local adaptation model. Several guidances and checklists have been published on good practices of model validation [36-38].

A couple of areas can be recommended for further development of this core model. First, although the R codes provided in this guide are verified by running the model in R, it was not tested to what extent this model can be applicable to actual datasets. By applying this core model to the published economic evaluations of biomarker-guided therapies, the generalizability of this model can be further validated. By doing so might give more insights into how and under what circumstances this core model is adaptable, difficult to adapt, or unadaptable at all. Second, this core model can be further developed to make it easily accessible to those unfamiliar with R. For example, the model can be further developed to user-friendly, interface, web-based applications using the Shiny R package, as undertaken by Strong et al. in assessing the value of information [39].

Below is the link to the electronic supplementary material.

Supplementary file1 (TXT 28 kb)

No clear methods guidance exists on how to model companion testing technologies as part of economic evaluations of biomarker-guided therapies. Few countries provide the health economic evaluation methods guide specific to co-dependent technologies such as companion diagnostics and biomarker-guided therapies.

This tutorial article provides a step-by-step guide on constructing a health economic model to assess the value for money of biomarker-guided therapies. A core model was developed as part of the worked examples of this tutorial, using R. Users can readily adapt the core model, with appropriate adjustments to data inputs and model structure.

This tutorial can also inform users of relevant data inputs of companion biomarker tests required to incorporate in economic evaluations of biomarker-guided therapies before designing studies/trials for data collection.