Improving clinical reality in chronic obstructive pulmonary disease economic modelling : development and validation of a micro-simulation approach.

BACKGROUND: Chronic obstructive pulmonary disease (COPD) is a progressive and irreversible disease responsible for the deaths of 3 million people worldwide in 2005, and predicted to be the third leading cause of death worldwide by 2030. Many COPD models developed to date have followed a Markov structure, in which patients or populations can move between defined health states over successive time periods or cycles. In COPD, health states are typically based on disease severity defined solely by lung function, as described by the Global Initiative for Chronic Obstructive Lung Disease (GOLD) guidelines. These current modelling methods may restrict the ability to reflect the disease progression/clinical pathway or clinical practice.
OBJECTIVES: Given these limitations in previous COPD models, the authors aimed to develop a more flexible model that could improve on the description of the clinical disease pathway. The overall objective of this model was to inform the development of policies, guidelines or cost-effectiveness analyses. A second objective was to validate the model in relation to existing epidemiology studies of COPD.
METHODS: A patient simulation model was developed in Microsoft Excel™. The predictability of the model was tested by populating it with data from natural history of disease studies as well as with clinical trial data. Each patient moves through the model with demographic characteristics randomly generated from a set distribution. These characteristics determine the risk of clinical events occurring in the model.
RESULTS: The validation with these studies found the model to have generally good predictive ability, yielding in this way a good degree of external validity.
CONCLUSIONS: The micro-simulation model is a flexible approach for modelling COPD that allows consideration of complex COPD treatment pathways. The model was found to be generally robust in terms of predicting clinical outcomes of published studies when tested against other studies. It has significant potential as a tool for supporting future COPD treatment positioning decisions as well as to inform the development of policies, guidelines or cost-effectiveness analyses.

Key Points for Decision Makers

• This new model offers flexibility in simulating complex chronic obstructive pulmonary disease (COPD) treatment pathways

• The model clinical predictability was validated using published COPD studies and found to be acceptable

• Treatment switching following specific events is a key functionality of the model

• This model has potential as a tool for supporting future COPD treatment positioning decisions as well as to inform the development of policies, guidelines or cost-effectiveness analyses

Introduction

Chronic obstructive pulmonary disease (COPD) is a progressive and irreversible disease predicted to be the third leading cause of death worldwide by 2030 [1]. COPD is caused by the inhalation of noxious gases including cigarette smoke in combination with host factors and is associated with limitation of airflow symptoms such as breathlessness and cough, as well as poor health-related quality of life (HR-QOL) [2]. A subset of COPD patients is also susceptible to exacerbations, which may require a change in maintenance medication, provision of emergency treatment or a hospital inpatient stay. COPD results in a social and economic burden to society and often impacts in later working life [3]. Current clinical guidelines for COPD indicate a standard treatment pathway according to disease progression [2, 4]. The treatment options available nowadays are increasing with inhaled therapies, alone or in combination, having a preponderant role. A thorough understanding of treatment sequences and therapy combinations is therefore essential when considering the best treatment choice for a COPD patient. However, given the current scarcity of resources in most healthcare systems, it is also imperative that the treatment options are value for money (cost effective) for the payers.

When examining the published literature on the cost effectiveness of COPD treatments, the majority of studies identified perform pharmacoeconomic evaluations alongside clinical trials, using effectiveness and cost data collected prospectively over the course of the trial [5, 6]. In a systematic review published in 2008, ten out of the 15 studies identified were evaluations alongside clinical trials and five were based on economic models [6]. While randomized controlled trials (RCTs) are considered the gold standard for the evaluation of clinical efficacy [7], the generalizability of an economic evaluation alongside an RCT to real-life clinical practice may be limited due to the choice of comparator and the availability of data over a long period of time (i.e. study duration). Buxton et al. [8] discuss the limitations of trial-based evaluation versus the dangers of modelling but acknowledge that models allow for the extrapolation of data when only short-term data exist, the linking of intermediate trial endpoints to long-term outcomes of interest to the payer, making adjustment to make the evaluation more generalizable to other settings, and the incorporation of other comparator data not included in trials. Economic models are useful alternatives when RCTs do not provide all the information necessary; however, models should endeavour to be as simple as possible while still meeting the stated objective.

Some economic evaluations have used models with observational data incorporated into them to increase the applicability to the real world [9]; however, the choice of model structure also imposes limitations on how the data can be used and therefore on the generalizability of the model to clinical practice.

Most of the COPD models to date follow a Markov structure, in which patients or populations can move between defined health states over successive time periods or cycles [6, 10]. In COPD, health states are typically based on disease severity defined solely by lung function, as described by the Global Initiative for Chronic Obstructive Lung Disease (GOLD) guidelines [2]. These current modelling methods may restrict the ability to reflect the disease progression/clinical pathway or clinical practice. This is related firstly to the memory-less nature of Markov modelling and secondly to the necessity to construct discrete health states for each event or combination of events if it impacts the expected pay-off in a meaningful way. Markov models do not have the ability to retain memory of a patient’s previous health states without the introduction of additional health states or tunnel states [11]. Most models currently lack the ability to describe a change in treatment [12-16], that is, a treatment switch or re-initiation of therapy for a patient who has discontinued therapy without the addition of more health states to describe what should be a similar clinical pathway but with slightly different risks of clinical events occurring. Moreover, designing a model with more complex treatment pathways may encounter a lack of appropriate data on transition probabilities between health states and costs. Defining health states based on GOLD criteria is also often problematic as several clinical studies [17, 18] base their endpoint measurements on pre-bronchodilator measurements, rather than post-bronchodilator measurements as prescribed by GOLD once past the screening stage; this limits the applicability of these data in a model.

Some studies have looked at alternative ways to model COPD. The most common of these is to apply regression techniques to predict the occurrence of specific clinical events that are significant. In the case of COPD, exacerbations are clear candidates for such an event due to their cost and impact on patient quality of life (QOL). Aaron et al. [19] and Mapel et al. [20] have both developed such methods allowing for the prediction of recurrent exacerbations. Mapel et al. [20] go one step further to apply the regression model to the context of an economic evaluation, arguing that the modelling of exacerbation data based on the healthcare utilization data is sufficient for cost-effectiveness analyses. Treatments that reduce the risk or severity of an exacerbation are likely to be shown to be cost effective, given the significant cost of treating a severe exacerbation. However, such a regression model does not take into account other events that may be impacted by treatment, such as lung function decline or impact on symptoms such as dyspnoea.

With these previous limitations in mind, the authors have developed an economic model incorporating micro-simulation techniques that will allow modelling a single patient at a time, generating clinical events and treatment switches according to preset probabilities and triggers. This type of model allows for the occurrence of any number of events such as exacerbations, decline in lung function, worsening of dyspnoea (as measured by clinical instruments) and improvement in QOL (as measured by disease-specific or generic instruments). Following the taxonomy described by Brennan et al. [21], this model may be considered an individual sampling model (ISM) that simulates each individual through discrete time periods. The major difference to previous COPD models is the absence of defined health states per se. Outcomes are estimated on the basis of the presence/absence of events and lung function levels modelled as a continuous variable. These changes increase the flexibility with which to describe the different combinations of clinical events that can be experienced by a patient over the course of their disease. GOLD state definitions are assigned whenever estimating outcomes that are related to disease severity, but there is potential to avoid using these state definitions if outcomes and pay-offs can be estimated independently of GOLD disease severity.

Objectives

The objective of this study was to develop a COPD model incorporating simulation techniques in an attempt to address the limitations mentioned in Sect. 1, with the purpose of informing the development of policies, guidelines or cost-effectiveness analyses. A second objective was to validate the model in relation to existing epidemiology studies of COPD.

Methods

Model Structure

This simulation-based model combines information derived from the natural history of COPD studies, clinical data and costs. The model is built in Microsoft Excel™ and follows a patient according to any predefined time horizon, set by default to lifetime to capture all relevant outcomes. The patient progresses through the model using Monte Carlo simulation methods at each cycle. A disease severity level is assigned at each cycle to estimate the probabilities of certain clinical outcomes. The disease severity will also influence some of the model’s pay-offs such as HR-QOL and costs.

The clinical profile (baseline characteristics) of a patient at the start of the model will have a determinant impact on the final outcomes of the model as these will determine the disease severity group in which the patient belongs at the start and in subsequent cycles by applying a rate of decline in lung function. The GOLD disease severity classification was used as it is current practice to collect this information in clinical trials and most other outcomes are presented in the literature by these disease severity levels. However, as stated earlier, if outcomes and pay-offs can be estimated without reference to GOLD disease severity, the need for assigning GOLD states becomes unnecessary.

In order to determine the lung function of a patient at the start of the model, some of the baseline patient characteristics are assumed to be correlated. In order to understand and quantify the correlation between these baseline characteristics, patient-level data from three multinational, multi-centre, randomized, double-blind, placebo-controlled studies evaluating the efficacy of indacaterol in over 3,800 patients with moderate to severe COPD were made available [22-24]. A correlation matrix was constructed employing the Cholesky decomposition method on this pooled dataset [25]. The distribution of each individual variable was tested and all continuous variables were found to be Normally distributed. The following variables were regressed against each other to test for correlations: forced expiratory volume in 1 second (FEV1)% predicted, age, gender, height and smoking status. The co-variance matrix linked gender and age to height, smoking status and lung function in order to generate the FEV1 value at the start of each simulation for each patient.

The generated patient progresses through the model and experiences clinical events according to preset probabilities. The model employs Monte Carlo simulation methods to follow a patient through the model as dictated by probabilities of developing complications, being symptomatic or progressing, taking into account his, or her, current treatment status. These include the rate of FEV1 decline, the exacerbation rate, the discontinuation rate, and changes in Baseline and Transition Dyspnoea Index (BDI-TDI) and the St. George’s Respiratory Questionnaire (SGRQ) scores. These clinical events may trigger a treatment switch, which can be specified at the start of the analysis by the user to a maximum of three criteria. For example, the user can specify a treatment switch after the patient has experienced two exacerbations over the course of the last year.

Progression through the model is captured in discrete cycles, the length of which may be adjusted by the user. Events are generated in a hierarchical fashion such that a patient is first tested against a probability of death, followed by discontinuation of treatment. Smoking status (i.e. smoker, intermittent smoker or ex-smoker) determines current level of lung function, which in turn determines the probabilities of experiencing exacerbations and changes in other measures.

Analyses are run in cohorts of patients and both the number of patients per cohort and the number of cohorts evaluated are determined by the user. Figure 1 describes the model in a schematic diagram.

Fig. 1

Model schematic. FEV forced expiratory volume in 1 second, SGRQ St. George’s Respiratory Questionnaire, TDI Transition Dyspnoea Index

Model Parameters

Time Horizon and Discounting

The model is designed to use any cycle length as long as it is defined in monthly units. The user has the possibility of setting both the cycle length and the time horizon as needed. For time horizons longer than 1 year, the user can define discount rates for benefits and costs, which can be selected as appropriate for the base-case analysis, sensitivity analyses or to comply with different local recommendations.

Number of Patients and Number of Cohorts

The default settings are defined as 1,000 patients for each of 500 cohorts: these values were selected for robustness and accuracy of the results, but can be changed by the user.

Interventions

The initial treatment received by the patient is chosen by the user. The treatment can be compared with an alternative treatment, placebo or a ‘no treatment’ option. All other treatments switches, including discontinuations, will be determined by patient symptoms as manifested by events. This decision rule can be set by the user in terms of defining the numbers of events and treatment choices following the fulfilment of those criteria. Withdrawal rates can also be specified by the user according to the data available or assumptions.

Model Inputs

Lung Function Improvement

As mentioned in Sect. 3.1, FEV1 level at baseline is determined via a co-variance matrix. The initial treatment effect is specified by the user according to data availability or assumptions. Default settings are based on improvement in FEV1 observed in the indacaterol phase III trials [22-24]; other RCT sources or meta-analyses may also be considered as a data source for this input. Data available from published studies were used in the validation process [17, 18, 26, 36]. Patients who discontinued treatment lose the benefit experienced by falling back to the expected level of FEV1 as if the patient had remained untreated. FEV1 declines at every cycle are set by default with known epidemiological data from the Lung Health Study [26]. However, the user is able to change the FEV1 rate of decline, again according to data availability or assumptions.

Exacerbations

A non-severe exacerbation in the model was described as one where the patient has had contact with a healthcare provider. A severe exacerbation was defined as any exacerbation requiring hospitalization. Mild exacerbations managed by patients were not included in the model. Probabilities of exacerbation are determined by severity of disease and treatment choice. Ratios of exacerbation rates were based on the indacaterol phase III trials as a default [22-24] or from published data when available (e.g. UPLIFT [Understanding Potential Long-term Impacts on Function with Tiotropium] [18]). Conservative assumptions were made when comparative data were not known. Disease severity is determined by FEV1 status as described by the GOLD guidelines. Exacerbations—either severe or non-severe—are defined in annual rates, which in turn are adjusted to the specified cycle length. The treatment choice informs the rate ratio that should be applied to the baseline rate to reflect the treatment effect of each intervention.

Mortality

Probabilities of death are determined by severity of disease [27] and general all-cause mortality. The mortality rates (Table 1) are defined as ‘excess’ mortality rates found for COPD patients in The Netherlands and may include other smoking-related causes of death [27]. The probabilities of death presented here are annual and get adjusted to the model cycle length. There is research that suggests that determinants of death in COPD patients may also be found in other clinical indices (i.e. other than COPD disease severity indices), but this requires further research [28]. Another source of COPD excess mortality was available, also by GOLD disease severity, which indicated a much lower overall mortality rate [30]. Both sources were tested and both were found to have limitations but Hoogendoorn’s data [27] are used as default and are presented in Table 1. General all-cause mortality data were derived using UK interim life tables from the Office of National Statistics [29]. These can also be adapted according to the user data or setting.

Table 1

Yearly incremental death rate by disease severity [27]

Disease severity [GOLD]	Incremental annual rate of death (%)
Mild (stage I)	2.25
Moderate (stage II)	3.56
Severe (stage III)	5.42
Very severe (stage IV)	7.74

GOLD Global Initiative for Chronic Obstructive Lung Disease

Transition Dyspnoea Index

Dyspnoea is a primary symptom of COPD and has been defined as an uncomfortable awareness of breathing or an increased respiratory effort [31]. The Transition Dyspnoea Index (TDI) is an instrument used to assess breathlessness and the impact of an intervention [32].

TDI scores are dependent on treatment and on the period of time on each treatment. A change in TDI can be selected as a trigger for switching treatment. The baseline assumption is that one unit decline in TDI is clinically significant and will result in a switch of patient medication.

Health-Related Quality of Life

HR-QOL is assessed using both generic and COPD-specific measures. Health status is represented by a specific measure of HR-QOL: the SGRQ. The SGRQ is designed to measure health impairment in patients with asthma and COPD [33]. From a clinical point of view, a significant change in health status can lead to treatment switching. The user can therefore select a change in SGRQ as a criterion for switching in the Control sheet. It was assumed that a worsening of four units in SGRQ (i.e. +4, since the higher the SGRQ, the worse the health status) can trigger a change in patient medication. SGRQ scores increase with time and this was reflected in the current assumptions.

The model is also designed to be populated with generic measures of HR-QOL such as utilities based on the EQ-5D or the SF-6D. Baseline utilities associated with each disease severity state are applied at each cycle. Whenever a patient experiences an exacerbation, a utility decrement is subtracted from the baseline utility value corresponding to the cycle where the event had occurred. An algorithm that predicts EQ-5D utility from the SGRQ was recently developed [34] to allow the use of utilities from the SGRQ.

Costs

Costs of drugs, exacerbations and maintenance are set by the user according to the setting of interest. In each cycle, the cost of maintenance is determined by disease severity. Exacerbation costs are added if a patient experiences an exacerbation episode within that cycle. It is assumed that there are no other costs associated with symptoms occurring to patients. The model adopts the healthcare payer’s perspective, therefore indirect costs are not considered.

Model Outputs

The model produces three sets of outputs: clinical, cost and cost effectiveness. These will help to inform decisions regarding the adoption of competing interventions by healthcare payers. Although these decisions are not based on cost-effectiveness criteria only, an intervention is considered to be cost effective when the incremental cost-effectiveness ratio is below a predefined accepted threshold representing the willingness to pay by the healthcare payer. This threshold is usually defined as a cost per measure of effectiveness. The decision rules for economic evaluation are described elsewhere [35].

A summary of model inputs and outputs is available in Table 2.

Table 2

Summary of the model’s inputs and outputs

	Clinical	Cost	Parameters
Model inputs	FEV1 boost/improvement Exacerbations Mortality TDI SGRQ Withdrawals HR-QOL	Drugs Exacerbations Maintenance	Time horizon Cycle length Discount rate Number of patients Number of cohorts to be run
Model outputs	Number of days in each disease severity Number of exacerbations (non-severe and severe) Average SGRQ score in each arm Mean difference of FEV1 Annual FEV1 decline rate QALYs	Total and average drug costs Exacerbations Maintenance costs	NA
Cost-effectiveness outputs	ICER ICUR Cost-effectiveness plane		NA

FEV forced expiratory volume in 1 second, HR-QOL health-related quality of life, ICER incremental cost-effectiveness ratio, ICUR incremental cost-utility ratio, NA not applicable, SGRQ St. George’s Respiratory Questionnaire, TDI Transition Dyspnoea Index

Uncertainty

In order to characterize parameter uncertainty (or second-order uncertainty), probability distributions were fitted to some of the model’s inputs. Gamma distributions were fitted to treatment efficacy (initial FEV1 boost) and to cost parameters; a log-Normal distribution was fitted to exacerbation parameters; and, finally, Beta distributions were fitted to utilities, withdrawals and mortality. The merits of assigning these distributions to some of the model’s parameters are described elsewhere [25]. Ninety-five percent confidence intervals (CIs) are presented around the results to represent this uncertainty. This also allows for the development of cost-effectiveness acceptability curves (CEAC) but the current software format emphasizes transparency rather than speed of computations; therefore, the inclusion of a sufficient number of iterations would require a correspondingly large amount of time.

Model Validation

The model was subjected to a comprehensive validation process. Firstly, it was tested to ensure that all mathematical calculations were accurate and consistent with its specifications. Secondly, the model was tested by being populated with four datasets, three of which were from clinical trials [17, 18, 26] and one a natural history of disease study [36]. Our aim was to assess if the model was able to replicate the results of these studies with an acceptable degree of accuracy, ensuring internal and external validity. Table 3 shows the inputs extracted from the four published studies used to assess the model validity [17, 18, 26, 36].

Table 3

Value input of studies used in the validation process

Variable	UPLIFT [18]	Lung Health Study [26]	TORCH [17]	OLIN [36]
Mean ± SD age at baseline (years)	64.5 ± 8.4	48.5 ± 6.8a	65 ± 8.2	56.8 ± 10.7b
Cycle length (months)	6	12	6	12
Time horizonc (years)	4	5	3	20
Treatments	Tiotropium vs. PL	NA	Salmeterol/fluticasone vs. PL	NA
Males (%)	75	63.85	75	64
Mean FEV1 vs. PL (ml)	56d	NA	75e	NA
FEV1% of predicted	NA	78	NA	70.7
Rate of FEV1 decline (ml/year)	NA	Smokers: 62 Ex-smokers: 31 Intermittent smokers: 43	NA	NA
GOLD distribution (%)	Stage I: 0 Stage II: 46 Stage III: 44 Stage IV: 9	NA	NA	Stage I: 36 Stage II: 50 Stage III: 9 Stage IV: 5
RR of exacerbations vs. PL	0.86	NA	NA	NA
Smokers (%)	NA	NA	43	NA
Withdrawal per cycle (%)	NA	NA	Salmeterol/fluticasone pts: 5.6 PL pts: 7.3	NA

FEV forced expiratory volume in 1 second, GOLD Global Initiative for Chronic Obstructive Lung Disease, NA not available, PL placebo, pts patients, RR relative risk, SD standard deviation

aThe SD is estimated from available sub-group data [26]

bThe mean age and SD are estimated from available age intervals [36]

cEquivalent to study duration

dThe mean absolute improvement in FEV1 was determined by the average of the mean improvements from both groups: 47–65 ml after bronchodilation [18]

eThe mean FEV1 improvement was estimated from the graph in the publication [17]

The parameters tested varied by the availability of information in each publication, but focused on clinical parameters such as mortality and exacerbation rates, as well as FEV1 decline. The model benefited from the clinical and methodological scrutiny of key opinion leaders in order to ensure not only that the model was consistent and coherent with the clinical pathway of COPD but also to certify that the methods used were robust. For the validation exercise, the micro-simulation model was run for 500 patients and 50 cohorts.

Results

The results of the model obtained by simulating the four studies subsequently are presented in this section.

UPLIFT

UPLIFT is the longest running and largest RCT to date in a COPD population [18]. It compared 4 years of therapy with either tiotropium or placebo in moderate to very severe patients with COPD. The primary endpoints were the rate of decline in the mean FEV1 before and after bronchodilation beginning on day 30. Secondary endpoints included measures of forced vital capacity (FVC), changes in response on SGRQ, exacerbations of COPD and mortality [18].

The co-variance matrix in the model predicted accurately the smoking status of the patients, although CIs were not available for comparison in the original publication [18] (Table 4). Estimates of relative risk (RR) of exacerbations from the model fell within the CI of the study. The model overestimated the annual FEV1 decline rate, which could be attributed to the fact that around 10 % of the UPLIFT trial population were very severe patients. This group typically has a lower rate of decline than patients in other GOLD stages.1 The estimated mortality RR fell within the CI of the published results [18], but the absolute rates of both arms are slightly overestimated. Treatment benefit in mortality is not as well captured when using mortality rates associated with GOLD states, as this only allows for a difference in mortality rate when the lung function improves sufficiently for a GOLD state change. Alternative possibilities include mapping the change in mortality risk directly to other clinical indicators, such as lung function decline or change in TDI, or related to exacerbations, which could be made possible with sufficiently long-term data analysed at the patient level.

Table 4

Results of external validation with UPLIFT [18]

Key results	Published results [18]	Model results
Smoking status [% of current smokers (95 % CI)]	29 (CI unpublished)	29.35 (28.76–29.94)
RR of exacerbations per patient-year of tiotropium vs. PL (95 % CI)	0.86 (0.81–0.91) Leading to hospitalization: 0.94 (0.82–1.07)	0.85 (0.84–0.85) Leading to hospitalization: 0.84 (0.82–0.85)
Annual FEV1 decline [ml]	Tiotropium (± SE): 40 (±1) PL (± SE): 42 ml (±1)	Tiotropium: 44.3 (95 % CI 44.0–44.6) PL: 43.2 (95 % CI 42.8–43.5)
Mortality	Tiotropium: 14.4 % PL: 16.3 % RR 0.87 (95 % CI 0.76–0.99)	Tiotropium: 17.63 % PL: 17.92 % RR 0.98 (95 % CI 0.98–0.99)

CI confidence interval, FEV forced expiratory volume in 1 second, PL placebo, RR relative risk

Lung Health Study

The Lung Health Study was a clinical trial that assessed the benefits of smoking cessation over a time horizon of 5 years [26].

The model was built originally with active treatment arms and a placebo arm only. When this validation was conducted using the Lung Health Study, it was observed that the modelled annual mean rate of FEV1 decline was overestimated. This was attributed to an initial FEV1 boost in the placebo arm based on clinical trial data [18, 22], which cause patients discontinuing from placebo to have a significant drop in their FEV1 in line with the assumption described in Sect. 3.3.1 of loss of treatment benefit upon treatment termination. In order to address this paradigm, a no treatment arm was added where it was assumed no initial benefit in FEV1 would be observed and patients would only be subject to a constant rate of FEV1 decline.

The model was re-run and it was observed that the mean rates of decline obtained after running the model were aligned with mean rates of decline published by Scanlon et al. [26], which highlights that the inclusion of a no treatment arm is appropriate and it is correctly used in the model (61.54 ml/year in smokers and 31.22 ml/year in ex-smokers obtained from the model vs. 62 ml/year in smokers and 31 ml/year in ex-smokers from the study publication [26]). These rates are used in the base case of the model and are not affected when no treatment is selected.

TORCH

TORCH (TOwards a Revolution in COPD Health) was a randomized, double-blind, placebo-controlled trial where salmeterol and fluticasone in combination was compared with placebo in moderate to very severe COPD patients [17]. The length of follow-up was 3 years and the primary endpoint was death from any cause. Secondary endpoints included frequency of exacerbations, health status using the SGRQ and spirometric values [17].

The correlation matrix was broken to allow for a higher proportion of smokers as this had an impact on the rate of FEV1 decline. Table 5 shows the results obtained in the model when implementing the TORCH trial data [17].

Table 5

Results of external validation with TORCH [17]

Key results	Published results [17]	Model results
Probability of death from any cause in 3 years (%)	Combination: 12.59 PL: 15.16	Combination: 17.90 PL: 17.99
Annual rate of exacerbations per patient	Combination: 0.85 PL: 1.13 RR 0.75 (95 % CI 0.69–0.81)	Combination: 0.74 PL: 1.05 RR 0.71 (95 % CI 0.709–0.710)
Adjusted mean change in FEV1 averaged over 3 years of combination therapy vs. PL [litre (95 % CI)]	0.092 (0.075–0.108); p < 0.001	0.067 (0.064–0.069)
Adjusted mean change in SGRQ score averaged over 3 years [units (95 % CI)]	Combination: −3.0 PL: +0.2	Combination: −2.10 (−2.08 to −2.12) PL: −1.54 (−1.53 to −1.54)

CI confidence interval, FEV forced expiratory volume in 1 second, PL placebo, RR relative risk, SGRQ St. George’s Respiratory Questionnaire

The RRs of experiencing an exacerbation are comparable between the model and the trial publication [17] (i.e. 0.72 vs. 0.75, respectively). However, absolute rates observed in the model were slightly lower than those reported in the publication [17]. This could be due to the baseline rate of exacerbations in the source indacaterol trials [22-24] being extremely low, and could be adjusted with other data sources.

The average mean change of FEV1 over the model time horizon (3 years) was smaller than that reported in the study [17]. This difference can be explained by the trial patients receiving placebo experiencing a negligible FEV1 improvement.2 However, this value was not adjusted for placebo patients in the model, who were given a 130 ml improvement in their first cycle. This led to an increase in FEV1 change between the comparators in TORCH [17] compared to that observed in the model. Although possible, the use of the no treatment arm in the model was not considered appropriate given that the TORCH trial uses placebo as a comparator.

The mortality rates obtained in the model were higher than those in the published data [17]. Distribution of GOLD states in the TORCH study was not available, but the mean FEV1% predicted in the population was 44 %, higher than the mean in UPLIFT of 39 %, indicating a slightly less severe population than UPLIFT [17, 18]. The current data source for mortality used may overestimate mortality rates associated with the milder health states, but the use of the alternative data source significantly underestimated overall mortality. Further research that correlates different clinical indices and patient characteristics, such as age and smoking status, should improve the accuracy of mortality rates specific to COPD patients.

SGRQ results are the most disparate between the model and the study. Change in HR-QOL over time is not well documented in literature. Change in SGRQ is usually reported over short periods of time, which makes long-term extrapolation of change in SGRQ difficult. Studies utilizing registry data or similar observational studies over the long term may be able to significantly improve the accuracy of SGRQ outputs.

OLIN

OLIN (Obstructive Lung Disease in Northern Sweden) was a longitudinal study that followed COPD patients (64 % male) over 20 years [36]. The primary endpoint was mortality and the secondary endpoint was the annual decline in FEV1 among survivors [36].

The correlation matrix was not used to generate patients for this simulation in order to use the published FEV1% predicted of the patient population [36], which was significantly higher than the original indacaterol trials [22-24].

The modelled proportion of patients who have died appears to be overestimated when compared with the average mortality rate reported in the study (70 % [95 % CI 69.61–70.69] as a model result vs. 54 % as reported in the publication [36]). The OLIN study cohort has considerably less severe disease than those in the indacaterol studies [22-24], UPLIFT [18] or TORCH [17] and bears out the hypothesis that the current data source overestimates the mortality rates associated with the less severe disease states. This is especially likely for the mild disease state, which carries an excess mortality rate of 2.25 % annually but is often considered a pre-COPD state. The model was run over a time horizon of 10 years in order to compare with the published data for the annual rate of FEV1 decline. The simulated mean decline in FEV1 is found to be marginally higher (27 ml/year [95 % CI 26.1–28] vs. 24 ml/year), which could be due to the higher proportion of smokers generated in the co-variance matrix (64 %) compared with the OLIN study (46 %) [36].

The validation with the four studies found the model to have generally good predictive ability, although some inputs, such as mortality and SGRQ, could benefit from further research. However, the overall validation yielded a good degree of external validity. It also allowed for refinement of the model through the addition of a no treatment arm.

Discussion

To our knowledge, this is the first COPD model featuring micro-simulation methods that allow users to incorporate treatment switches into their analyses. We also aimed to circumvent some of the restrictions arising from the rigidity of set health states. These changes allow modelling that both describes COPD disease progression in a more accurate way and allows alignment with different treatment guidelines where treatment switches are recurrent. In the absence of a clear standard algorithm for determining treatment switches, the current model triggers are based on clinical practice as obtained from a UK physician.

Further accuracy was achieved through access to three COPD clinical trials [22-24] that allowed the pooling of patient-level data. This allowed the construction of a co-variance matrix to randomly generate patients with characteristics of actual COPD patients. While the inclusion/exclusion criteria of these trials impose certain restrictions on the generalizability of the results, it is a notable improvement from randomly generating patient characteristics independently. If the model should be used to study a particular type of COPD patient population, having patient-level data corresponding to that population would be an advantage, but the use of this default matrix ensures that the basic relationships between gender, age, height, smoking status and FEV1 levels are maintained.

The current model settings mean that the model is heavily reliant on lung function as the driver. This is a reasonable and historically accepted framework for COPD modelling, especially as spirometry is the standard clinical endpoint included in most clinical trials. However, when exploring the determinants of treatment initiation and switching for the UK practice, it was clear that lung function measurements are secondary considerations: breathlessness, cough, exacerbations and impact on a patient’s QOL are more likely to be triggers for treatment switch. For this reason, the model is able to accommodate data on HR-QOL, SGRQ, TDI and history of exacerbations to ensure a more accurate alignment with actual clinical practice. The constraint here is on the lack of longitudinal data around these outcomes measures. In some areas, such as history of exacerbation, research may be forthcoming to allow a more evidence-based approach to mapping the risk of a recurrent exacerbation.3 In other areas, real-world evidence needs to be collected to understand the long-term consequences of treating certain COPD symptoms.

The results of testing published data in the model highlighted some discrepancies across trial validation. Although the model produced mortality rates that were acceptable compared with the UPLIFT trial, they appeared higher for TORCH. The difference between these results may indicate that COPD-related mortality rates need to be further refined according to GOLD stages or independently of GOLD stages.

While this model has endeavoured to be comprehensive in the description of COPD, as a multi-factorial disease there are still areas that could be included in the model. Examples of these are co-morbidities such as myocardial infarction triggered by a severe exacerbation. The intention and scope of this study, however, was to focus on the disease progression of COPD and the impact of pharmaceutical interventions. Decline in TDI and SGRQ was also assumed to be independent of patient characteristics. To include such co-morbidities would first require that all such relationships be documented with risk probabilities and would widen the scope considerably to evaluate the disease on a holistic population level.

The model also has the potential to step further away from the GOLD-based disease severity classification, which has recently been acknowledged as inadequate to wholly capture the complexity of COPD. This requires more research to quantify the relationship between lung function decline, symptoms and HR-QOL as well as the impact of pharmaceutical interventions on them over a longer period of time. These research needs must be met by both changes in RCT design and comprehensive post-marketing observational studies.

In spite of these potential limitations, the simulation-based model has shown good predictive ability in the external validity testing we conducted. The model was able to replicate the results of published clinical trials and studies within reasonable ranges of acceptability. It has also highlighted areas of COPD clinical research that could benefit from more work to refine this model.

Conclusion

The micro-simulation model is a flexible approach for modelling COPD that allows consideration of complex COPD treatment pathways. The model was found to be generally robust in terms of predicting clinical outcomes of published studies when tested against other studies. It has significant potential as a tool for supporting future COPD treatment positioning decisions as well as to inform the development of policies, guidelines or cost-effectiveness analyses.