Real world effects of COPD medications: a cohort study with validation against results from randomised controlled trials.

Real-world data provide the potential for generating evidence on drug treatment effects in groups excluded from trials, but rigorous, validated methodology for doing so is lacking. We investigated whether non-interventional methods applied to real-world data could reproduce results from the landmark TORCH COPD trial.We performed a historical cohort study (2000-2017) of COPD drug treatment effects in the UK Clinical Practice Research Datalink (CPRD). Two control groups were selected from CPRD by applying TORCH inclusion/exclusion criteria and 1:1 matching to TORCH participants, as follows. Control group 1: people with COPD not prescribed fluticasone propionate (FP)-salmeterol (SAL); control group 2: people with COPD prescribed SAL only. FP-SAL exposed groups were then selected from CPRD by propensity score matching to each control group. Outcomes studied were COPD exacerbations, death from any cause and pneumonia.2652 FP-SAL exposed people were propensity score matched to 2652 FP-SAL unexposed people while 991 FP-SAL exposed people were propensity score matched to 991 SAL exposed people. Exacerbation rate ratio was comparable to TORCH for FP-SAL versus SAL (0.85, 95% CI 0.74-0.97 versus 0.88, 0.81-0.95) but not for FP-SAL versus no FP-SAL (1.30, 1.19-1.42 versus 0.75, 0.69-0.81). In addition, active comparator results were consistent with TORCH for mortality (hazard ratio 0.93, 0.65-1.32 versus 0.93, 0.77-1.13) and pneumonia (risk ratio 1.39, 1.04-1.87 versus 1.47, 1.25-1.73).We obtained very similar results to the TORCH trial for active comparator analyses, but were unable to reproduce placebo-controlled results. Application of these validated methods for active comparator analyses to groups excluded from randomised controlled trials provides a practical way for contributing to the evidence base and supporting COPD treatment decisions.

Introduction

The long-acting β2-agonist (LABA)–inhaled corticosteroid (ICS) combination product fluticasone propionate (FP) + salmeterol (SAL) is one of the most widely used COPD treatments. It was studied in large randomised trials (such as the landmark TORCH study [1]), but the effects of treatment in important patient groups who were either not included or underrepresented in trials are unknown. While real-world observational data has the potential to be used to carry out non-interventional studies of COPD drug treatment effects in groups excluded from trials, the use of these data for estimating treatment effectiveness is in its infancy. Demonstrating that non-interventional methods can account for the absence of treatment randomisation in real-world settings is a particular challenge. Rigorous, validated methodology is needed to translate these complex data into reliable evidence [2].

One approach is to analyse “randomised controlled trial (RCT)-analogous” cohorts from non-interventional data sources; if results are comparable to those generated by the reference RCT, this should increase confidence in the validity of the results, and in the non-interventional methods used to obtain them. The validated non-interventional methodology could then go on to be used for the analysis of treatment effects within people prescribed drugs in clinical practice who would have been excluded from (or were underrepresented in) RCTs [2].

In this study, we 1) applied trial inclusion and exclusion criteria to detect trial “eligible” participants from real-world data; 2) selected from these “eligible” participants to obtain a group who were as similar to the TORCH participants as possible by individual matching using TORCH data; and 3) applied standard observational methods to account for confounding [1].

We then assessed whether treatment effects in this real-world cohort were comparable to those measured by the TORCH trial in terms of the effect of the FP-SAL fixed combination product on 1) exacerbations; 2) mortality; and 3) pneumonia.

Materials and methods

Study design

A historical cohort study, with validation against RCT results.

Setting/data sources

This study uses individual trial data from the TORCH RCT (obtained via www.clinicalstudydatarequest.com), and non-interventional data from the UK Clinical Practice Research Datalink (CPRD) linked to Hospital Episodes Statistics (HES) data. We have described the use of CPRD in this study and the characteristics of TORCH in a previous publication (replicated in appendix 1) [2].

Diagnostic and therapeutic codelists

All diagnostic and therapeutic codelist files are available for download (https://datacompass.lshtm.ac.uk/1655/).

Selection of participants: FP-SAL exposed versus unexposed

Step 1: selection of all potentially eligible patients

An initial cohort was selected from all HES-linked patients actively registered in the CPRD between January 1, 2004 and January 1, 2017, who fulfilled the TORCH inclusion criteria (supplementary table A1-1) [1]. The date that an individual met all inclusion criteria with ≥12 months prior registration in the CPRD was the “eligible for TORCH inclusion” date.

Step 2: selection of pool of unexposed patients

Time periods during which patients were unexposed to FP-SAL that occurred on or after the eligible for TORCH inclusion date and which did not meet any of the TORCH drug exposure exclusion criteria (supplementary table A1-1) were selected (supplementary figure A1-1) [1]. The start of follow-up date (index date) for the unexposed time period was selected as a random date between the start and end of the unexposed period (supplementary figure A1-1). Individuals in CPRD were able to contribute more than one such unexposed time period to the total pool of unexposed time periods (supplementary figure A1-1). Unexposed time periods were then removed from the cohort if the patient met any of the remaining TORCH study exclusion criteria prior to the index date [1].

Step 3: selection of unexposed to FP-SAL people by 1:1 matching FP-SAL time periods to TORCH participants

Each individual TORCH participant from the TORCH RCT (obtained via www.clinicalstudydatarequest.com, as described earlier) was matched 1:1 with the closest available unexposed to FP-SAL time period on the following TORCH baseline characteristics: age, sex, body mass index (BMI), 1-year history of exacerbations requiring hospitalisation, history of cardiovascular disease and lung function (forced expiratory volume in 1 s (FEV1)) (appendix 1). An individual could only contribute one unexposed period to the final TORCH-matched unexposed cohort (supplementary figure A1-1); therefore, the output of this step was a cohort of unexposed to FP-SAL people.

Step 4: selection of exposed to FP-SAL time periods and application of TORCH exclusion criteria

All prescriptions for FP-SAL that started 1) on or after the initial eligible for TORCH inclusion date specified in Step 1 and 2) ≥4 weeks after the end of a prescription for any of the TORCH study drugs were identified. FP-SAL exposed time periods were created with the index date assigned as the start of an FP-SAL prescription. The same exclusion critiera applied to the unexposed FP-SAL time periods (Step 3) were applied. If an individual contributed time periods to both the unexposed (Step 2) and exposed (Step 4) cohorts, they were contributing different periods of their person-time to each cohort (pre-FP-SAL treatment for Step 2 versus post-FP-SAL treatment for Step 4) (supplementary figure A1-1).

Step 5: selection of comparable FP-SAL exposed participants by matching FP-SAL exposed time periods to FP-SAL unexposed people

Using the index date baseline characteristics, propensity scores for receiving FP-SAL were calculated for the (TORCH-matched) FP-SAL unexposed people selected in Step 3 and the FP-SAL exposed time periods selected in Step 4. Each FP-SAL unexposed (TORCH-matched) person selected in Step 3 was matched 1:1 with the FP-SAL exposed time period from Step 4 with the closest propensity score. An individual could only appear once as an exposed participant in the final propensity score matched cohort, meaning that this step selected FP-SAL exposed participants from the initial pool of FP-SAL exposed time periods. It was possible for the same person to be included in the FP-SAL unexposed and FP-SAL exposed cohorts, with different start of follow-up dates in each cohort.

Note that we did not apply matching to the TORCH trial in order to select our FP-SAL exposed group, because we wanted to develop propensity score methodology for obtaining balanced groups that could then be applied to the study of groups of patients who were not included in the trial at all.

Selection of participants: FP-SAL exposed versus SAL exposed

The participant selection approach was analogous to the FP-SAL exposed versus FP-SAL unexposed participant selection, except the comparator group was those exposed to SAL, and the study period was 2000–2017 (to obtain sufficient SAL-exposed numbers). The other differences in participant selection are detailed in appendix 1.

Exposures, outcomes and covariates

Exposures

Exposure status was determined using CPRD prescribing records. Being prescribed FP-SAL was the primary exposure of interest and the comparison exposure groups were 1) people not being prescribed FP-SAL and 2) people being prescribed SAL only. Refer to appendix 1 for further details.

Outcomes

Outcomes were COPD exacerbation, all-cause mortality, pneumonia and time to treatment discontinuation (refer to appendix 1: Outcomes – further details on outcome definitions).

Covariates

Covariates available for inclusion in the propensity score models have been detailed previously and are listed in appendix 1 [2].

Statistical analysis

Propensity score for addressing confounding

A pool of initial variables were selected based upon a priori knowledge/clinical expertise. Those variables not associated with outcome in crude analysis were then removed, before applying multivariable logistic regression (on drug exposure status) to generate propensity scores [2]. Variables were selected for inclusion in the final propensity score multivariable logistic regression model using likelihood ratio tests for goodness of fit. Starting from an initial fully adjusted model that included all initial variables found to be associated with outcome, goodness of fit was tested after removing variables sequentially from the logistic regression model (starting with the variable most weakly associated with exposure in the fully adjusted model). Variables with likelihood ratio test p-value >0.1 were removed from the model. Separate propensity scores were developed in this way for each outcome. Standardised differences were used to assess any residual imbalances after matching (with sd >0.1 indicating substantial/important imbalance) [3].

Methods of analysis

Comparisons were made for each outcome over 3 years between 1) people exposed to FP-SAL versus people unexposed to FP-SAL (matched on propensity score) and 2) people exposed to FP-SAL versus people exposed to SAL (matched on propensity score). All analyses were performed according to the intention-to-treat principle (appendix 1), estimating the same effect measures as TORCH. The number of exacerbations was modelled using a negative binomial model with the log of treated time as an offset variable. Time to mortality and treatment discontinuation were analysed using Cox proportional hazards regression, and pneumonia risk was analysed using Poisson regression.

Validation of results against TORCH

Detailed criteria for considering results to be comparable with TORCH were pre-specified and have been published previously (appendix 1) [2].

Missing data

Complete records analysis was applied given the low proportion of missing data (only socioeconomic status, alcohol or BMI had any missing data, and of these variables, none had >5% missing data).

Analysis of impact of TORCH matching and TORCH criteria (post hoc analysis)

A post hoc analysis was performed assessing the impact of 1) omitting TORCH matching and 2) omitting both TORCH matching and application of TORCH trial inclusion/exclusion criteria (refer to appendix 1 for details).

Ethics

Scientific approval was provided by the London School of Hygiene and Tropical Medicine research ethics commitee (ref 11997) and the independent scientific advisory committee of the Medicines and Healthcare Products Regulatory Agency (protocol no. 17_114R). CPRD data are already approved via a national research ethics committee for purely non-interventional research of this type. Approval for use of the TORCH trial data was obtained from the Wellcome Trust, the relevant sponsor (GSK) and an independent review panel.

Results

Participants

FP-SAL exposed versus FP-SAL unexposed

Between January 1, 2004 and January 1, 2017 there were 125 671 people in CPRD with a diagnosis of COPD, 73 889 (59%) of whom were from HES-linked CPRD practices (figure 1). Application of TORCH inclusion criteria reduced this to 18 715 people, contributing 35 746 unexposed to FP-SAL time periods and 26 390 exposed to FP-SAL time periods. After applying TORCH exclusion criteria, dropping records with missing covariate data and matching the unexposed patients to TORCH participants, there were 4196 unexposed patients available for propensity score matching to 10 463 FP-SAL exposed time periods. The final propensity score matched cohorts included 2652 patients in each exposure group for the exacerbations analysis, 2708 for mortality and 2779 for pneumonia.

FIGURE 1

Flow of number of individuals included in the exposed to fluticasone propionate (FP)-salmeterol (SAL) versus unexposed to FP-SAL cohort analysis. Current/previous use of COPD drugs relates to any of the drugs studied in TORCH, long-acting bronchodilators and oral corticosteroids; refer to supplementary table A1-1 for specific details. CPRD: Clinical Practice Research Datalink; HES: Hospital Episodes Statistics; BMI: body mass index; SES: socioeconomic status. #: see supplementary table A1-1 for detailed list of inclusion criteria; ¶: see supplementary table A1-1 for detailed list of exclusion criteria.

FP-SAL exposed versus SAL exposed

For the FP-SAL versus SAL analysis, there were 154 785 people with a diagnosis of COPD in CPRD between January 1, 2000 and January 1, 2017, 91 733 (59%) of whom were from HES-linked CPRD practices (figure 2). 1146 SAL exposed patients were available for propensity score matching to 11 235 FP-SAL-exposed periods. The final propensity score matched cohorts included 991 (exacerbations), 432 (mortality), 935 (pneumonia) and 996 (treatment discontinuation) patients per exposure group.

FIGURE 2

Flow of number of individuals included in the exposed to fluticasone propionate (FP)-salmeterol (SAL) versus exposed to SAL cohort analysis. Current/previous use of COPD drugs relates to any of the drugs studied in TORCH, long-acting bronchodilators and oral corticosteroids (OCS); refer to supplementary table A1-1 for specific details. CPRD: Clinical Practice Research Datalink; HES: Hospital Episodes Statistics; BMI: body mass index; SES: socioeconomic status. #: see supplementary table A1-1 for detailed list of inclusion criteria; ¶: see supplementary table A1-1 for detailed list of exclusion criteria.

Application of TORCH inclusion/exclusion criteria and matching to TORCH

Applying the TORCH inclusion/exclusion criteria and matching to TORCH resulted in cohorts that were much more similar to those recruited to the TORCH trial (e.g. FEV1 for the FP-SAL versus unexposed to FP-SAL analysis was 66.3% predicted in CPRD before applying any criteria or matching, compared to 47.2% pred after these steps, compared to a TORCH placebo group value of 44.2% pred; supplementary table A2-1). The largest residual difference to the TORCH placebo group was for prior cardiovascular disease for both comparisons (supplementary tables A2-1 and A2-2).

Propensity score matching of CPRD cohorts

Details of the variables included in the final propensity score models are provided in supplementary table A2-3.

Prior to propensity score matching, differences by exposure status were noted for sex, FEV1, BMI, prior exacerbations, coronary heart disease, peripheral vascular disease, cerebrovascular disease, prescriptions for aspirin, COPD medications, number of general practice (GP) consultations and number of distinct medications (table 1). After propensity score matching, only the difference with respect to coronary heart disease, peripheral vascular disease, LABA and consultations persisted (table 1). Plots of propensity score distributions indicated close propensity score matching for exacerbations and all other outcomes under study (supplementary figure A2-1).

TABLE 1

Characteristics of the fluticasone propionate (FP)-salmeterol (SAL) versus unexposed to FP-SAL cohort before and after propensity score matching for the exacerbations analysis

	CPRD non-interventional population
	Before propensity score matching			After propensity score# matching
	Unexposed to FP-SAL¶	Exposed to FP-SAL+	Standardised difference	Unexposed to FP-SAL	Exposed to FP-SAL	Standardised difference
Subjects	4196	10 463 exposed time periods from 4259 people		2652	2652
Age# years	67 (61–73)	68 (62–74)	0.103	68 (61–73)	68 (62–74)	0.083
Male#	3175 (76)	6515 (62)	0.293	1868 (70)	1850 (70)	0.015
Lung function#,§
FEV1 % pred	47 (38–56)	50 (40–60)	0.297	49 (39–57)	48 (38–56)	0.024
FEV1/FVC %	53 (44–61)	53 (44–63)	0.073	53 (44–62)	52 (43–61)	0.045
BMI#,§	26 (22–29)	26 (23–31)	0.191	26 (23–30)	26 (22–30)	0.024
Prior exacerbationsƒ	0.51±0.92	0.66±1.13	0.148	0.56±0.96	0.62±1.04	0.060
Cardiovascular disease##
Coronary heart disease	1114 (27)	1783 (17)	0.232	720 (27)	441 (17)	0.257
Peripheral vascular disease	390 (9)	648 (6)	0.116	253 (10)	166 (6)	0.122
Cerebrovascular disease#	434 (10)	714 (7)	0.126	212 (8)	222 (8)	0.014
Other atherosclerosis	11 (0)	20 (0)	0.015	7 (0)	7 (0)	0.008
Statin prescription#,¶¶	2066 (49)	4614 (44)	0.103	1227 (46)	1238 (47)	0.008
Aspirin prescription¶¶	1563 (37)	3129 (30)	0.156	954 (36)	828 (31)	0.101
Other COPD medication prescriptions¶¶
LABA++	295 (7)	333 (3)	0.175	197 (7)	106 (4)	0.148
ICS#,++	530 (13)	842 (8)	0.151	280 (11)	333 (13)	0.063
LAMA#,++	1450 (35)	6284 (60)	0.528	1166 (44)	1177 (44)	0.008
ICS plus LABA#,§§	526 (13)	488 (5)	0.284	196 (7)	258 (10)	0.084
Type 2 diabetes##	543 (13)	1496 (14)	0.040	373 (14)	337 (13)	0.04
History of cancer##	696 (17)	2105 (20)	0.091	486 (18)	451 (17)	0.035
Chronic kidney disease##	540 (13)	1477 (14)	0.037	389 (15)	333 (13)	0.062
Healthcare utilisationƒ
GP consultations# n	21 (15–29)	16 (10–26)	0.409	18 (14–29)	16 (10–26)	0.143
Distinct medications# n	4 (2–7)	5 (3–8)	0.180	4 (2–7)	5 (3–8)	0.073
Hospitalisations# n	0 (0–1)	0 (0–1)	0.008	0 (0–1)	0 (0–1)	0.007
Hospital procedures# n	0 (0–0)	0 (0–1)	0.022	0 (0–0)	0 (0–0)	0.011

Data are presented as n, median (interquartile range), n (%) or mean±sd, unless otherwise stated. CPRD: Clinical Practice Research Datalink; FEV1: forced expiratory volume in 1 s; FVC: forced vital capacity; BMI: body mass index; LABA: long-acting β-agonist; ICS: inhaled corticosteroid; LAMA: long-acting muscarinic antagonist; GP: general practitioner. #: variables in this table that were included in the propensity score (see supplementary table A2-3 for list of variables included in final exacerbations propensity score model); ¶: TORCH inclusion/exclusion criteria applied and matched to TORCH individual patient data; +: TORCH inclusion/exclusion criteria applied; §: closest record prior to index date; ƒ: all counted within the year prior to index, includes exacerbations recorded in primary or secondary care; ##: any diagnosis for condition prior to index date; ¶¶: number who had at least one prescription within the previous year; ++: single product only; §§: combination product.

For the FP-SAL versus SAL exacerbations analysis, after propensity score matching there were notable imbalances in prior prescriptions for a LABA or an ICS, with smaller imbalances for lung function, BMI, coronary heart disease, statin prescription, aspirin prescription, LAMA and ICS plus LABA (table 2). Plots of propensity score distribution indicated that overall groups were well matched on propensity score for each outcome (supplementary figure A2-1).

TABLE 2

Characteristics of the fluticasone propionate (FP)-salmeterol (SAL) versus SAL cohort before and after propensity score matching for the exacerbations analysis

	CPRD non-interventional population
	Before propensity score matching			After propensity score# matching
	SAL¶	FP-SAL+	Standardised difference	SAL	FP-SAL	Standardised difference
Subjects	1146	11 235 exposed time periods from 4523 people		991	991
Age years	68 (62–73)	68 (62–74)	0.051	68 (62–73)	67 (61–73)	0.038
Male#	728 (64)	6960 (62)	0.033	628 (63)	637 (64)	0.019
Lung function§
FEV1# % pred	49 (41–57)	50 (40–60)	0.272	50 (41–57)	49 (40–57)	0.107
FEV1/FVC %	53 (44–61)	53 (44–62)	0.022	53 (45–62)	51 (42–60)	0.122
BMI§	26 (23–30)	26 (22–30)	0.057	26 (23–30)	26 (22–29)	0.123
Prior exacerbationsƒ	0.63±1.02	0.61±1.07	0.017	0.62±1.01	0.61±1.03	0.010
Cardiovascular disease##
Coronary heart disease	207 (18)	1958 (17)	0.017	175 (18)	129 (13)	0.129
Peripheral vascular disease	71 (6)	749 (7)	0.019	62 (6)	62 (6)	0.000
Cerebrovascular disease	87 (8)	792 (7)	0.021	81 (8)	64 (6)	0.066
Other atherosclerosis	1 (0)	21 (0)	0.027	1 (0)	1 (0)	0.026
Statin prescription¶¶	462 (40)	4906 (44)	0.068	411 (41)	344 (35)	0.140
Aspirin prescription¶¶	333 (29)	3376 (30)	0.022	297 (30)	246 (25)	0.116
Other COPD medication prescriptions¶¶
LABA++	793 (69)	98 (1)	2.052	648 (65)	15 (2)	1.839
ICS#,++	419 (37)	862 (8)	0.742	275 (28)	387 (39)	0.241
LAMA++	477 (42)	6598 (59)	0.347	432 (44)	487 (49)	0.111
ICS plus LABA§§	28 (2)	537 (5)	0.125	24 (2)	50 (5)	0.139
Type 2 diabetes#,##	116 (10)	1549 (14)	0.113	101 (10)	100 (10)	0.003
History of cancer##	200 (17)	2252 (20)	0.066	178 (18)	163 (16)	0.040
Chronic kidney disease#,##	104 (9)	1535 (14)	0.145	89 (9)	85 (9)	0.014
Healthcare utilisationƒ
GP consultations# n	15 (9–23)	16 (9–26)	0.765	15 (9–23)	15 (9–25)	0.021
Distinct medications n	5 (3–8)	5 (3–8)	0.039	5 (3–8)	5 (3–8)	0.019
Hospitalisations# n	0 (0–1)	0 (0–1)	0.063	0 (0–1)	0 (0–1)	0.005
Hospital procedures# n	0 (0–0)	0 (0–1)	0.065	0 (0–0)	0 (0–0)	0.035

Data are presented as n, median (interquartile range), n (%) or mean±sd, unless otherwise stated. CPRD: Clinical Practice Research Datalink; FEV1: forced expiratory volume in 1 s; FVC: forced vital capacity; BMI: body mass index; LABA: long-acting β-agonist; ICS: inhaled corticosteroid; LAMA: long-acting muscarinic antagonist; GP: general practitioner. #: variables in this table that were included in the propensity score (see supplementary table A2-3 for list of variables included in final exacerbations propensity score model); ¶: TORCH inclusion/exclusion criteria applied and matched to individual TORCH patients; +: TORCH inclusion/exclusion criteria applied; §: closest record prior to index date; ƒ: all counted within the year prior to index; ##: any diagnosis for condition prior to index date; ¶¶: number who had at least one prescription within the previous year; ++: single product only; §§: combination product.

Main results

For the exacerbations analysis, the rate ratio in the propensity score matched groups was 1.30 (95% CI 1.19–1.42) (table 3). According to our pre-specified protocol this (harmful) association was not considered to be consistent with the (protective) TORCH placebo-controlled result for the same outcome (0.75, 0.69–0.81) [2]. Similarly, our result for the mortality outcome (hazard ratio (HR) 1.11, 95% CI 0.95–1.26) was in the opposite direction to the TORCH placebo-controlled result (0.83, 0.68–1.00). For the pneumonia analysis, we found weak evidence for a 14% increased risk associated with FP-SAL (risk ratio 1.14, 0.96–1.34) which was not consistent with the stronger harmful association found by the TORCH placebo-controlled analysis (1.59, 1.35–1.88).

TABLE 3

Results for the analysis of exacerbations, mortality, pneumonia and time to treatment discontinuation for fluticasone propionate (FP)-salmeterol (SAL) versus no FP-SAL (compared to TORCH results)

	CPRD non-interventional population		TORCH trial population#
	Unexposed to FP-SAL	Exposed to FP-SAL	Placebo	FP-SAL
Subjects	4196	10 463	1524	1533
Exacerbations
Person-years at risk	9330	22 054
Events	4994	15 944
Rate per person per year	0.53	0.72	1.13	0.85
Crude rate ratio	1	1.35 (1.28–1.43)
Propensity matched rate ratio	1	1.30 (1.19–1.42)+	1	0.75 (0.69–0.81)
Mortality
Person-years at risk	9330	22 054
Events	543	1245
Probability at 3 years¶ %	16.13	16.04	15.16	12.59
Crude hazard ratio	1	0.98 (0.88–1.08)
Propensity matched hazard ratio	1	1.11 (0.95–1.26)§	1	0.83 (0.68–1.00)
Pneumonia
Events	350	998
Percentage of total patients	8.34	9.54	12.31	19.60
Crude risk ratio	1	1.14 (1.01–1.28)
Propensity matched risk ratio	1	1.14 (0.96–1.34)ƒ	1	1.59 (1.35–1.88)
Time to treatment discontinuation
Person-years at risk	##	20 402
Events		2255
Probability at 3 years¶ %		28.20	43.50	33.70
Crude hazard ratio
Propensity matched hazard ratio			1	0.69 (0.62–0.78)

Data are presented as n, unless otherwise stated. CPRD: Clinical Practice Research Datalink. #: only results reported in the TORCH trial publication are shown; ¶: calculated using a Cox proportional-hazards model; +: n=2652 in each exposure group after propensity score matching (supplementary table A2-3 for list of variables contributing to propensity score for exacerbations analysis); §: n=2708 in each exposure group after propensity score matching (see supplementary table A2-3 for list of variables contributing to propensity score for mortality analysis); ƒ: n=2779 in each exposure group after propensity score matching (see supplementary table A2-3 for list of variables contributing to propensity score for pneumonia analysis); ##: time to treatment discontinuation analysis not applicable for unexposed to FP-SAL group.

For the exacerbations analysis, we obtained a propensity score matched rate ratio of 0.85 (95% CI 0.74–0.97). According to our pre-specified protocol this (protective) effect was considered to be consistent with the TORCH FP-SAL versus SAL result for the same outcome (0.88, 0.81–0.95) (table 4) [2]. Similarly, our result for the mortality outcome (HR 0.93, 95% CI 0.65–1.32) was consistent with the TORCH FP-SAL versus SAL result (0.93, 0.77–1.13). For the pneumonia analysis, we found evidence for a 39% increased risk associated with FP-SAL (risk ratio 1.39, 95% CI 1.04–1.87), which was also consistent with the harmful association found by the TORCH FP-SAL versus SAL analysis (1.47, 1.25–1.73). For the time to treatment discontinuation analysis, the effect was apparently much stronger outside of the trial setting (non-interventional HR 0.23, 95% CI 0.20–0.27 versus TORCH 0.89, 0.79–0.99).

TABLE 4

Results for the analysis of exacerbations, mortality, pneumonia and time to treatment discontinuation for fluticasone propionate (FP)-salmeterol (SAL) versus SAL (compared to TORCH results)

	CPRD non-interventional population		TORCH trial population#
	SAL	FP-SAL	SAL	FP-SAL
Subjects	1146	11 235	1521	1533
Exacerbations
Person-years at risk	2566	24 062
Events	1515	14 034
Rate per person per year	0.73	0.59	0.97	0.85
Crude rate ratio	1	0.80 (0.72–0.88)
Propensity matched rate ratio	1	0.85 (0.74–0.97)+	1	0.88 (0.81–0.95)
Mortality
Person-years at risk	2566	24 062
Events	138	1445
Probability at 3 years¶ %	15.09	16.84	13.48	12.59
Crude hazard ratio	1	1.12 (0.94–1.34)
Propensity matched hazard ratio	1	0.93 (0.65–1.32)§	1	0.93 (0.77–1.13)
Pneumonia
Events	86	1137
% of total patients	7.50	10.12	13.29	19.60
Crude risk ratio	1	1.35 (1.09–1.66)
Propensity matched risk ratio	1	1.39 (1.04–1.87)ƒ	1	1.47 (1.25–1.73)
Time to treatment discontinuation
Person-years at risk	1251	21 587
Events	740	2449
Probability at 3 years¶ %	77.02	28.04	36.40	33.70
Crude hazard ratio	1	0.22 (0.20–0.23)
Propensity matched hazard ratio	1	0.23 (0.20–0.27)##	1	0.89 (0.79–0.99)

Data are presented as n, unless otherwise stated. CPRD: Clinical Practice Research Datalink. #: only results reported in the TORCH trial publication are shown; ¶: calculated using a Cox proportional-hazards model; +: n=991 in each exposure group after propensity score matching (see supplementary table A2-3 for list of variables contributing to propensity score for exacerbations analysis); §: n=443 in each exposure group after propensity score matching (see supplementary table A2-3 for list of variables contributing to propensity score for mortality analysis); ƒ: n=996 in each exposure group after propensity score matching. (see supplementary table A2-3 for list of variables contributing to propensity score for pneumonia analysis); ##: n=935 in each exposure group after propensity score matching (see supplementary table A2-3 for list of variables contributing to propensity score for time to treatment discontinuation analysis).

Repeating the FP-SAL versus SAL analysis omitting the TORCH matching step led to an exacerbations rate ratio of 0.87 (95% CI 0.81–0.94) (table 5), very similar to both the main analysis and TORCH result. In contrast, applying neither the TORCH criteria nor matching led to a completely different effect estimate (1.64, 1.52–1.77).

TABLE 5

Impact of choice of selection methods on ability to replicate trial results for the analysis of exacerbations, in people exposed to fluticasone propionate (FP)-salmeterol (SAL) versus people exposed to SAL

	Rate ratio		Subjects per exposure group n
	SAL	FP-SAL
TORCH trial	1	0.88 (0.81–0.95)	1524
CPRD non-interventional selection method#
TORCH inclusion and exclusion criteria and matched to TORCH¶	1	0.85 (0.74–0.97)	991
TORCH inclusion and exclusion criteria only	1	0.87 (0.81–0.94)	3225
No TORCH criteria or matching	1	1.64 (1.52–1.77)	5951

CPRD: Clinical Practice Research Datalink. #: SAL and FP-SAL groups were propensity score matched for all selection methods; ¶: as per the main analysis and presented in table 4.

Discussion

We have demonstrated that methods applied to non-interventional data can generate results comparable to active comparator trials for COPD treatment effects. In contrast, we found that the same methods were unable to replicate placebo-controlled trial results.

Comparison with previous studies

Previous studies applying similar “trial-replication” approaches

Although a number of papers have compared the designs of observational studies with RCTs [4-9], and some studies have generated results similar to an earlier or subsequent trial [10-12], there are very few non-interventional studies that have set out to explicitly replicate a specific trial cohort and its results.

Hernán et al. [13] replicated the design and result of the Women's Health Initiative randomised trial on the effect of oestrogen/progestin therapy on coronary heart disease risk. Smeeth et al. [14] analysed the effect of statins on a range of health outcomes and replicated the Heart Protection Study randomised trial. Fralick et al. [15] applied trial criteria and utilised propensity score matching to replicate cardiovascular results from the ONTARGET trial (comparing telmisartan to ramipril).

Previous studies of COPD drug treatment effects

Results of five (LABA/ICS versus LABA) interventional studies (including TORCH) were summarised in a Cochrane review (rate ratio 0.76, 95% CI 0.68–0.84) [16]. Three out of these five studies estimated effect sizes considerably greater than TORCH; as we mirrored TORCH, our results aligned most closely to TORCH.

A number of studies have found strong survival benefits of ICS therapy after hospital discharge [17, 18]. After accounting for likely time-related biases impacting these studies, a null effect was obtained (rate ratio 0.94, 95% CI 0.81–1.09) [19]. The methodology we applied obtained a mortality effect estimate comparable to the analysis designed to account for time-related biases (0.93, 95% CI 0.65–1.32).

In line with TORCH, previous studies have found an increased risk of pneumonia associated with ICS-containing treatments for COPD [20, 21, 16]. Our result (risk ratio 1.39, 95% CI 1.04–1.87) was consistent with results of a meta-analysis of trials comparing LABA/ICS to LABA formulations (1.55, 1.20–2.01) [16] and very similar to a recent non-interventional study comparing LABA/ICS to LAMA formulations (HR 1.37, 95% CI 1.17–1.60) [22].

Our 3-year probability of treatment discontinuation for FP-SAL (28%) is comparable to non-adherence figures from previous non-interventional real-world data studies (49% [23] and 43% [24]). The probability of discontinuation of salmeterol that we observed (77%) was higher than these two previous non-interventional studies, leading to the discrepancy with TORCH. We hypothesised that during our study period a large proportion of the patients who would have been initially prescribed salmeterol would have been likely to switch to FP-SAL due to prescribing decisions in primary care; a post hoc analysis found that 43% of people prescribed salmeterol switched to FP-SAL during follow-up (compared to only 2% switching from FP-SAL to salmeterol).

Implications and further work

When studying COPD treatment effects, if 1) the analysis is of active comparators; 2) trial exclusion and inclusion criteria are applied; and 3) the propensity score models that we developed for each outcome are applied to balance exposure groups, then the results of studies carried out in routinely collected non-interventional data can be considered robust in the sense that they will be highly comparable to trial results. This now provides a methodological framework for being able to analyse COPD drug treatment effects in real-world data, focusing on groups that were either not included or underrepresented in trials [2].

Our inability to replicate placebo-controlled analyses suggests uncontrolled confounding by indication [25]. One possibility for how this confounding by indication may be manifesting relates to the aspect of our study design that allowed people to be included in both the exposed and unexposed cohorts; the result we obtained could be strongly influenced by people initially in the unexposed group who are relatively healthy, but then get sicker over time and require FP-SAL treatment and end up in the exposed group. However, in a post hoc analysis where we dropped the 730 (out of a total of 2652 per group) people who appeared in both cohorts, our effect estimate was nearly identical (rate ratio 1.33, 95% CI 1.20–1.47). However, we do consider that because COPD treatment is based on a step-up approach, it is highly likely that patients not exposed to FP-SAL in routine primary care are generally likely to be those with milder COPD.

One point of note relates to the large difference in incidence rate between the TORCH placebo group (1.13 exacerbations per person per year) and our FP-SAL unexposed group (0.53 exacerbations per person per year). In order to investigate underlying reasons for this discrepancy, we performed a post hoc analysis where we compared the characteristics of the 1753 people from TORCH who were not able to be matched to our unexposed to FP-SAL population in Step 3 with those who were successfully matched. We found that those not matched were younger (mean age 60.7 years versus 65.8 years), sicker (e.g. history of cardiovascular disease 93% versus 46%), had worse lung function (e.g. FEV1 34.9% pred versus 45.9% pred), and included a higher proportion recruited from Eastern European trial sites (27% versus 17%). People with these characteristics may have been highly suitable for recruitment to clinical trials, but are very difficult to find in UK primary care, and illustrate why it is likely to be challenging to obtain comparable absolute rates in emulated cohorts within a single country based on historical international trials.

Previous authors have recommended that when trying to emulate trial results, it is important to choose an active comparator trial [15]. However, there are examples where placebo-controlled analyses have been successfully replicated [13, 14]. One possibility is that replication of placebo-controlled results works better when the drug studied is 1) preventative and 2) used in a generally healthy cohort (for example, the cited studies were of statins and of post-menopausal hormone therapy both prescribed in some instances to people without a specific underlying chronic disease, in contrast to the patients with COPD who received therapy in our study). We consider that further avenues of research could be followed to understand if there remains a possibility of replicating placebo-controlled studies within a non-interventional setting for COPD therapies. These could include application of high-dimensional propensity scores or the use of instrumental variables. In addition, our work suggests that treatment discontinuation in the setting of non-interventional data may be driven by very different factors to those seen in trials and, at least in the setting of COPD, may not be a useful outcome to study. For example, it is difficult to establish from routinely collected data whether a patient has truly stopped taking their medicine, or is just taking the medicine differently than prescribed (e.g. is taking less than has been prescribed over a longer period).

Finally, in our post hoc analysis we found that the application of the trial-matching step did not confer any advantage over application of trial criteria alone in this setting. This suggests that treatment-covariate interactions are not as critical as we initially thought in this therapeutic area.

Limitations

Some of the TORCH inclusion criteria were not fully assessable using CPRD data, meaning that the inclusion/exclusion criteria are analogous with TORCH criteria, but we acknowledge they are not identical. We originally planned to apply frequency of COPD therapy prescriptions in the previous year as a matching character/criteria. In practice, this was not feasible. However, it appears that matching at this level of detail was not required to be able to replicate trial results for active comparator analysis. Finally, within TORCH, the dose of the fixed combination product FP-SAL was specified as 500 µg of FP and 50 µg of SAL (500/50), and the dose of SAL alone as 50 µg, whereas in our study we did not limit to a specific dose. The reason for this is that dosage information is incompletely captured in CPRD, but as these are the only approved doses of FP-SAL and of SAL for COPD in the UK, we consider the doses that people were prescribed in our study would have been generally similar to that administered in the TORCH trial.

Conclusions

By replicating the COPD TORCH trial selection procedures and inclusion/exclusion criteria in real-world data and developing propensity score models to account for any remaining differences between groups, we were able to obtain highly comparable relative effect estimates to the TORCH RCT active comparator analysis for exacerbations, mortality and pneumonia. Replication of placebo-controlled analyses was not possible, and further work to investigate whether likely residual confounding by indication can ever be accounted for in this therapeutic area is warranted. Application of the same selection procedures and propensity score models developed here to active comparator analyses of COPD drug treatment effects in groups underrepresented or excluded from trials provides a practical way for key evidence gaps to be filled.

Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.

Supplementary material: appendix 1 and 2 ERJ-01586-2020.SUPPLEMENT

This one-page PDF can be shared freely online.

Shareable PDF ERJ-01586-2020.Shareable