Severe COPD Exacerbation Risk and Long-Acting Bronchodilator Treatments: Comparison of Three Observational Data Analysis Methods.

OBJECTIVE: Results from three observational methods for assessing effectiveness of long-acting bronchodilator therapies for reducing severe exacerbations of chronic obstructive pulmonary disease (COPD) were compared: intent-to-treat (ITT), as protocol (AP), and an as-treated analysis that utilized a marginal structural model (MSM) incorporating time-varying covariates related to treatment adherence and moderate exacerbations. STUDY DESIGN AND
SETTING: Severe exacerbation risk was assessed over a 2-year period using claims data for patients aged ≥40 years who initiated long-acting muscarinic antagonist (LAMA), inhaled corticosteroid/long-acting beta-agonist (ICS/LABA), or triple therapy (LAMA + ICS/LABA).
RESULTS: A total of 5475 COPD patients met inclusion criteria. Six months post-initiation, 53.5 % of patients discontinued using any therapy. The ITT analysis found an increased severe exacerbation risk for triple therapy treatment (hazard ratio [HR] 1.24; 95 % confidence interval [CI] 1.00-1.53). No increased risk was found in the AP (HR 1.00; 95 % CI 0.73-1.36), or MSM analyses (HR 1.11; 95 % CI 0.68-1.81). The MSM highlighted important associations among post-index events.
CONCLUSION: Neglecting to adjust for treatment discontinuation may produce biased risk estimates. The MSM approach is a promising tool to compare chronic disease management by illuminating relationships between treatment decisions, adherence, patient choices, and outcomes.

Key Points

Introduction

In observational comparative effectiveness studies of chronic disease treatments, treatment effectiveness is assessed over a specified time period that starts with an ‘index date’ when treatment is initiated. Regression analyses are used to assess differences in outcomes between treatments, and traditionally the covariates considered in the regression analyses are baseline factors—sociodemographic characteristics and clinical factors assessed at the index date and a period prior to the index date. Many studies use an intent-to-treat (ITT) approach in which all outcomes post-index are attributed to the index treatment in an attempt to mimic randomized clinical trials (RCTs) [1-4]. In an ITT analysis, the focus is on effectiveness of that initial treatment decision; irrespective of whether subjects persist with treatment post-index. For the most part, persistence to treatment is not problematic in RCT analyses. If it is, an ‘as protocol’ (AP) sensitivity analysis, in which subjects who discontinued their assigned treatment are censored, is normally conducted. These traditional analysis methods may also use propensity matching to select comparison groups similar in measured baseline factors, although propensity matching may result in a decreased sample size. Additionally, propensity analyses often provide similar results to traditional regression analyses when using the same measured covariates [5, 6]. When treatment switching is prevalent, differences in treatment effectiveness in ITT analyses will tend toward the null; poor outcomes may result from poor persistence to therapy treatment, and effectiveness will be attenuated. And when time-varying confounding is present, traditional methods of estimating effectiveness may not adequately control for bias.

Time-varying confounders are factors that relate not only to past, current, and future treatment choices, but also to outcomes of interest [7]. A well known example is the time-varying factor CD4 lymphocyte count for zidovudine (AZT) treatment for individuals with human immunodeficiency virus (HIV). AZT has an impact on the CD4 lymphocyte count, but the CD4 lymphocyte count is also a factor in the decision to initiate AZT treatment as well as being significantly associated with mortality [7]. Traditional methods of analysis which consider only baseline characteristics (at the index date or from a prior period) have been found to not provide an adequate assessment of treatment effectiveness in the presence of time-varying confounders [7].

A marginal structural model (MSM) is a method of handling time-varying confounding [7-11]. MSMs describe causal effects (structural models) and produce population-average effect estimates (marginal estimates). MSMs are weighted, repeated measures analyses in which treatment is modeled as a time-varying covariate post-index [12]. By accommodating time-varying treatment choices and events, MSMs may provide better assessments of effectiveness where substantial switching or discontinuation of treatments occurs [13]. Weights balance confounding characteristics across treatment groups and incorporate informative censoring (e.g., loss to follow-up, treatment discontinuation), creating a balanced ‘pseudo-population’ similar to that achieved through an RCT randomization process. Using inverse-probability treatment weights, similar to propensity score weights, is common for observational MSM studies [5, 14–17].

Poor persistence to therapy is often a problem among patients with chronic disease [18-21]. Switching or discontinuation of treatment can be widespread, creating challenges in determining unbiased estimates of treatment effectiveness in retrospective studies, particularly when associated with time-varying confounders. Treatment for chronic obstructive pulmonary disease (COPD) is a case in point. An objective of COPD treatment is to reduce the occurrence and severity of exacerbations, periods of acute worsening of chronic respiratory symptoms (i.e., shortness of breath, wheezing, or cough) that can be life threatening and result in permanent loss of lung function [22]. However, one indication for long-acting bronchodilator (LABD) and inhaled corticosteroid (ICS) treatments is a history of exacerbations. Exacerbations tend to become more frequent and more severe as COPD progresses [23], and, as in other chronic conditions, when patients are not experiencing symptoms and adverse events, adherence often becomes poor [18, 19, 24, 25]. For patients with COPD, exacerbation experience and prior treatment are confounding factors for treatment and outcome events.

MSMs could aid in addressing challenges inherent in observational chronic disease studies, and more specifically for COPD, in comparing long-acting treatments. Of late, there has been interest in COPD in comparisons between an ICS/long-acting beta-agonist (ICS/LABA) combination therapy or long-acting muscarinic antagonist (LAMA) monotherapy, and triple therapy (use of both concurrently). There has been one RCT [26] and two observational studies [3, 4] that have compared the effectiveness of triple therapy with use of either therapy alone in reducing exacerbations over at least a 1-year period. The RCT reviewed outcomes up to 1 year post-index [26]. Findings suggested fewer exacerbations occurred among the triple therapy group, but sample sizes were modest (approximately 150 in each group) and substantial numbers discontinued therapy [26]. The two observational studies found triple therapy use beneficial. One compared triple therapy with LAMA [3], the other with ICS/LABA [4]. For the LAMA comparison, analyses were ITT, and therapy discontinuation for treatment groups was not reported. In the ICS/LABA comparison, mean follow-up time was 4.65 years, but no information was provided on the degree to which patients discontinued or switched medications post-index [4]. Sensitivity analyses of a propensity-matched analysis and an analysis that considered the triple therapy component tiotropium as a time-dependent covariate within a Cox-regression analysis were conducted, both of which affirmed a reduced risk for triple therapy [4]. Only cardiovascular disease and diabetes mellitus were included as baseline factors and details were not provided on the time-dependent analysis [4]. Moderate exacerbations and symptoms of unstable disease, such as use of relief medication, as time-varying confounders for treatment and experience of severe exacerbation were not considered in either study.

The purpose of this study was to examine the benefits of using the MSM approach to conduct an as-treated analysis that adjusted for time-varying confounding as compared with the commonly used ITT or AP methods in a retrospective observational study of COPD treatments. Specifically, we used claims data from two large Southwestern United States health systems to examine the effectiveness of COPD treatments on severe exacerbations among patients using either ICS/LABA or LAMA alone in comparison with patients using triple therapy, and compared the results of the MSM approach with those from ITT and AP study analyses.

Methods

This retrospective observational study compared patients receiving triple therapy with those receiving either ICS/LABA or LAMA therapy. Data consisted of administrative claims from two managed care plans for July 1, 2004 through September 30, 2012. Subjects were followed up to 24 months after treatment initiation (index date). Institutional Review Board approval came from The University of New Mexico Health Science Center, Human Research Protections Office.

Subjects were age ≥40 years, initiating ICS/LABA and/or LAMA therapies, with at least one COPD hospitalization or two COPD outpatient encounters (emergency department or clinic visit) pre-index (see electronic supplementary material [ESM]).

In tracking ICS/LABA and LAMA medication use post-index, an estimated days’ supply was calculated that allowed for 50 % of the optimal use (e.g., a 30-day supply could exist for up to 60 days).

Severe exacerbations were defined as hospitalizations due to COPD or a respiratory-related diagnosis. Moderate exacerbations were defined as a need for systemic corticosteroids and/or antibiotics, and lasted up to 10 days (see ESM). In addition to baseline exacerbations, post-index moderate exacerbations were identified that occurred prior to a severe exacerbation or, in the case where no severe exacerbation occurred, occurred during the post-index period. Hospitalizations during the post-index period for which COPD was recorded as a secondary diagnosis were also captured.

Use of other COPD treatments in the baseline period was summarized. Post-index use of short-acting beta-agonist (SABA) bronchodilators as a potential time-varying confounder was included. SABAs are used to relieve acute symptoms and have been associated with more severe disease and/or poorly controlled disease [27].

Comorbidities were assessed using two commonly used classification systems: Centers for Medicare and Medicaid (CMS) Chronic Condition Data Warehouse (CCW) morbidity definitions [28], and morbidities summarized by Elixhauser and colleagues [29, 30]. Additional baseline diagnoses, symptoms, and procedures were also identified (see Tables 1, 2). Patients with medium or high complexity of COPD were identified using criteria developed by Mapel et al. [31].

Table 1

Baseline demographics and comorbidities, treatment at index

Baseline characteristic	Total (n = 5475)	%	ICS/LABA or LAMA (n = 4991)	%	Triple (n = 484)	%	p value
Age (mean, SD)	70.55	11.5	70.60	11.5	70.02	11.3	0.29
Male	2393	43.7	2145	43.0	248	51.2	<0.001
Health insurance at index
Commercial	1778	32.5	1606	32.2	172	35.5	0.13
Medicaid	655	12.0	594	11.9	61	12.6	0.65
Medicare	3071	56.1	2818	56.5	253	52.3	0.08
CMS comorbidities
Atrial fibrillation	534	9.8	490	9.8	44	9.1	0.61
Cancers	370	6.8	344	6.9	26	5.4	0.20
Chronic kidney disease	870	15.9	789	15.8	81	16.7	0.59
Diabetes	1358	24.8	1243	24.9	115	23.8	0.58
Heart failure	986	18.0	891	17.9	95	19.6	0.33
Ischemic heart disease	1343	24.5	1214	24.3	129	26.7	0.26
Rheumatoid or osteo-arthritis	1443	26.4	1349	27.0	94	19.4	<0.001
Stroke/TIA	387	7.1	353	7.1	34	7.0	0.97
Any CMS comorbidity	3615	66.0	3315	66.4	300	62.0	0.05
Elixhauser comorbidities
Congestive heart failure	903	16.5	825	16.5	78	16.1	0.82
Deficiency anemias	773	14.1	704	14.1	69	14.3	0.93
Depression	917	16.7	837	16.8	80	16.5	0.89
Diabetes (without chronic complications)	1304	23.8	1201	24.1	103	21.3	0.17
Diabetes (with chronic complications)	442	8.1	406	8.1	36	7.4	0.59
Fluid and electrolyte disorders	686	12.5	614	12.3	72	14.9	0.10
Hypertension	3374	61.6	3098	62.1	276	57.0	0.03
Hypothyroidism	1032	18.8	936	18.8	96	19.8	0.56
Obesity	717	13.1	671	13.4	46	9.5	0.01
Other neurological disorders	514	9.4	471	9.4	43	8.9	0.69
Peripheral vascular disease	878	16.0	793	15.9	85	17.6	0.34
Psychoses	573	10.5	534	10.7	39	8.1	0.07
Pulmonary circulation disease	625	11.4	545	10.9	80	16.5	<0.001
Renal failure	468	8.5	425	8.5	43	8.9	0.78
Rheumatoid arthritis	343	6.3	319	6.4	24	5.0	0.21
Solid tumor without metastasis	533	9.7	493	9.9	40	8.3	0.25
Valvular disease	630	11.5	567	11.4	63	13.0	0.28
Weight loss	327	6.0	298	6.0	29	6.0	0.98
Any Elixhauser diagnosis	5028	91.8	4601	92.2	427	88.2	0.002
Other comorbidities
Asthma	1813	33.1	1669	33.4	144	29.8	0.01
Hypoxemia	1994	36.4	1756	35.2	238	49.2	<0.0001
Pneumonia	1908	34.8	1730	34.7	178	36.8	0.35

p values are from Chi-square test (percentages) and Student’s t test (continuous variables)

CMS Centers for Medicare and Medicaid, ICS inhaled corticosteroid, LABA long-acting beta-agonist, LAMA long-acting muscarinic antagonist, SD standard deviation, TIA transient ischemic attack

Table 2

Baseline COPD characteristics and related utilization, treatment at index

Baseline characteristic	Total (n = 5475)	%	ICS/LABA or LAMA (n = 4991)	%	Triple (n = 484)	%	p value
COPD complexity
Low complexity	2740	50.0	2556	51.2	184	38.0	<0.0001
Medium complexity	2510	45.8	2239	44.9	271	56.0	<0.0001
High complexity	225	4.1	196	3.9	29	6.0	0.03
Emphysema diagnosis	1210	22.1	1048	21.0	162	33.5	<0.0001
SABA use
No SABA use	2314	42.3	2119	42.5	195	40.3	0.36
SABA use 1–15 % of year	1435	26.2	1287	25.8	148	30.6	0.02
SABA use 16–40 % of year	803	14.7	756	15.1	47	9.7	0.001
SABA >40 % of year	923	16.9	829	16.6	94	19.4	0.12
SAMA use
Any SAMA	632	11.5	572	11.5	60	12.4	0.54
Any SABA/SAMA	876	16.0	784	15.7	92	19.0	0.06
Any SAMA or SABA/SAMA	1381	25.2	1241	24.9	140	28.9	0.05
ICS use, any	1752	32.0	1616	32.4	136	28.1	0.05
Oxygen use
No oxygen	3090	56.4	2851	57.1	239	49.4	0.001
Oxygen ≤50 % of year	876	16.0	781	15.6	95	19.6	0.02
Oxygen >50 % of year	1509	27.6	1359	27.2	150	31.0	0.08
Symptoms
Dyspnea diagnoses
Breathlessness
None	3454	63.1	3170	63.5	284	58.7	0.04
1	1051	19.2	943	18.9	108	22.3	0.07
>1	970	17.7	878	17.6	92	19.0	0.44
Shortness of breath
None	3450	63.0	3175	63.6	275	56.8	0.003
1	1128	20.6	1021	20.5	107	22.1	0.39
>1	897	16.4	795	15.9	102	21.1	0.004
Any wheezing	419	7.7	374	7.5	45	9.3	0.15
Any breathlessness, shortness of breath, or wheezing	3057	55.8	2751	55.1	306	63.2	<0.001
Other symptoms
Chest pain unspecified	1662	30.4	1506	30.2	156	32.2	0.35
Painful respiration	272	5.0	248	5.0	24	5.0	0.99
Any chest pain	1742	31.8	1578	31.6	164	33.9	0.31
Other malaise/fatigue	1379	25.2	1252	25.1	127	26.2	0.58
Cough	1963	35.9	1804	36.1	159	32.9	0.15
Procedures
Chest CT	822	15.0	736	14.7	86	17.8	0.08
Chest X-ray	3432	62.7	3118	62.5	314	64.9	0.30
Echocardiography	1113	20.3	1019	20.4	94	19.4	0.60
ECG	2452	44.8	2238	44.8	214	44.2	0.79
Nebulizer treatment	1771	32.3	1586	31.8	185	38.2	0.004
Non-invasive ventilator	339	6.2	310	6.2	29	6.0	0.85
Spirometry	2284	41.7	2041	40.9	243	50.2	<0.0001
Any vaccination (influenza or pneumonia)	2632	48.1	2404	48.2	228	47.1	0.66
COPD-related exacerbations
Index and 2 weeks prior
Any moderate	783	14.3	708	14.2	75	15.5	0.44
Any severe	434	7.9	367	7.4	67	13.8	<0.0001
Any other hospitalizationsa	124	2.3	107	2.1	17	3.5	0.05
Prior 3–12 weeks
Any moderate	894	16.3	806	16.1	88	18.2	0.25
Any severe	337	6.2	293	5.9	44	9.1	0.005
Any other hospitalizationsa	209	3.8	190	3.8	19	3.9	0.90
Prior 13–52 weeks
No moderate	4200	76.7	3835	76.8	365	75.4	0.48
1 Moderate	1018	18.6	920	18.4	98	20.2	0.33
>1 Moderate	257	4.7	236	4.7	21	4.3	0.70
Any severe	409	7.5	361	7.2	48	9.9	0.03
Any other hospitalizationsa	384	7.0	352	7.1	32	6.6	0.72
All inpatient utilization
Non-COPD inpatient (any)	852	15.6	773	15.5	79	16.3	0.63
COPD inpatient (any)	1117	20.4	983	19.7	134	27.7	<0.0001
Total inpatient (any)	1690	30.9	1510	30.3	180	37.2	0.002

p values are from Chi-square test (percentages) and Student’s t test (continuous variables)

COPD chronic obstructive pulmonary disease, CT computed tomography, ECG electrocardiogram, ICS inhaled corticosteroid, LABA long-acting beta-agonist, LAMA long-acting muscarinic antagonist, SABA short-acting beta-agonist, SAMA short-acting muscarinic agent

aCOPD as secondary diagnosis

Statistical Analyses

Analyses compared risk for a severe COPD exacerbation. In the ITT and AP analyses, Cox proportional hazard models were used to estimate a hazard ratio (HR) and 95 % confidence interval (CI) for severe exacerbation risk (measured as days without a severe exacerbation). Survival analyses started at day 30 since subjects could not have a severe exacerbation event within 30 days post-index. In the ITT approach, outcomes were attributed to subjects’ index treatment and subjects were censored at (1) 24 months, or (2) loss to follow-up, whichever was earlier. In the AP analysis, individuals were additionally censored if they discontinued their index treatment. The Kaplan–Meier estimator was used to estimate unadjusted survival functions within the ITT and AP analyses.

In the MSM analysis, the post-index period was partitioned into short intervals, similar to a clinical trial. Claims data was evaluated at the beginning of each time period, starting with the index date, enabling incorporation of time-varying factors post-index into analyses. The hazard ratio for severe exacerbation risk was approximated using a pooled logistic model that included baseline covariates and used generalized estimating equations (GEE) to adjust for the repeated, and therefore correlated, observations for each study subject across the time periods [32, 33]. The 24 months post-index was divided into seven periods ending at 12, 18, 26, 38, 52, 88, and 104 weeks.

In the MSM, observations were weighted to adjust for confounding due to treatment and censoring. Specific details about how the weights were calculated can be found in the ESM, and are briefly summarized here. For each observation, a treatment weight and a censoring weight were estimated that considered time-varying covariates [9, 34, 35]. The treatment and censoring weights were multiplied to arrive at the overall observation weight. The treatment weight was the inverse of the probability of receiving the treatment (IPTW) actually received at the beginning of each period, starting with the index date. Treatment probability models after the index date included baseline information and post-index information from the prior period (time-varying covariates). The censoring weight was estimated by looking forward and was the inverse of the probability of remaining uncensored (IPCW) for each time period. Individuals were allowed to switch treatment groups and were censored at (1) 24 months, (2) loss to follow-up, or (3) discontinuation of use of any of the index treatments (subjects could switch but had to use one of the index treatments). Censoring probability models included baseline information and post-index information for the current period. Since the treatment and censoring weights were the inverse of a probability estimate, very small probability values resulted in extremely large weights. Therefore, the IPTW and IPCW values were stabilized by a probability estimated from baseline covariates [33, 36]. Since the IPTW estimate at index only included baseline covariates, it was stabilized using the unadjusted probability for treatment at index [35].

All main analyses were pre-specified. SAS (Version 9.2) was used for statistical analyses. Analyses were two-tailed with a p value of <0.05 to determine statistical significance. Significant univariate differences between groups were determined using a Chi-square test for frequencies and Student’s t test for continuous variables. Adjusted odds ratios (OR) and 95 % CIs were estimated from logistic regression results.

Results

A total of 5475 subjects met inclusion criteria (see ESM). Of these, 9 % (n = 484) were using triple therapy at index and 91 % (n = 4991) either LAMA (35 %) or ICS/LABA (56 %). Tables 1 and 2 present baseline characteristics present in more than 5 % of either treatment group.

Treatment Changes

All subjects were managed care health plan members for at least 6 months post-index. In tracking medication use, less than optimum medication use was allowed (twice the days’ supply), but despite this, by 6 months post-index only 46 % (n = 2547) of subjects had persistently used study medications, with 88 % (n = 2238) of persistent users still using their same index medication and 12 % (n = 309) having switched.

Among those using triple therapy at index, 47 % (n = 226) were also using triple therapy at 6 months, 18 % (n = 86) had stepped down to only ICS/LABA or LAMA, and 35 % (n = 172) had discontinued use of both ICS/LABA and LAMA. In contrast, among those using ICS/LABA or LAMA at index, 41 % (n = 2067) were still using either at 6 months (1.0 % [n = 51] had switched treatments within the ICS/LABA or LAMA group), 3.4 % (n = 169) were using triple therapy, and 55 % (n = 2755) had discontinued either use. Table 3 summarizes subjects discontinuing use of any index medications in each time period, subjects disenrolling from the health plan, and among those not discontinuing or disenrolling, those having a severe exacerbation in the time period. Overall, discontinuation affected a smaller percentage of those using triple therapy, and for the MSM model, this was true for each of the seven time periods.

Table 3

Study subjects by follow-up time period

Time period (weeks elapsed)	Total subjects at beginning of period	Status at end of time period						Included in MSM analysisa
		Discontinued therapy		Disenrolled		Discontinued therapy and disenrolled		Total		Had a severe exacerbation during period
		N	%b	N	%b	N	%b	N	%b	N	%c
1 (12)	5475	2068	37.8	0	0.0	0	0.0	3407	62.2	146	4.3
2 (6)	3261	261	8.0	0	0.0	0	0.0	3000	92.0	48	1.6
3 (8)	2952	599	20.3	0	0.0	0	0.0	2353	79.7	67	2.8
4 (12)	2286	508	22.2	49	2.1	10	0.4	1719	75.2	72	4.2
5 (14)	1647	282	17.1	59	3.6	21	1.3	1285	78.0	47	3.6
6 (26)	1238	248	20.0	62	5.0	52	4.2	876	70.8	59	6.7
7 (26)	817	167	20.4	44	5.4	40	4.8	566	69.3	37	6.5
Total		4133		214		123				476

MSM marginal structural model

aSubjects still using an index therapy and still enrolled at end of the period

bPercentage of total subjects at beginning of period

cPercentage of subjects included in MSM analysis

Weighted Analyses for Treatment and Remaining Uncensored

In MSMs, a key point is that the overall mean of the stabilized weights for each time period should approximate 1.0, with smaller weight value ranges preferred [35, 37]. Extreme values or means other than 1.0 may be indicative of misspecified weight models or of subjects extremely unlikely to have received one of the study medications [35]. Mean weight values were approximately 1.0 for all periods in this analysis and ranges did not contain extreme values (see ESM).

Given that the MSM observation weights include baseline covariates in both the estimation of the inverse probabilities for treatment and censoring and the stabilizing probabilities, the time-varying covariates are the influential components of the stabilized weights. Table 4 provides OR estimates for time-varying covariates included in the MSM regression models for periods 2–7 of the treatment weight models and for all periods of the censoring weight models. Baseline covariates were not incorporated in the treatment stabilizing probability for period 1, so baseline covariates relating to exacerbations and SABA use are also presented. For these, at index, only having a severe exacerbation in the 2 weeks prior to the index date was significantly associated with triple therapy use at index. In the follow-up period, triple therapy at index was the strongest factor related to later use of triple therapy (p < 0.0001), but prior period SABA use and moderate exacerbation experience were also found to be significantly associated with use of triple therapy through the follow-up period (p = 0.01 and p = 0.006, respectively). As the follow-up periods progressed, the odds of using triple therapy increased (p = 0.0001). Triple therapy at index was also strongly related to remaining uncensored in the first 12-week period (p < 0.0001), but less so in the subsequent periods (p = 0.09). And as the follow-up periods progressed, the odds of remaining uncensored decreased (p < 0.0001). Current SABA use (p < 0.0001), moderate exacerbations (p = 0.01), and hospitalizations with secondary COPD-related diagnoses (p = 0.01) were all significantly associated with remaining uncensored in the first 12-week period, but moderate exacerbation experience was not in later periods. Hospitalizations that were not severe exacerbations were positively associated with remaining uncensored in the first 12 weeks, but then had a negative association in later periods.

Table 4

Odds ratios for triple therapy treatment and remaining on one of the study treatments, influential covariates, marginal structural model analysis

Factor	Triple therapy at the beginning of period				Using a study treatment during period
	Period 1		Periods 2–7		Period 1			Periods 2–7
	OR (95 % CI)	p value	OR (95 % CI)	p value	OR (95 % CI)	p value	OR (95 % CI)		p value
Triple therapy at index			36.9 (28.3–47.9)	<0.0001	2.50 (1.97–3.17)	<0.0001	1.25 (0.96–1.61)		0.09
Baseline covariates
SABA use, % of baseline 12-month period
None vs 1–15 %	0.93 (0.70–1.23)	0.61
15–40 % vs 1–15 %	0.71 (0.47–1.06)	0.09
>40 % vs 1–15 %	1.20 (0.85–1.69)	0.30
Moderate exacerbation, any pre-index
0–2 weeks pre-index	1.29 (0.93–1.79)	0.12
3–12 weeks pre-index	1.05 (0.77–1.44)	0.76
13–52 weeks pre-index	1.03 (0.78–1.37)	0.82
Severe exacerbation, any pre-index
0–2 weeks pre-index	1.90 (1.24–2.92)	0.003
3–12 weeks pre-index	0.94 (0.61–1.47)	0.80
13–52 weeks pre-index	1.09 (0.71–1.65)	0.70
Hospitalization with secondary COPD-related diagnoses, 13–52 weeks pre-index	1.05 (0.67–1.66)	0.83
Time-varying covariates
Number of 2-week periods			1.02 (1.01–1.03)	0.0001			0.96 (0.96–0.97)		<0.0001
Prior period
% of period with moderate exacerbations			1.95 (1.22–3.14)	0.006
Number of hospitalizations with secondary COPD-related diagnoses			1.01 (0.66–1.55)	0.97
Any SABA use in period			1.28 (1.05–1.56)	0.01
Current period
% of period with moderate exacerbations					1.78 (1.13–2.80)	0.01	0.89 (0.61–1.30)		0.56
Number of hospitalizations with secondary COPD-related diagnoses					1.67 (1.12–2.51)	0.01	0.74 (0.58–0.94)		0.01
Any SABA use in period					1.47 (1.26–1.73)	<0.0001	1.52 (1.36–1.70)		<0.0001

COPD chronic obstructive pulmonary disease, OR odds ratio, SABA short-acting beta-agonist

Severe Exacerbation Risk

Figure 1 shows the unadjusted survival functions by index treatment for the ITT and AP analyses. In the ITT analysis, 855 (15.6 % of 5475 subjects) had a severe exacerbation event in the first 2 years, and in the AP analysis, 411 did (7.5 % of 5475 subjects). After adjusting for baseline characteristics, the estimated HR for triple therapy at index for severe exacerbation risk in the ITT analysis was 1.24 (95 % CI 1.00–1.53). In the AP analysis it was 1.00 (95 % CI 0.73–1.36) (Fig. 2). Baseline characteristics significantly associated with greater severe exacerbation risk in both analyses were similar (data not shown); among them, pneumonia (p < 0.05), higher COPD complexity (p ≤ 0.02), any SAMA or SABA/SAMA use (both p < 0.02), and oxygen use (p ≤ 0.01). Greater than 40 % baseline SABA use was significant for ITT only (ITT, p < 0.001; AP, p = 0.07).

Fig. 1

Unadjusted survival functions by index treatment, intent to treat (a) and as protocol (b). ICS inhaled corticosteroid, LABA long-acting beta-agonist, LAMA long-acting muscarinic antagonist

Fig. 2

Adjusted risk estimates for a severe exacerbation after treatment initiation. (Asterisk) risk estimates are the point estimate with 95 % confidence interval

In the MSM analyses, the HR for patients using triple therapy was still elevated, but there was not a significantly higher risk. Adjusting for all baseline covariates included in the ITT and AP models, the estimated HR for severe exacerbation was 1.11 (95 % CI 0.68–1.81). Baseline characteristics significantly associated with greater severe exacerbation risk were similar to those for the ITT and AP analyses; and, as with the AP analysis, >40 % baseline SABA use did not reach significance (p = 0.07).

Discussion

We designed this study to examine the potential benefits of a MSM analysis approach contrasted to traditional models when comparing the effectiveness of COPD treatments using retrospective claims data. In this analysis of study methods, we compared use of long-acting therapies and their effectiveness in reducing severe exacerbations, currently a topic of great interest in COPD management. The ITT approach suggested that persons initiating triple therapy had significantly higher risk of subsequent severe COPD exacerbations. However, the AP and the MSM analyses found that by accounting for adherence to therapy during the follow-up period, there was no significant increased risk. The MSM analysis had some advantages over the AP method; including a larger number of patients in the study, and allowing examination of time-varying clinical factors occurring after initiation of COPD therapy that may affect treatment choices or outcomes. Patients with chronic disease are frequently not adherent to their prescribed controller therapies. This analysis demonstrates that it is important to consider treatment changes and clinical factors that affect treatment and outcomes when conducting comparative effectiveness studies.

We found that the MSM approach provides useful information on time-varying confounders in effectiveness studies. We adjusted for time-varying confounders (i.e., moderate exacerbation events, use of rescue medications (SABA), and inpatient stays with a secondary COPD-related diagnosis) by incorporating their effects through the stabilized weights in the MSM analysis. We used separate models for the first 12 weeks post-index and the time periods following that up until 2 years post-index. Two of the time-varying confounding factors also appeared to be time-modified in the censoring models [8]. That is, the effect of moderate exacerbations and hospitalizations with a secondary diagnosis of COPD changed between the first 12 weeks and later time periods. This is an aspect that bears further investigation in future research studies of COPD treatments and should be considered in other chronic disease treatment studies.

In our study, the MSM risk estimate was midway between the AP estimate of 1.00 and the ITT estimate of 1.24. The review by Suarez and colleagues [38] found that in 40 % of exposure–outcome associations (measured as OR or coefficient of linear regression), the MSM estimate differed by at least 20 % from the conventional estimate. It has been suggested that the full power of the MSM approach is best appreciated when there are numerous time-varying covariates [39], when there is evidence of strong confounding by time-dependent covariates, and past treatment has a sizable effect on the covariates [40]. In our study there was substantial discontinuation of treatment. While there was some switching from triple therapy use to ICS/LABA or LAMA use, there was minimal switching from ICS/LABA or LAMA to triple therapy. An MSM approach may offer greater explanatory benefit when there is more movement between treatment groups than was present in this study. Although it would increase the complexity of the analysis and require a larger sample size than ours, a multinomial treatment probability model that would allow for more than two treatment choices and that included no long-acting treatment as an option may provide additional information [41]. Discontinuation of treatment was treated as censoring in our model, but reasons for discontinuation that may not be apparent in the claims data include patient perceived lack of need for medication or ineffectiveness of medication, adverse events from medications or COPD not captured in claims data, and adverse events from comorbid conditions not included in the study design [18, 25, 42].

Comparison with Previous Studies

From our literature review, MSMs had not been utilized in COPD treatment effectiveness studies, and a minority of published MSM studies concern chronic disease treatments [38, 43]. It has been speculated that advantages for COPD patients may be gained from long-acting triple therapy since ICS/LABA combination therapies are known to have anti-inflammatory properties, and the LAMA therapy, tiotropium, had been shown to reduce exacerbations in the absence of any anti-inflammatory activity [44].

One RCT study has been conducted comparing triple therapy to LAMA (tiotropium) use, finding a reduced but not significant OR for triple therapy (0.85, 95 % CI 0.52–1.38) using an ITT analysis during a 52-week study period [26]. Similar to our study, discontinuation of triple therapy among patients was differential between treatment arms: 47 % for tiotropium, and 26 % for triple therapy subjects.

Effectiveness estimates from our study were contrary to two retrospective observational studies that found better outcomes to be associated with triple therapy use [3, 4]. In those studies, no information was provided on the degree to which patients discontinued or switched medications post-index. The second retrospective analysis used a US population and was somewhat similar in sample size to our study (852 triple therapy, 2481 LAMA) [3]. Analyses were ITT, and patients treated with LAMA who later switched to triple therapy were excluded from the study. Study subjects had at least one COPD-related exacerbation event and at least one claim for a SAMA medication in the 12-month pre-index period, and post-index had at least two LAMA claims. Despite these additional requirements, the population was similar to ours in that triple therapy patients were more likely to be younger and to have had a baseline period inpatient stay for COPD. However, baseline indicators of more unstable disease (rescue inhaler use) were all lower among triple therapy patients compared with the non-triple (LAMA) group [3]—associations that were reversed for our study. For this study, and for ours, the treatment group with the higher prevalence of baseline adverse indicators also had the higher risk for post-index severe exacerbations, despite adjustment for these characteristics, suggesting that residual and unmeasured confounding may have been present in both studies.

We hypothesized that the MSM model would highlight that the experience of moderate exacerbations and related events (other hospitalizations and SABA use) were associated with post-index treatment decisions. Our results from the treatment and censoring weight calculations for the MSM analysis support that hypothesis. In the first 12 weeks post-index, subjects who had higher probability of remaining uncensored (of continuing to use one of the study COPD LABD treatments) were those who were having moderate exacerbations, using SABA medication, and/or having hospitalizations for which COPD was a secondary diagnosis. Our analyses serve to highlight that issues related to poor treatment adherence and exacerbation experience have substantial impact in observational COPD studies.

Limitations of the Study

Limitations of this study include the potential for unmeasured confounders. Additional information not available in our dataset may have improved treatment probability models, including provider specialty, spirometry measurements to assess disease severity, patient-reported symptoms, economic status of patients, and information about patient prescription copayments.

Subjects were censored in our study when there was no evidence of persistence to any of the study medications given an allowable gap period. There is the possibility that the outpatient pharmacy database was incomplete and that patients were filling prescriptions outside of the managed care system. However, we have no evidence to suggest that if this occurred it would be differential between the treatment groups being compared. Finally, there may be misclassification due to the exacerbation measures used for severe and moderate exacerbations. Because this was a retrospective analysis and we did not utilize medical chart information, we cannot verify whether events were in fact events indicative of worsening symptoms related to COPD. However, definitions and criteria for these events were the same as have been used for other COPD treatment effectiveness studies, allowing comparison across studies.

Conclusions

Few MSM studies have been conducted for chronic disease treatment effectiveness and none for COPD [38, 43]. This study provided a comparison of three different retrospective observational study design analysis approaches. ITT analyses demonstrated outcomes for the initial treatment groups as assigned. Estimates from MSMs are meant to reflect the effect of full adherence, or as patients as treated [33], however, discontinuation of all therapies was prevalent in this study sample. The ITT and AP analyses showed dramatic risk estimate differences, demonstrating the potential for the existence of underlying differences between the two groups as treated. The MSM analysis helped to emphasize how salient events associated with unstable disease (e.g., moderate exacerbations and SABA use) are associated with use of triple therapy and treatment persistence.

It has been stated that “exacerbations are heterogeneous events occurring in a heterogeneous disease” [45]. Understanding how patient characteristics cluster and how clusters relate to both treatment propensity and outcomes may help to improve disease management for patients with COPD. The MSM approach could be a useful tool for identifying the relationship between those clusters and treatment and outcomes.

The full advantages of the MSM approach may not have been illustrated in this study due to minimal switching from the non-triple to the triple therapy group. Nonetheless, this study highlighted the importance of understanding post-index events such as treatment switching and discontinuation that may confound assessment of chronic disease treatment effectiveness. Based on this COPD effectiveness study, we find that MSMs may be a useful and informative complementary analysis to include in studies of treatment effectiveness in chronic disease where time-varying confounding is present, and switching between treatments is common.

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 73 kb)

Marginal structural models (MSMs) are informative, account for poor treatment adherence or switching, and allow inclusion of more patients than simply censoring patients in an ‘as protocol’ (AP) analysis. MSMs provide insights into time-varying factors occurring after initiation of disease controller therapy that may affect treatment choices or outcomes, and are not apparent in intent-to-treat (ITT) or AP analyses.

Retrospective comparative effectiveness studies using ITT or AP analysis methods often fail to include treatment adherence or switching in their analyses, leading to biased effect estimates. Other time-varying factors such as acute exacerbations of chronic disease can affect treatment decisions and outcomes, and thus also introduce biases. In this analysis of retrospective data from two regional health systems, we demonstrate that failure to account for treatment adherence can make the outcomes of patients who use controller therapies concurrently appear to be significantly worse than those of patients who use these treatments independently.

Based on this effectiveness study, we find that MSMs may be a useful and informative complementary analysis to include in studies of treatment effectiveness in chronic disease where time-varying confounding is present, and switching between treatments is more common.