Influence of healthy candidate bias in assessing clinical effectiveness for implantable cardioverter-defibrillators: cohort study of older patients with heart failure.

OBJECTIVE: To assess the potential contribution of unmeasured general health status to patient selection in assessments of the clinical effectiveness of implantable cardioverter-defibrillator (ICD) therapy.
DESIGN: Retrospective cohort study.
SETTING: Linked data from an ICD registry, heart failure registry, and Medicare claims data for ICDs implanted in 2005 through 2009. PARTICIPANTS: 29,426 patients admitted to hospital with heart failure aged 66 years or older and eligible for ICD therapy for primary prevention. MAIN OUTCOME MEASURES: Non-traumatic hip fracture, admission to a skilled nursing facility, and 30 day mortality-outcomes unlikely to be improved by ICD therapy.
RESULTS: Compared with 17,853 patients without ICD therapy, 11,573 patients with ICD therapy were younger and had lower ejection fraction and more cardiac admissions to hospital but fewer non-cardiac admissions to hospital and comorbid conditions. Patients with ICD therapy had greater freedom from unrelated events after adjusting for age and sex: hip fracture (hazard ratio 0.77, 95% confidence interval 0.64 to 0.92), skilled nursing facility admission (0.53, 0.50 to 0.55), and 30 day mortality (0.12, 0.10 to 0.15).
CONCLUSIONS: Lower risks of measured outcomes likely reflect unmeasured differences in comorbidity and frailty. The findings highlight potential pitfalls of observational comparative effectiveness research and support physician consideration of general health status in selecting patients for ICD therapy.

Introduction

After several landmark clinical trials showed the efficacy of implantable cardioverter-defibrillator (ICD) therapy,1 2 the US Centers for Medicare & Medicaid Services (CMS) expanded coverage of ICD therapy to include primary prevention of sudden cardiac death in Medicare beneficiaries with heart failure.3 Medicare beneficiaries mostly consist of those aged 65 years or older, in whom the risk of sudden cardiac death is lower than in the trial patients.4 5 6 None the less, one study projected that ICD therapy may be indicated for as many as 800 000 additional people with heart failure in the United States, most of whom are 65 years or older.7

In real world modern clinical practice, patients are selected to receive ICD therapy on the basis of multiple factors, including indications and contraindications, underlying disease severity, comorbid conditions, and overall prognosis. Guideline recommendations for both heart failure and device based therapy include “anticipated survival of at least a year with good functional capacity,” which is a crucial but subjective clinical assessment. In older patients, cognitive and functional capacity and social support can influence not only prognosis but also preferences for a device that prevents sudden cardiac death without improvement in quality of life. Information about these factors is often unavailable in large databases, including claims data, registries, and medical records, which affect the validity of assessing comparative effectiveness of ICD therapy in real world patients using these data.

To understand the contribution of unmeasured general health status to selection of patients for ICD therapy and potential bias in observational studies using registries and claims data, we compared three outcomes among patients with and without ICD therapy that can reflect general health status but are unlikely to be improved by ICD therapy: hip fracture, admission to a skilled nursing facility, and 30 day mortality.

Methods

Data sources

We conducted a retrospective cohort study of older patients with heart failure using the CMS ICD registry (2005-08); the National Cardiovascular Database Registry’s ICD registry (2005-08); a national clinical registry of patients with heart failure aggregated from several quality improvement and accreditation programs, including the American Heart Association’s Get With the Guidelines program (2005-08); and data files on Medicare institutional and non-institutional claims (2004-09) (see web extra appendix I).

We linked the combined ICD registry and the heart failure registry to Medicare claims data using four non-unique identifiers: date of birth, sex, admission date for implantation of the ICD, and provider ID, which is described in detail elsewhere.8 Briefly, we validated this linkage among the subset of 196 923 patients who had a unique identifier in the ICD registry. Our linkage using non-unique identifiers yielded 98% specificity, 95% sensitivity, and 98% positive predictive value compared with the linkage method using both non-unique and unique identifiers.8

Study population, cohorts, and treatment status

Our study population represented patients with heart failure who were 66 years or older and eligible for ICD therapy for primary prevention (see web extra appendix II for detailed information on the study population). We identified two cohorts of patients with and without ICD therapy in the linked databases (fig 1). Cohort 1 consisted of the population of Medicare beneficiaries who were linked to the ICD registry (areas A and B in fig 1) and the population of Medicare beneficiaries who were linked to the heart failure registry but not with the ICD registry (area C). Cohort 2, a smaller but conceptually more appropriate study population, consisted of Medicare beneficiaries who were linked to the heart failure registry with ICD therapy (area B) and without ICD therapy (area C). We conducted analyses in both cohorts to ensure that the findings were similar. All study patients had an ejection fraction of 35% or less. We excluded patients who had a history of cardiac arrest or sustained ventricular tachycardia. All patients were eligible for Medicare for one year before the index procedure or the index admission to hospital and were 66 years or older. We excluded patients who underwent both cardiac resynchronization therapy and ICD therapy, because biventricular pacing improves cardiac function and could improve patients’ functional status, potentially reducing the risk of the outcomes of interest.

Fig 1 Conceptual relation of data sources and schematic presentation of cohort 1 (implantable converter-defibrillator (ICD)+heart failure registry population) and cohort 2 (heart failure registry population). Cohort 1 consists of patients receiving ICD therapy from overlapping area between ICD registry and Medicare data (area A+B) and patients not receiving ICD therapy from overlapping area between heart failure registry and Medicare data but no overlap with ICD registry (area C). Cohort 2 consists of patients receiving ICD therapy from overlapping area between the heart failure registry, ICD registry, and Medicare data (area B) and patients not receiving ICD therapy from area C

Outcomes

We selected three outcomes that are not causally associated with ICD therapy, are markers for frailty and other unmeasured pre-existing patient characteristics, and can be accurately measured with high specificity in Medicare data. These included admission for non-traumatic hip fracture (that is, fracture not due to trauma other than a fall and often a consequence of osteoporosis and a fall), admission to a skilled nursing facility (a licensed facility that provides short term or long term skilled care, commonly called a “nursing home”), and 30 day mortality. We identified admissions to hospital for non-traumatic hip fracture on the basis of a diagnosis of hip fracture (international classification of diseases, 9th revision, clinical modification (ICD-9-CM) diagnosis code 820.xx or 733.14) and a related procedure code (ICD-9-CM procedure code 78.55, 79.05, 79.15, 79.25, 79.35, or 79.65 or current procedural terminology (CPT) code 27230-27248).9 We identified admissions to a skilled nursing facility in the Medicare institutional files. Because some such skilled nursing facility admissions identified in Medicare data are short term and may not be totally related to frailty, we also identified a subset of nursing home admissions that resulted in stays of more than 20 days (beyond the period for full Medicare coverage) or death within 20 days as a more accurate indicator of frailty. We chose mortality within 30 days, a timeframe during which prophylactic ICD therapy did not provide a benefit in the landmark clinical trials. For example, the Sudden Cardiac Death in Heart Failure Trial (SCD-HeFT) found that the mortality benefit started to appear approximately 1.5 years after implantation of an ICD.1 All patients were followed until the occurrence of the outcomes and death from the index date, which was the date of implantation for those who received ICD therapy and the discharge date from their admission to hospital for heart failure for those who did not receive ICD therapy.

Statistical analysis

In patients with or without ICD therapy, we described baseline characteristics, including use of several preventive care and screening tests as markers for health seeking behaviors. We reported counts and person time for each outcome and graphed cumulative outcome event risks over time using Kaplan-Meier estimators. We used proportional hazards regression to derive crude and adjusted hazard ratios comparing patients with and without ICD therapy.

Some values were missing for systolic blood pressure and serum levels of sodium, B type natriuretic peptide, and creatinine in some patients (see table 1). We handled missing values by using multiple imputation10 and assuming an underlying multivariate normal distribution. Our analysis was based on five imputed datasets where the imputation model included all variables in the outcome model (ICD use outcomes and potential confounders) as well as variables related to missingness (see web extra appendix IV).11

Sensitivity analyses

We repeated the same analyses after excluding patients older than 80 years and those with a previous admission to a skilled nursing facility, because the indication criterion of “expected survival with good functional capacity for >1 year” is less likely to be met for these patients.

We used SAS version 9.2 (SAS Institute, Cary, NC) for all analyses.

Results

We identified 29 426 patients with heart failure (11 573 with ICD therapy and 17 853 without ICD therapy) in cohort 1 who met the eligibility criteria. Patients who received ICD therapy were younger and more likely to be men and white than patients who did not receive ICD therapy. Patients with ICD therapy also had lower ejection fractions, more previous admissions to hospital for cardiac diseases, more physician visits, and ischemic etiology of heart failure (table 1). Patients without ICD therapy had a higher prevalence of non-cardiac admissions to hospital, previous admissions to a skilled nursing facility, more chronic kidney or lung disease, metastatic cancer, and other non-cardiovascular diseases. The prevalence of many preventive procedures and screening tests was higher in patients with ICD therapy. These findings were similar in cohort 2.

Table 1

 Measured baseline characteristics of patients with or without implantable cardioverter-defibrillator (ICD) therapy. Values are numbers (percentages) unless stated otherwise

Characteristics	Cohort 1 (ICD+heart failure registry population)			Cohort 2 (heart failure registry population)
	No ICD (n=17 853)	ICD (n=11 573)		No ICD (n=17 853)	ICD (n=1089)
Mean (SD) age (years)	80.0 (7.8)	75.0 (5.8)		80.0 (7.8)	75.1 (5.9)
Men	9321 (52)	8514 (74)		9321 (52)	770 (71)
White race	15 068 (84)	10 035 (87)		15 068 (84)	888 (82)
Previous admissions and outpatient visits:
≥1 heart failure admission	2844 (16)	2196 (19)		2844 (16)	354 (33)
≥1 non-cardiac admission	6501 (36)	3290 (28)		6501 (36)	318 (29)
≥1 SNF admission	2679 (15)	663 (6)		2679 (15)	84 (8)
Mean (SD) outpatient visits	10.3 (9.4)	12.7 (8.8)		10.3 (9.4)	13.0 (8.7)
Medical history:
Cancer
Any cancer*	3020 (17)	1946 (17)		3020 (17)	177 (16)
Metastatic cancer	566 (3)	161 (1)		566 (3)	17 (2)
Cerebrovascular disease	3860 (22)	2150 (19)		3860 (22)	229 (21)
Chronic kidney disease	8009 (45)	3819 (33)		8009 (45)	407 (37)
COPD	8283 (46)	4908 (42)		8283 (46)	523 (48)
Dementia	3503 (20)	947 (8)		3503 (20)	83 (8)
Depression	2821 (16)	1282 (11)		2821 (16)	142 (13)
Diabetes mellitus	8648 (48)	5698 (49)		8648 (48)	584 (54)
Dialysis	527 (3)	246 (2)		527 (3)	26 (2)
Gastrointestinal ulcer or bleeding	2755 (15)	1405 (12)		2755 (15)	131 (12)
Heart failure of ischemic origin	14 165 (79)	10 028 (87)		14 165 (79)	954 (88)
Liver disease	1118 (6)	629 (5)		1118 (6)	73 (7)
Peripheral vascular disease	4147 (23)	2591 (22)		4147 (23)	270 (25)
Clinical characteristics:
Median (interquartile range) Charlson comorbidity score (%)	3 (1-4)	2 (1-4)		3 (1-4)	3 (1-4)
Median (interquartile range) eGFR (mL/min/1.73 m2)†	47.1 (32.3-63.5)	58.4 (43.4-73.5)		47.1 (32.3-63.5)	58.0 (42.8-72.3)
Median (interquartile range) ejection fraction (%)	29 (20-33)	25 (20-30)		29 (20-33)	25 (20-30)
Median (interquartile range) serum BNP (pg/mL)	1249 (657-2258)	681 (304-1477)		1249 (657-2258)	842 (346-1560)
Missing	12,810 (72)	8989 (78)		12,810 (72)	786 (72)
Median (interquartile range) serum creatinine (mg/dL)	1.3 (1.0-1.8)	1.2 (1.0-1.5)		1.3 (1.0-1.8)	1.2 (1.0-1.5)
Missing	11 488 (64)	114 (1)		11 488 (64)	14 (1)
Median (interquartile range) serum sodium (mEq/L)	138 (136-141)	139 (137-141)		138 (136-141)	139 (137-141)
Missing	12 032 (67)	157 (1)		12 032 (67)	27 (2)
Median (interquartile range) systolic blood pressure (mm Hg)	133 (116-152)	131 (118-148)		133 (116-152)	130 (115-148)
Missing	9230 (52)	196 (2)		9230 (52)	19 (2)
Screening and prevention history:
Bone mineral density test‡	677 (8)	373 (12)		677 (8)	34 (11)
Fecal occult blood test	1111 (6)	876 (8)		1111 (6)	82 (8)
Influenza vaccine	8423 (47)	6117 (53)		8423 (47)	547 (50)
Mammography†	1472 (17)	909 (30)		1472 (17)	86 (27)
Pneumococcal vaccine	1398 (8)	1076 (9)		1398 (8)	125 (11)
Prostate specific antigen test§	2622 (28)	2910 (34)		2622 (28)	262 (34)

COPD=chronic obstructive pulmonary disease; eGRF=estimated glomerular filtration rate; SNF=skilled nursing facility; BNP=B type natriuretic peptide.

*Does not include non-melanoma skin cancer.

†eGFR was calculated according to the Chronic Kidney Disease Epidemiology Collaboration equation42 after discounting serum creatinine values by 5% to account for assay variability.43

‡In women only.

§In men only.

During 42 580 person years of follow-up (mean, 1.4 years) in cohort 1, we observed 676 admissions to hospital for hip fracture and 9475 admissions to a skilled nursing facility (table 2). During the first 30 days, 2428 deaths occurred. Patients without ICD therapy had substantially higher incidence rates of non-traumatic hip fracture (17 v 9 per 1000 person years), admissions to a skilled nursing facility (354 v 112 per 1000 person years), and 30 day mortality (1699 v 165 per 1000 person years).

Table 2

 Incidence of admission to a skilled nursing facility (SNF), hospital admission for hip fracture, and 30 day mortality in patients with or without implantable cardioverter-defibrillator (ICD) therapy

Event	No ICD				ICD
	Person years	Events	Incidence/1000 person years (95% CI)		Person years	Events	Incidence/1000 person years (95% CI)
Cohort 1 (ICD and heart failure registry population):
Hip fracture admission	26 819	467	17 (16 to 19)		24 066	209	9 (8 to 10)
SNF admission	19 852	7037	354 (346 to 363)		21 718	2438	112 (108 to 117)
Death in SNF or length of stay >20 days	25 662	1995	78 (74 to 81)		23 814	556	23 (21 to 25)
30 day mortality	1337	2272	1699 (1631 to 1770)		945	156	165 (141 to 193)
Cohort 2 (heart failure registry population):
Hip fracture admission	26 819	467	17 (16 to 19)		2190	16	7 (4 to 12)
SNF admission	19 852	7037	354 (346 to 363)		1934	258	133 (118 to 150)
Death in SNF or length of stay >20 days	25 662	1995	78 (74 to 81)		2154	63	29 (23 to 37)
30 day mortality	1337	2272	1699 (1631 to 1770)		89	10	112 (58 to 199)

The cumulative risk of hip fracture at three years was 4.8% (95% confidence interval 4.3% to 5.3%) among patients without ICD therapy and 2.6% (2.3% to 3.1%) among patients with ICD therapy. The cumulative incidence curves diverged immediately after implantation of the ICD (fig 2). The cumulative risk of admission to a skilled nursing facility at one year was 13.4% (12.8% to 14.0%) with ICD therapy and 35.7% (34.9% to 36.4%) without ICD therapy. These curves diverged immediately and paralleled within a few days after implantation of the ICD (fig 2). The pattern was similar for 30 day mortality (fig 2). The findings were similar in cohort 2 (see web extra appendix V).

Fig 2 Crude survival curves of patients receiving and not receiving implantable converter-defibrillator (ICD) therapy in cohort 1 for admissions for non-traumatic hip fracture, admissions to a skilled nursing home, and 30 day all cause mortality

Unadjusted rates of outcomes in patients with ICD therapy were lower than in patients without ICD therapy for admission for non-traumatic hip fracture (hazard ratio 0.51, 95% confidence interval 0.43 to 0.59), admission to a skilled nursing facility (0.39, 0.37 to 0.41), and 30 day mortality (0.10, 0.09 to 0.12) (table 3). After adjustment for age and sex, these estimates moved toward 1. After adjustment for all patient measured characteristics, these estimates moved further but only moderately toward 1 and the risks remained substantially lower among patients who received ICD therapy (table 3).

Table 3

 Hazards of admission to a skilled nursing facility (SNF), hospital admission for hip fracture, and 30 day mortality in patients with or without an implantable cardioverter-defibrillator (ICD)

Event	Hazard ratio (95% CI)
	Unadjusted	Age and sex adjusted	Multivariable adjusted*
Cohort 1 (ICD and heart failure registry population):
Non-traumatic hip fracture	0.51 (0.43 to 0.59)	0.77 (0.64 to 0.92)	0.84 (0.66 to 1.05)
SNF admission	0.39 (0.37 to 0.41)	0.53 (0.50 to 0.55)	0.71 (0.67 to 0.76)
Death in SNF or length of stay >20 days	0.36 (0.33 to 0.39)	0.50 (0.45 to 0.56)	0.69 (0.60 to 0.80)
30 day mortality	0.10 (0.09 to 0.12)	0.12 (0.10 to 0.15)	0.20 (0.17 to 0.24)
Cohort 2 (heart failure registry population):
Non-traumatic hip fracture	0.42 (0.23 to 0.67)	0.63 (0.36 to 1.01)	0.53 (0.31 to 0.92)
SNF admission	0.47 (0.41 to 0.53)	0.60 (0.53 to 0.68)	0.68 (0.60 to 0.78)
Death in SNF or length of stay >20 days	0.45 (0.34 to 0.57)	0.61 (0.47 to 0.78)	0.78 (0.53 to 1.15)
30 day mortality	0.07 (0.03 to 0.12)	0.09 (0.04 to 0.15)	0.12 (0.06 to 0.22)

*Adjusted for all covariates listed in web extra appendix III, including clinical variables such as ejection fraction, systolic blood pressure, sodium level, serum B type natriuretic peptide, and estimated glomerular filtration rate.

The refined outcome for admission to a skilled nursing facility, which used a stay of 20 days or longer or death within 20 days, yielded similar results (tables 2 and 3). After adjustment for all measured factors, the risks remained significantly lower among patients with ICD therapy. Restricting the cohorts to patients 80 years or younger (see web extra appendix VI) or with no previous admission to a skilled nursing facility (see web extra appendix VII) did not change the results meaningfully.

Discussion

We compared older patients with and without implantable converter-defibrillator (ICD) therapy for the risk of three outcomes that are unlikely to be directly improved by ICD therapy to assess the impact of unmeasured patient characteristics on patient selection for ICD therapy. We observed a 50% to 60% lower risk for admission for non-traumatic hip fracture, admission to a skilled nursing facility, and 30 day mortality in patients with ICD therapy compared with patients with heart failure without ICD therapy. These differences in the risks of unrelated outcomes were observed immediately after implantation of the ICD. The differences lessened after adjustment for measured patient characteristics, but risks remained lower in patients who received ICD therapy by 16% to 34%. Given the timing and size of the observed differences, these results likely reflects baseline differences between the candidates for ICD therapy and patients who did not receive ICD therapy that are not measured in Medicare or registry data, leading to selection bias or “healthy candidate bias.” From a clinical perspective, these differences reflect appropriate integration of multiple health dimensions into clinical decision making in modern clinical practice to select patients who are most likely to benefit.

The healthy candidate bias that we observed is analogous to biases described in other fields of epidemiology. Occupational epidemiologists have long recognized the “healthy worker survivor effect” when evaluating effects of environmental exposures. This bias arises in comparisons of workers exposed to work related toxins and workers not exposed because relatively healthy people are likely to gain employment and remain employed, whereas severely ill and chronically disabled people are ordinarily excluded from employment.12 Similar bias, known as “healthy user bias” in pharmacoepidemiology, has been observed among users of preventive medications and vaccines, such as hormone replacement therapy,13 14 statins,15 16 and influenza vaccine.17 Because implantable devices and surgical procedures typically pose short term risks in exchange for long term benefits, patients at high risk of complications or deemed too sick to benefit are less likely to be selected. In the Cardiac Arrhythmia Suppression Trial, a similar effect was observed.18 This healthy candidate effect is one of the biggest threats to validity in observational comparative effectiveness research, especially in comparisons of invasive interventions to less invasive alternatives.

Frailty reflected by low functional status and impaired cognitive function, lack of social support, unrecognized or untreated depression, and diminished quality of life are associated with decreased survival in patients with heart failure.19 20 21 22 23 Each of these factors may be associated with the decision to offer and receive ICD therapy. For example, preference for survival was strongly associated with longer survival in patients with heart failure in the Evaluation Study of Congestive Heart Failure and Pulmonary Artery Catheter Effectiveness.24 Among patients who survived fewer than 105 days, 31% indicated that they would trade more than 90% of their survival days to feel well for the time remaining, compared with 6% of patients who survived all 180 days (P<0.005), indicating that those most likely to die were least likely to have cared about prolonging survival.24 In another survey study of 105 consecutive patients with mean ejection fraction of 21%, 65% of whom received ICD therapy for primary prevention, recipients of ICD therapy expressed greater confidence that the device would save their own lives than those without ICD therapy (P<0.001).25 These data indicate that recipients of ICD therapy tend to have a strong preference for survival, which is likely to correlate directly with lower rates of death, admission to a skilled nursing facility, and even hip fracture indirectly. Finally, incomplete depiction of non-cardiovascular comorbidity is likely another source of unmeasured differences between patients with and without ICD therapy.25 26

Clinical and policy implications

Eligibility criteria for ICD therapy from both the heart failure guidelines26 27 28 and the device based therapy guidelines29 28 from the American College of Cardiology, the American Heart Association, and the European Society of Cardiology include anticipated survival with “good functional capacity” for at least one year. We found that 13% of patients with ICD therapy and 36% of patients without ICD therapy were admitted to skilled nursing facilities within a year if they did not die. Considering a composite outcome of admission to a skilled nursing facility and death, 55% of patients without ICD therapy would not meet the expectation of one year survival with good functional capacity, whereas a smaller fraction (21%) of patients with ICD therapy would meet the expectation.

A few previous studies have concluded that ICD therapy may be underused in general or certain Medicare patients without specific contraindications to implantation of a ICD for primary prevention of sudden cardiac death.29 30 31 While existing cardiovascular registries and Medicare data include information on skilled nursing facilities and ICD-9-CM diagnosis codes for comorbid conditions, they lack documentation of the severity of comorbidity, socioeconomic factors, functional status, cognitive function, and quality of life. Therefore, apparent gaps in ICD use assessed using information available in registries and Medicare data may arise, at least in part, from incomplete depiction of the appropriateness of patient selection for ICD therapy.

Our results indicate that a substantial fraction of patients who were potentially eligible for ICD therapy but did not receive it may have been at high risk for not “surviving with good quality of life.” However, our study had limited ability to identify patients who were specifically at higher risk for admission to a skilled nursing facility. There are without doubt patients who did not receive ICD therapy who would have derived meaningful life extension from prevention of sudden death, which are also not possible to identify in our analyses. Furthermore, some patients may have declined ICD therapy despite an appropriate recommendation.

Implications for observational comparative effectiveness research

Although observational studies provide data on real world patients in real world settings, they are subject to potential bias by selection, confounding, survivor bias, and inaccurate measurement of variables.31 32 Confounding by indication is described well in the pharmacoepidemiology literature32 33and can cause significant bias. However, registries typically include information on indications and contraindications for treatments. Therefore, confounding by indication was not a problem in our study using linked registry and claims data. Healthy candidate bias, a type of selection or confounding bias, can be the most challenging bias in observational comparative effectiveness research for devices and surgeries. We showed the existence of substantial healthy candidate bias in our ICD example, which was not controlled by multivariable adjustments for measured factors and might have affected, at least in part, previous ICD effectiveness studies.33 34 35

Various design-based and statistical approaches have been suggested to overcome healthy worker survivor effect or healthy user bias. These include restriction to survivors,35 36 lagged time or latency analysis,36 37 g-computation,37 38 39 and comparison to active users of similar preventive medications.39 40 The technique of restriction to a subset of survivors or to more comparable groups is a standard approach in epidemiology but one that is sometimes overlooked or intentionally avoided in favor of a larger sample size. Other methods, including latency analysis and g-computation, should be considered and used when appropriate. Finally, adjustment for proxies for unmeasured factors by maximizing available information in large databases using appropriate methods (for example, high dimensional propensity score methods) can potentially achieve better control for bias in some settings.40 41

This study suggests two important implications. Firstly, those responsible for the allocation of scarce healthcare resources in any country must carefully evaluate whether the efficacy evidence of expensive technologies such as ICD therapy apply to their use outside of clinical trials. Secondly, evaluations of effectiveness from clinical and administrative databases may overestimate the benefit of device or surgical therapies for which selection is influenced by healthy candidate bias.

Several limitations should be considered in interpreting these findings. Firstly, we identified patients who received and did not receive ICD therapy from different registries, which limited our ability to adjust for some factors such as New York Heart Association functional classification that were only measured in one of the registries. However, all patients were admitted to hospital with heart failure, both registries had equally limited information on non-cardiovascular comorbid conditions, and neither registry included information on frailty or social support. Also, all patients were Medicare beneficiaries, which was the target population of our study, and the findings were replicated in the secondary cohort (cohort 2) identified in the heart failure registry. In the comparison of 30 day mortality, it is possible that there was early excess non-sudden death in the ICD arm, as reported after myocardial infarction.41 42 However, this would only have served to diminish the noticeable excess of 30 day mortality observed among patients who did not receive ICD therapy. For the non-fatal outcomes, competing risks by death exist in the current study. However, if ICD survival benefit from the trials was translated into this population, our results would have been underestimates of the true differences in the occurrence of hip fracture and admissions to a skilled nursing facility, rendering the selection due to frailty and other unmeasured factors even larger than observed here.

Conclusion

Difference in survival between patients receiving and not receiving ICD therapy in this large observational study was influenced by unmeasured differences in baseline health status, as indicated by lower risk of non-traumatic hip fracture, admission to a skilled nursing facility, and 30 day mortality among recipients of ICD therapy. Our analyses of early mortality and non-fatal outcomes highlight potential pitfalls of observational comparative effectiveness analyses of outcomes associated with one treatment versus another due to healthy candidate bias. It is not possible to determine from this study the degree to which apparent underutilization of such therapies may instead reflect thoughtful clinical decisions in which doctors use factors such as frailty to allocate expensive therapies to patients who are most likely to benefit.

Clinical trials of implantable cardioverter-defribrillators (ICDs) under-represent older patients with comorbidities, in whom the risk of sudden death might be lower, but many ICDs have been used in the United States and other countries

In clinical practice, treatments are not randomly allocated but selected based on indications and other considerations

Unmeasured selection factors can influence estimates of comparative effectiveness in observational studies, as shown by the healthy worker effect or healthy user bias in occupational research and pharmacoepidemiology

Evaluations of effectiveness from clinical and administrative databases may overestimate the benefit of device or surgical therapies for which selection is influenced by healthy candidate bias

Apparent underutilization of ICDs in older patients may instead reflect thoughtful clinical decisions in clinical practice in which clinicians use factors such as frailty to allocate expensive therapies to patients who are most likely to benefit