Defining unavoidable delays in primary percutaneous coronary intervention: discordance among patients excluded from National Cardiovascular Quality Registries.

BACKGROUND: The Centers for Medicaid and Medicare Services (CMS) and the National Cardiovascular Data Registry (NCDR) track primary percutaneous coronary intervention (PCI) performance in the form of door-to-balloon time. For quality assessment, exceptions are made for patients with "unavoidable delays" in both registries, yet it remains unclear how consistently such patients are identified. METHODS AND
RESULTS: All primary PCI patients at 3 Massachusetts hospitals (Brigham and Women's, Massachusetts General, and North Shore Medical Center) from 2009 to 2011 were evaluated for CMS inclusion/exclusion and NCDR nonsystems delay (NSD) status. We subsequently analyzed patient characteristics and outcomes based on these strata. Among 456 total patients, 128 (28%) were excluded from CMS reporting, whereas 56 (12%) were listed in the NCDR registry as having an NSD. Forty of 56 (71%) patients with NSD were also excluded from CMS reporting, whereas 312 of 400 (78%) patients reported without NSD were included in CMS reports. Between-registry agreement on patients with unavoidable delays was modest (κ=0.32). Among CMS-included patients without NSD, 94% received PCI within 90 minutes compared with 29% of CMS-excluded patients with NSD (P<0.001). Likewise, CMS-included patients without NSD had a 4-fold better 1-year mortality rate compared with CMS-excluded patients with NSD (P<0.001).
CONCLUSIONS: More than twice as many primary PCI patients are excluded from CMS quality analyses compared with NCDR. With the use of currently available cardiovascular quality registries, it is unclear how many patients truly require unavoidable delays during primary PCI. Patients with NSD had the worst outcomes regardless of CMS status.

Introduction

Primary percutaneous coronary intervention (PCI) is endorsed as initial therapy for ST‐segment elevation myocardial infarction (STEMI) provided it can be delivered in a timely manner[1]. National quality improvement efforts for the treatment of STEMI have focused on improving primary PCI systems of care using timeliness of reperfusion, often referred to as “door‐to‐balloon” time, as the foremost performance metric.[2-3] Hospital median door‐to‐balloon times are currently tied to reimbursement rates from Medicare using data tracked by the Centers for Medicaid and Medicare Services (CMS).[4] However, in an effort to make such tracking equitable across institutions, CMS has chosen to exclude data from certain patients with “unavoidable” delays before primary PCI.[5]

Recent data suggests that patients excluded from CMS reporting may account for greater than a quarter of all patients who receive primary PCI and that their outcomes are significantly worse than patients included in the CMS registry.[5] However, CMS is not the only organization that tracks primary PCI performance. The National Cardiovascular Data Registry (NCDR) is a national quality improvement registry sponsored by the American College of Cardiology.[6] Since 2009, the NCDR has tracked “non‐systems reasons for delay in PCI” (NSD) for patients undergoing primary PCI for STEMI. This element broadly mimics the exclusion criteria put forth by CMS and has been an active area of exploration since inception. Recent data suggest these patients also have far worse outcomes than patients who do not have NSD, similar to patients excluded from the CMS registry.[7]

Though both CMS and NCDR consider the importance of patients with unavoidable delays, the mechanisms and criteria for their assessment differ. Nevertheless, it is reasonable to consider patients with unavoidable delays (or who otherwise should be appropriately excluded from quality measurement assessment) as a relatively fixed cohort per institution. We, therefore, sought to understand how patients designated to have NSD in NCDR corresponded to patients excluded from the CMS registry, as this comparison helps inform our understanding of how such designations are used and whether patients with unavoidable delays are uniformly designated for the purposes of quality measures.

Methods

The Partners Long Term Outcomes Database is an ongoing institutionally sponsored registry of all patients receiving PCI at capable Partners institutions (Brigham and Women's Hospital and Massachusetts General Hospital, Boston, MA, and North Shore Medical Center, Salem, MA) since 2003. This registry captures standard NCDR data elements including NSD in PCI and is linked to the National Death Index for long‐term vital status. We subsequently linked this registry to corresponding CMS reports filed by the same institutions using medical record numbers and dates. Details regarding this linkage have been published previously.[5] Briefly, we obtained the CMS files submitted from source institutions from January 1, 2005, to September 30, 2011. The CMS reporting data from each institution were collected from the University Health Consortium, which served as the vendor for reporting to CMS for all study institutions.

For our analysis, we evaluated all patients who underwent primary PCI during the study period. Patients who were not eligible for primary PCI on the basis of thrombolytic administration were not included in the analysis. Likewise, we excluded inpatients and interfacility transfers because their PCI pathways are inherently different.

CMS exclusion status, as recorded by UHC, is based on published CMS algorithms as adjudicated by dedicated nonclinical staff.[8] Primary PCI patients met exclusion status if they were not formally interpreted to have ST‐segment elevation on the ECG closest to hospital arrival; if they had a length of stay >120 days; if they were enrolled in clinical trials; and if they were “described as nonprimary by a physician/advanced practice nurse/physician assistant” or “did not receive PCI within 90 minutes and had a reason for delay documented by a physician/APN (advanced practice nurse)/PA (physician's assistant) (eg, social, religious, initial concern or refusal, cardiopulmonary arrest, balloon pump insertion, respiratory failure requiring intubation).”[8] Consistent with NCDR definitions and current practice, treating interventional cardiology staff identify patients as having NSD if any of the following conditions were deemed applicable: (1) difficult vascular access, (2) cardiac arrest and/or need for intubation, (3) patient delays in providing consent, (4) difficult lesion crossing, and (5) “other.” Similar to CMS reporting status, the NSD designation in NCDR does not require a specified primary PCI duration. By definition, NSD does suggest that the primary PCI took longer than it otherwise would have, but this duration may still be <90 minutes in total.

Using the linked data sources, we calculated the percentage of cases excluded from CMS relative to those reported to NCDR to have NSD, from January 2009 through September 2011. For the purposes of comparison, we used CMS status as the reference standard since it is the more established measure and has current application through Medicare.

We then divided the study population into 4 nonoverlapping groups based on CMS status and the NSD element. Patients were designated as CMS‐included and without NSD, CMS‐included with NSD, CMS‐excluded without NSD, or CMS‐excluded with NSD. We performed unadjusted comparisons of demographic, clinical, and procedural characteristics among these 4 groups. To compare outcomes among these groups, we constructed Kaplan–Meier survival curves for all‐cause mortality using data obtained from the National Death Index and assessed differences using the log‐rank test. The follow‐up period began on the date of primary PCI and ended on the date of death or September 30, 2011, whichever came first. All models were unadjusted as we intended to study differences between groups as they were designated.

Dichotomous data are presented as percentages. Continuous variables are presented as means (SDs) or medians (25th and 75th interquartile ranges) for parametric and nonparametric data, respectively. The χ2 test, Kruskal–Wallis test, and 1‐way ANOVA were used for comparisons of categorical and continuous variables as appropriate. Fisher's exact test was used where frequencies were expected to be small. All statistical analyses were performed with SAS version 9.3 (SAS Institute). Study approval was obtained through the Partners‐wide institutional review board.

Results

During the study period, 456 patients underwent the primary PCI process for a STEMI. Among those patients, 128 (28%) were excluded from CMS reporting, whereas 56 (12%) were listed as having an NSD in the NCDR registry. As shown in Table 1, among the 56 patients identified with NSD, 40 (71%) were also excluded from CMS reporting. Conversely, among the 400 patients reported without NSD in NCDR, 312 (78%) were included in CMS reports. Thus, the overall concordance between presence or absence of unavoidable delays between NCDR and CMS was 77% (352 of 456 patients), with the majority of the discordant cases being excluded from CMS reports but not identified as having NSD in NCDR. However, simple concordance will overestimate agreement in the setting of unusual occurrences, and agreement as measured by κ was only 0.32 (where −1 is perfect disagreement and 1 is perfect agreement). Using the CMS designation as the reference standard, the more recently developed NSD element had a sensitivity of 95% but a specificity of 31%.

Table 1.

Comparison of CMS Inclusion Status Versus NCDR's Nonsystems Reasons for Delay

	CMS Excluded	CMS Included	Total
Nonsystems reason for delay, %	40 (8.8)	16 (3.5)	56 (12.3)
Difficult vascular access	6	0
Cardiac arrest/intubation	8	7
Delay in procedural consent	1	0
Difficult lesion crossing	8	1
Other	17	8
No nonsystems reason for delay, %	88 (19.3)	312 (68.4)	400 (87.7)
Total, %	128 (28.1)	328 (71.9)	456 (100)

CMS indicates Centers for Medicaid and Medicare Services.

Generally, patients reported to NCDR with NSD tended to be sicker with significantly greater rates of cerebrovascular disease, peripheral vascular disease, and shock on admission and the highest rates of expected in‐hospital mortality (Table 2). Accordingly, these patients also had significantly greater intra‐aortic balloon pump utilization rates (Table 3) and the highest mortality rates (Figure 1). Likewise, patients with NSD were much less likely to receive reperfusion within the targeted 90‐minute window (Figure 2). These performance and outcome differences were especially true of patients with NSD who were also excluded from CMS reporting.

Table 2.

Baseline Characteristics

	Total	CMS Included, No NSD	CMS Excluded, No NSD	CMS Included, +NSD	CMS Excluded, +NSD	P Value
	n=456	n=312	n=88	n=16	n=40
Age (SD), y	62.0±12.8	61.6±13.0	63.4±13.2	56.7±8.8	63.8±11.4	0.18
Male, n (%)	331 (73)	226 (72)	64 (73)	12 (75)	29 (73)	0.99
White race, n (%)	385 (84)	265 (85)	78 (89)	11 (69)	31 (78)	0.28
Insurance payer						0.02
Government	199 (44)	126 (40)	42 (48)	6 (38)	25 (63)
Commercial	228 (50)	165 (53)	42 (48)	8 (50)	13 (33)
None	28 (6)	21 (7)	4 (5)	2 (12)	1 (3)
Hypertension	274 (60)	178 (57)	61 (69)	8 (50)	27 (68)	0.12
Dyslipidemia	336 (74)	222 (71)	75 (85)	11 (69)	28 (70)	0.06
Family history of premature CAD	102 (22)	74 (24)	21 (24)	2 (13)	5 (13)	0.31
Prior MI	85 (19)	47 (15)	21 (24)	3 (19)	14 (35)	0.01
Prior heart failure	15 (3)	7 (2)	4 (5)	1 (6)	3 (8)	0.24
Prior PCI	77 (17)	49 (16)	17 (19)	1 (6)	10 (25)	0.28
Prior CABG	19 (4)	8 (3)	9 (10)	0 (0)	2 (5)	0.01
Cerebrovascular disease	41 (9)	26 (8)	5 (6)	0 (0)	10 (25)	<0.01
Peripheral arterial disease	29 (6)	14 (4)	5 (6)	0 (0)	10 (25)	<0.01
Chronic lung disease	36 (8)	25 (8)	3 (3)	2 (13)	6 (15)	0.13
Diabetes mellitus	84 (18)	54 (17)	15 (17)	3 (19)	12 (30)	0.27
Anginal classification						0.02
No symptoms, no angina	72 (16)	37 (12)	15 (17)	7 (44)	13 (33)
Canadian class angina ≥3	378 (83)	271 (87)	72 (82)	9 (57)	26 (66)
Cardio shock at admission	30 (7)	17 (5)	3 (3)	2 (13)	8 (20)	<0.01
Estimated glomerular filtration rate, mean±SD, mL/min	76±22	78±22	74±20	79±20	69±25	0.06
Predicted mortality rate, mean±SD, %*	5.0±0.10	4.3±0.08	4.2±0.08	5.5±0.12	12.0±0.19	<0.01

CMS indicates Centers for Medicaid and Medicare Services; CAD, coronary artery disease; MI, myocardial infarction; NDS, nonsystems delay; PCI, percutaneous coronary intervention; CABG, coronary artery bypass graft surgery.

Predicted mortality was derived from previously published models.[12]

Table 3.

Primary PCI Procedural Characteristics

	Total	CMS Included, No NSD	CMS Excluded, No NSD	CMS Included, +NSD	CMS Excluded, +NSD	P Value
	n=456	n=312	n=88	n=16	n=40
Arterial access site						<0.01
Femoral	410 (89)	286 (92)	81 (92)	13 (81)	30 (75)
Radial	38 (8)	23 (7)	5 (6)	3 (19)	7 (18)
Number of lesions	1.21±0.5	1.17±0.4	1.28±0.6	1.31±0.5	1.30±0.7	0.09
Preprocedural TIMI flow, n (%)						0.06
<3	393 (86)	267 (86)	72 (82)	15 (94)	37 (92)
3	64 (14)	44 (14)	16 (18)	1 (6)	3 (8)
High lesion complexity, n (%)	209 (46)	140 (45)	41 (47)	7 (44)	21 (53)	0.83
Inta‐aortic balloon pump use, n (%)	42 (9)	21 (7)	3 (3)	5 (31)	13 (33)	<0.01
Vessel stented, n (%)
Right coronary	213 (47)	151 (48)	42 (48)	6 (38)	14 (35)	0.37
Left main coronary	3 (1)	2 (1)	1 (1)	0 (0)	0 (0)	0.88
Left anterior descending	180 (39)	123 (39)	26 (30)	9 (56)	22 (55)	0.03
Left circumflex	76 (17)	45 (14)	21 (23.86)	3 (19)	7 (18)	0.21
Stents deployed (SD), n	1.28±0.8	1.26±0.8	1.41±0.8	1.44±1.3	1.08±0.8	0.12
Drug‐eluting stents used, n (%)	173 (42)	112 (39)	46 (54)	5 (33)	10 (31)	0.05
Minimum stent diameter (SD), mm	2.88±0.5	2.89±0.5	2.87±0.4	2.98±0.5	2.79±0.6	0.52
Median total stent length (IQR), mm	23 (18, 31)	23 (18, 30)	24 (18, 32)	18 (15, 26)	23 (18, 30)	0.50
Procedural medications used, n (%)
Bivalirudin	110 (24)	68 (22)	24 (27)	10 (63)	8 (20)	<0.01
Glycoprotein 2b/3a inhibitors	281 (62)	210 (67)	45 (51)	4 (25)	22 (55)	<0.01

CMS indicates Centers for Medicaid and Medicare Services; IQR, interquartile range; NDS, nonsystems delay; TMI, Thrombosis in Myocardial Infarction.

Figure 1.

Kaplan–Meier survival plots for primary PCI patients stratified by their CMS inclusion/exclusion and NCDR nonsystems delays (NSD) designations. CMS indicates Centers for Medicaid and Medicare Services; NCDR, National Cardiovascular Data Registry; PCI, percutaneous coronary intervention.

Figure 2.

Door‐to‐balloon times and 1‐year mortality stratified by their CMS inclusion/exclusion and NCDR nonsystems delays (NSD) designations. CMS indicates Centers for Medicaid and Medicare Services; NCDR, National Cardiovascular Data Registry.

Conversely, patients excluded from CMS reporting who were not reported to have NSD had the numerically lowest rates of shock on presentation, the lowest expected in‐hospital mortality rates, and the lowest intra‐aortic balloon pump use. Additionally, their mortality rates were consistent with patients included in CMS reporting without NSD.

Focusing solely on patients excluded from CMS reports, there were significant differences in the rationale for CMS exclusion between those with NSD and those without NSD (Table 4). Patients without NSD were more likely to be excluded from CMS due to a diagnostic dilemma on presentation than were those reported to have an NSD (42% vs 7.5%). Conversely, procedural difficulties as a rationale for CMS exclusion were more frequent among those with NSD (42.5% vs 7%).

Table 4.

Rationale for CMS Exclusion by NCDR Status

Rationale for CMS Exclusion (%)	No NSD	+NSD	P Value
	n=88	n=40
Diagnostic dilemma	37 (42)	3 (7.5)	<0.001
Critical illness	15 (17)	10 (25)
Concomitant illness	4 (5)	8 (20)
Procedural issues	6 (7)	17 (42.5)
Coding issues*	6 (7)	1 (2.5)
Unclear/trial enrollment	20 (23)	1 (2.5)

CMS indicates Centers for Medicaid and Medicare Services; NCDR, National Cardiovascular Data Registry; NDS, nonsystems delay.

Patients who were excluded from CMS reporting due to errors in coding identified upon subsequent adjudication.

Conclusions

These data suggest that the CMS and NCDR registries demonstrate wide heterogeneity in the number of patients considered to require unavoidable delays before primary PCI. Specifically, twice as many patients were excluded from CMS quality reports as were designated as having NSD in NCDR. Though most NSD patients are also excluded from CMS reports, the majority of patients excluded by CMS algorithms were not designated as having NSD. Furthermore, there appear to be systematic differences among CMS‐excluded patients based on their NSD status. Specifically, those CMS‐excluded patients also reported to have NSD had the highest inpatient and long‐term mortality rates and tended to be excluded from CMS reports for different reasons.

Quality improvement in primary PCI care has been an area of great interest for many years. Though CMS uses data collected following primary PCI for national evaluation and hospital reimbursement levels, they do not actively record patients who might qualify for evaluation but are ultimately excluded from their quality analyses. Inclusion and exclusion from CMS reporting have recently taken on greater significance with the advent of value‐based purchasing, which uses performance data on CMS‐included patients to determine Medicare reimbursement levels. Since it is generally not feasible to collect data on patients excluded from CMS analyses, it may be desirable in certain circumstances to use NCDR's NSD cohort as a surrogate population. However, the degree of discordance between the CMS excluded population and NCDR's NSD population suggests caution in considering the 2 to be interchangeable.

Interestingly, many fewer patients were self‐reported to NCDR as having NSD compared with the number of patients adjudicated as meeting CMS exclusion criteria. CMS reporting on door‐to‐balloon time has previously been criticized for being easily manipulated despite adjudication.[9-10] The NSD element in NCDR, on the other hand, is a self‐reported measure determined by the interventional team with less restrictive designation criteria. Yet, despite self‐reporting and the latitude to feasibly abuse the NSD designation, its application was much less frequent. This finding may be related to the exclusion of primary PCI patients from CMS reports by clinical adjudicators following rigid guidelines, as previously described,[5,11] or the ramifications of suboptimal performance in the data reported to CMS, which in turn may incentivize a more aggressive approach to identifying patients who meet criteria per CMS guidelines. However, this finding may also be related to unfamiliarity with a relatively new metric in NCDR or missing details regarding potential difficulties in the emergency department among the interventional cardiology staff who ultimately determine the NCDR designation. Whatever the reason, almost a quarter of the total patients in our study population not considered to have NSD were ultimately excluded from CMS quality measures reporting. This might be interpreted to suggest that CMS exclusion criteria are excessively used, that the NCDR element is insensitive as applied, or some combination of the 2. While it is feasible that both registries are intended to capture different populations and have done just that, it is reasonable to assume that patients with unavoidable delays during primary PCI are a generally discrete and identifiable population.

Our study should be considered in terms of its limitations. While it is currently impossible to track patients excluded from CMS reports nationally, our data come from 3 regionally specific hospitals and, thus, our findings may not be broadly applicable. Additionally, the total number of patients analyzed may not have afforded us the power to observe all significant between‐group differences. Finally, patients' vital status during the observation period was obtained from the Massachusetts' Vital Status Registry, and it is possible that events among patients who migrated out of state during the follow‐up period were not captured.

Ongoing quality improvement efforts in primary PCI through both NCDR and CMS have recognized that unavoidable delays or delays not related to an institutions' systems of care require consideration to avoid unfair cross‐hospital comparisons. Though the CMS and NCDR measures are designed to be broadly consistent, there appears to be significant heterogeneity in the number of patients considered to meet criteria. Using currently available cardiovascular quality registries, it is unclear how many patients truly require unavoidable delays during primary PCI. There may be a temptation to use the available NSD patients as representative of patients excluded (and thus unavailable) by CMS quality reports, but our data do not support considering the 2 populations interchangeable. Further study into the relationship between primary PCI patients excluded from CMS reports and deemed to have NSD in NCDR is warranted.