Failure to Rescue as an Outcome Metric for Pediatric and Congenital Cardiac Catheterization Laboratory Programs: Analysis of Data From the IMPACT Registry.

Background Risk-adjusted adverse event (AE) rates have been used to measure the quality of pediatric and congenital cardiac catheterization laboratories. In other settings, failure to rescue (FTR) has demonstrated utility as a quality metric. Methods and Results A multicenter retrospective cohort study was performed using data from the IMPACT (Improving Adult and Congenital Treatment) Registry between January 2010 and December 2016. A modified FTR metric was developed for pediatric and congenital cardiac catheterization laboratories and then compared with pooled AEs. The associations between patient- and hospital-level factors and outcomes were evaluated using hierarchical logistic regression models. Hospital risk standardized ratios were then calculated. Rankings of risk standardized ratios for each outcome were compared to determine whether AEs and FTR identified the same high- and low-performing centers. During the study period, 77 580 catheterizations were performed at 91 hospitals. Higher annual hospital catheterization volume was associated with lower odds of FTR (odds ratio: 0.68 per 300 cases; P=0.0003). No association was seen between catheterization volume and odds of AEs. Odds of AEs were instead associated with patient- and procedure-level factors. There was no correlation between risk standardized ratio ranks for FTR and pooled AEs (P=0.46). Hospital ranks by catheterization volume and FTR were associated (r=-0.28, P=0.01) with the largest volume hospitals having the lowest risk of FTR. Conclusions In contrast to AEs, FTR was not strongly associated with patient- and procedure-level factors and was significantly associated with pediatric and congenital cardiac catheterization laboratory volume. Hospital rankings based on FTR and AEs were not significantly correlated. We conclude that FTR is a complementary measure of catheterization laboratory quality and should be included in future research and quality-improvement projects.

Clinical Perspective

What Is New?

A failure to rescue (FTR) metric was developed specifically for the pediatric and congenital catheterization laboratory setting and compared with pooled adverse events (AEs).

Risk of pooled AEs was related to participant and procedural factors, whereas FTR risk was associated with hospital volume.

Hospital ranks using risk‐adjusted AEs and FTR differed significantly.

What Are the Clinical Implications?

FTR as a quality metric adds unique information about programmatic quality.

FTR should be included in future research studies, quality‐improvement efforts, and evaluations of programmatic performance.

Cardiac catheterization remains the gold standard for hemodynamic evaluation of young patients with cardiac disease and as a means of intervening for an increasing range of conditions. Adverse events (AEs) in the pediatric and congenital cardiac catheterization laboratory (PCCL) are relatively uncommon. Nonetheless, PCCL procedures are a source of significant morbidity.1, 2 Previous efforts to define quality metrics have pooled different AEs.3, 4, 5, 6, 7, 8 Pooled AEs were chosen over mortality because deaths following catheterization are rare, and concerns exist regarding (1) attributability of deaths to the catheterization procedure, (2) significant medical comorbidity, and (3) frequent crossover to and from the care of cardiac surgeons.

An important question is whether the risk‐adjusted AE rate is the single, optimal quality measure for PCCL programs. Failure to rescue (FTR), defined as death following a major AE, was introduced as an alternative to the pooled AE outcome metric.9 In diverse settings, FTR has demonstrated attractive characteristics relative to both pooled AE and mortality rates. In these settings, significant associations have been demonstrated between FTR and known measures of hospital quality, whereas the factors associated with pooled AEs have been patient‐level factors.9, 10, 11 In addition, ranks of hospitals based on risk‐adjusted AEs differ significantly from those based on FTR.11 These findings suggest that FTR has value in measuring the quality of care delivered at different hospitals. Although initially developed in a cohort of adult surgical patients,9 FTR has been applied in a broad array of medical and surgical contexts.9, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23

Using data from the IMPACT (Improving Adult and Congenital Treatment) Registry, we performed a multicenter retrospective cohort study to develop an FTR metric adapted to the PCCL environment and to determine whether that metric provided value in addition to that provided by risk‐adjusted AEs. We sought to determine what patient‐, procedure‐, and hospital‐level factors were associated with risk of complications and FTR and whether hospital rankings based on FTR and pooled AEs were correlated. We hypothesized (1) that patient and procedure factors would be associated with the odds of pooled AEs but not FTR and that, conversely, potential markers of programmatic quality (specifically, hospital procedural volume) would be associated with the odds of FTR but not pooled AEs and (2) that rankings of hospitals by FTR and AEs would not be significantly correlated. Together, these findings would show that FTR provided unique information in addition to that provided by risk‐adjusted AEs.

Methods

Data Source

IMPACT is a clinical registry funded by the American College of Cardiology and managed by the National Cardiovascular Data Registry with data from 91 pediatric and general hospitals performing cardiac catheterizations in children and adults with congenital heart disease at the time of this analysis. Participating centers collect demographics, medical and surgical history, procedural information, and AEs through hospital discharge on all patients undergoing cardiac catheterization. Data are recorded using standardized data elements and definitions. The database is subject to quality assurance standards.24 The current study used data from IMPACT v1.0.1. The institutional review board of the Children's Hospital of Philadelphia reviewed the proposed project and determined that it did not represent human subjects research in accordance with the Common Rule (45 CFR 46.102(f)). Data, analytic methods, and study materials will not be made available to other researchers for the purposes of reproducing the results or replicating the procedures because that would not be consistent with the data‐use agreement between the study staff and the American College of Cardiology Foundation and National Cardiovascular Data Registry.

Study Population

Cases performed between January 2010 and December 2016 at centers contributing to IMPACT in patients between birth and 25 years of age were eligible for inclusion. Cases reported as emergent or salvage status were excluded because the patients were already in jeopardy; therefore, these cases would not be informative about the quality of care in the catheterization laboratory. For the same reasons, cases involving patients who were already receiving mechanical circulatory (extracorporeal membrane oxygenation or ventricular assist devices) were also excluded. Although patients with recent cardiac surgery were potentially at higher risk of adverse outcomes, their cases represent a common and important part of clinical practice and so were not excluded unless urgency of the procedure was classified as emergency or salvage. As noted below, statistical adjustment for history of recent cardiac surgery was performed to mitigate any bias introduced by differential utilization of catheterization in the postoperative period between hospitals.

Study Data

Data were extracted directly from the IMPACT Registry, specifically demographics, medical history (including preprocedure risk factors), hemodynamic data, and details of the procedure (specific interventions, anesthesia at start of the case, device used, and presence of a trainee). AEs recorded in IMPACT (in‐hospital mortality; cardiac arrest; new arrhythmia; new heart valve regurgitation; tamponade; air embolus; embolic stroke; device malposition; device embolization; airway event; blood transfusion; initiation of dialysis; new extracorporeal membrane oxygenation; new ventricular assist device; unplanned cardiac, vascular, or other surgery; vascular complication requiring treatment; repeat catheterization [all before discharge], and other events) were recorded. Of note, the registry instructs centers to report only unplanned surgical and catheterization procedures that are attributable to a catheterization complication. Hospital procedural volume was calculated as prorated annual averages, as described previously.25, 26, 27

Definitions

FTR was originally defined as death following a complication, reflecting the hospital's inability to prevent that progression.9 We chose to adapt this definition to the PCCL environment with 2 changes. First, we counted all deaths occurring within 2 days from catheterization. In previous FTR studies of surgical patients, it has been assumed that all deaths within 30 days were attributable to preceding procedure‐related complications.9, 12 In the PCCL, patients may undergo technically successful catheterizations without complication and then subsequently die during the same hospitalization either after cardiac surgery or because of progression of their condition. In a multicenter series, only 10% of deaths within 30 days of PCCL were attributable (after adjudication) to the catheterization.28 We reviewed the incident and cumulative rate of deaths after catheterization and found that the 2‐day mark was a natural qualitative cut point in the incidence of postcatheterization deaths. This approach was also consistent with previous research.25, 26 These choices were made to minimize inclusion of deaths due to other causes and were necessary because it is not possible to review and adjudicate all deaths and other events in IMPACT. Deaths and other events that occurred in the same hospitalization but after an elective surgical procedure were not included regardless of timing. This definition of deaths was used for both FTR and pooled AEs. Second, because we intended to measure the quality of catheterization laboratory care, we included other catastrophic AEs for which care escalated beyond the capacity of the catheterization laboratory team in our list of failures (Table 1). We acknowledge that in some cases, mechanical circulatory support was initiated electively or semielectively in catheterization patients. For our analysis, these failure events were also considered separately as catastrophic AE.

Table 1

Outcomes

Proximal events

New arrhythmia

New heart valve regurgitation

Cardiac tamponade

Air embolus

Embolic stroke

Device malposition

Device embolization

Airway event

Initiation of dialysis

Unplanned cardiac, vascular, or other surgery (due to catheterization complication)a

Vascular complication requiring treatment (due to catheterization complication)a

Repeated catheterization (due to catheterization complication)a

Other major events

Catastrophic events

Death within 2 days of catheterization

Cardiac arrest

Initiation of mechanical circulatory support

Initiation of extracorporeal membrane oxygenation

Unplanned cardiac, vascular, or other surgery (due to catheterization complication)a

Vascular complication requiring treatment (due to catheterization complication)a

Repeat catheterization (due to catheterization complication)a

Outcomes were determined using IMPACT (Improving Adult and Congenital Treatment) Registry standard definitions. Failure to rescue was defined as a catastrophic adverse outcome occurring in a case with a proximal event. Pooled adverse events encompass any of the proximal or catastrophic events.

These events are defined as being due to a catheterization complication, so they were included in both proximal and catastrophic events.

Next we evaluated the possible AEs after catheterization and created a list of those that could potentially lead to a catastrophic adverse outcome (Table 1). Blood transfusion was the most common AE noted in the registry, but it was not included as a proximal event because it was judged not likely to be directly and independently responsible for a catastrophic event. This choice was supported by the observation that removing it from the list of preceding events did not change the precedence rate (see below). Of note, repeated catheterization and surgeries are recorded in IMPACT only when the procedure is attributable to a catheterization complication. Consequently, even if no other AE were recorded, cases in which unplanned surgeries (cardiac, vascular, or other) or repeated catheterizations were reported were recorded as having both proximal and catastrophic AEs. The proportion of these unplanned procedures with proximal events is reported in Table S1.

To evaluate whether the list of proximal events was comprehensive, we calculated the proportion of catastrophic AEs with a preceding proximal event (the precedence rate). In this sample, the precedence rate was 70% (687/977). This rate is close to what has been seen in other studies using FTR.22 It may also reflect that patients arriving in the PCCL are potentially more medically complicated (with greater possibility of deterioration independent of a preceding complication) than older patients undergoing isolated surgical procedures.

To summarize, the definition of our modified FTR metric is the occurrence of one of the catastrophic AEs after one of the proximal events.

Statistical Analysis

Descriptive statistics were calculated. Observed rates for total AEs, proximal AEs, catastrophic AEs, and FTR were calculated.

The goals of subsequent analyses were to compare the characteristics of the different study outcomes as quality metrics. First, generalized linear models with binomial frequency distribution, adjusted for the listed covariates as fixed effects and clustered by hospital (using a random intercept for each hospital, implemented using glimmix in SAS v9.4; SAS Institute), were calculated to evaluate the degree to which patient, procedure, and hospital factors were associated with each outcome of interest. Additional random effects (eg, random slope) and interaction were not explored. FTR was the primary outcome of interest. Pooled AEs (including and excluding deaths), proximal AEs, and catastrophic AEs were secondary outcomes. We hypothesized that odds of pooled AEs would be associated with patient and procedural factors and that odds of FTR would not be associated with these factors. Moreover, we sought to determine whether markers of hospital quality were associated with risk of these outcomes, hypothesizing that odds of FTR would be associated with these factors and that odds of pooled AEs would not. To our knowledge, no hospital characteristics are unequivocally associated with the quality of a PCCL program. Hospital procedural volume has been associated with improved outcomes across disparate medical fields.29 Consequently, hospital annual catheterization volume was used as the primary hospital characteristic of interest.

Covariates were identified before model calculation and included patient, procedure, and hospital characteristics. Patient characteristics were age by category, sex, race, previous cardiac surgery in the prior 30 days, cardiac catheterization in the prior 30 days, preprocedure inotropes, preprocedure vasodilator, preprocedure sepsis, chronic lung disease, genetic syndrome, renal insufficiency, hemodynamic vulnerability (as defined previously7), low systemic arterial saturation, low mixed venous saturation, high mean pulmonary pressure, and elevated systemic ventricular end‐diastolic pressure. Procedure characteristics were procedural risk category as defined in the Catheterization for Congenital Heart Disease Adjustment for Risk Method,4, 6, 7 use of general anesthesia versus conscious sedation, and presence of a trainee. Hospital characteristics were procedural volume >35% adult and academic or university hospital. These separate covariates were used rather than the IMPACT risk‐adjustment model30 because that risk model combines procedures and patient ages. It was also not clear, a priori, that this component of the risk model would be equally applicable to all outcomes, and it was not desirable to forego information about both ages and procedure types. Hemodynamic vulnerabilities were considered individually rather than as a number of vulnerabilities, as had been done previously,7 to avoid bias from missing data and to evaluate whether the effect of hemodynamic vulnerabilities differed for AEs and FTR. To determine whether either deviation from the validated risk adjustment model introduced bias, sensitivity analyses were performed using the most recent procedure codes and the sum of the number of hemodynamic vulnerabilities present. No changes in the observed results were seen (data not shown). A third sensitivity analysis was performed by recalculating all models excluding patients with cardiac surgery in the prior 30 days. Again, no changes in the reported results were seen (data not shown). The threshold of >35% cases in adults was chosen based on examination of the distribution of adult cases in the hospitals contributing data to the IMPACT Registry, which identified a qualitatively different cluster of hospitals, as has been reported previously.31, 32

The primary model used FTR as an outcome. Secondary analyses planned before the beginning of the analyses included calculating analogous models for the risk of (1) all AEs, (2) catastrophic AEs, (3) all AEs excluding death, and (4) proximal AEs. It was also recognized that the database includes multiple catheterization procedures in individual subjects. To address this, a sensitivity analysis restricted to the first catheterization of each participant was performed for our primary model, as in previous studies.25, 26 To ensure that the associations between hospital factors and outcomes were not due to correlation between these factors, additional models were calculated with only 1 of the 3 center factors for each outcome, confirming that no change in the observed associations was seen.

Next we sought to evaluate whether hospital rankings based on case‐mix–adjusted AEs and FTR identified the same high‐ and low‐performing hospitals. For each hospital, a risk standardized ratio (RSR) was calculated, and hospitals were ranked. RSR is a ratio of the predicted outcome for each hospital to the expected rate for a hospital with similar case mix based on previously calculated hierarchical models. Correlation between rankings of the RSR for FTR, composite AEs, and catastrophic AEs were evaluated using Spearman rank correlation coefficients, as has been done in previous studies using FTR.10, 22 Similarly, the association between the aforementioned outcomes and hospital PCCL volume was also evaluated.

Not all hemodynamic factors were recorded in every case. To avoid the potential bias introduced by excluding cases with missing hemodynamics, a variable for missing was included, as described previously.27, 33 Multiple imputation was not used because the missing data were not plausibly predicted by collected data. As described, the primary analysis and secondary analyses (exposure, outcome, and covariates) were defined before analysis. To avoid bias, no post hoc model refinement was performed. No formal steps were taken to penalize for multiple comparisons, and analyses beyond the primary analyses should be considered exploratory. A threshold for statistical significance was set at P<0.05. All data analysis was performed using SAS 9.4.

Results

Over the study period, 77 580 procedures were performed on 53 056 individual participants at 91 hospitals (Figure 1). Of the cases included, the median age of participants was 4 years (interquartile range: 0–12 years). Overall, 47% were female, and 70% were white. Patient and procedural characteristics are summarized in Table 2. Of the hospitals contributing data to IMPACT, 46% (42/91) were university or teaching hospitals, and 10% (9/91) had a large proportion of their reported catheterizations in adults (>35%). The median annual catheterization volume of hospitals included in the study was 284 cases (interquartile range: 117–416 cases).

Figure 1

Study population. ECMO indicates extracorporeal membrane oxygenation; VAD, ventricular assist device.

Table 2

Study Population

Characteristic	Result
No. of cases	77 580
No. of participants	53 056
No. of hospitals	91
Age, y, median (IQR)	4.0 (0–12)
Age group
<30 d	5 (4010)
>30 d to 1 y	21 (16 392)
1–8 y	38 (29 104)
8–18 y	28 (21 894)
>18 y	8 (6180)
Female sex	47 (36 108)
Race
White	70 (54 648)
Black	17 (13 426)
Other	12 (9506)
Cardiothoracic surgery within 30 d prior	2 (1568)
Cardiac catheterization within 30 d prior	4 (3112)
Chronic lung disease	6.5 (5009)
Single ventricle	21 (16 018)
Genetic syndrome	12 (8912)
Renal insufficiency	2.8(2166)
Preprocedure inotropes	6.8 (5273)
Preprocedure vasodilator	9.6 (7456)
Hemodynamic vulnerability
Systemic arterial saturation	27 (19 542/71 500)
Mixed venous saturation	13 (8981/68 853)
Pulmonary artery pressure	18 (11 184/62 658)
Systemic ventricular end‐diastolic pressure	6 (3158/50 575)
Procedure risk category
1	41 (31 966)
2	33 (25 586)
3	19 (14 625)
4	7 (5381)
Sedation strategy
General anesthesia	87 (67 564)
Conscious sedation	12 (9144)
Trainee present	63 (48 743)
Case performed at hospital with >35% adult catheterization volume	2 (1300)
Case performed at a teaching institution	93 (71 594)

Data are shown as % (n) except as noted. IQR indicates interquartile range.

Outcomes

AEs occurred in 4.7% of cases overall (Table 3). The risk of the composite outcome for catastrophic AEs was 1.2%. The risk of proximal AEs was 4.4% (frequencies of individual proximal AEs are summarized in Table S2). The risk of catastrophic AEs after proximal AEs (observed FTR) was 20.3%.

Table 3

Outcomes

Outcome	Result, % (n)
Any AE	4.7 (3682)
Catastrophic adverse outcome	1.2 (960)
Death within 2 d	0.2 (141)
Cardiac arrest	0.6 (465)
Initiation of ECMO/VAD	0.2 (138)
Unplanned cardiac operation due to catheterization complication	0.3 (232)
Unplanned vascular operation due to catheterization complication	0.1 (41)
Unplanned other operation due to catheterization complication	0.2 (153)
Unplanned catheterization due to catheterization complication	0.3 (252)
Proximal adverse outcomes	4.4 (3416)
FTR	20.3 (694/3416)

AE indicates adverse event; ECMO, extracorporeal membrane oxygenation; FTR, failure to rescue; VAD, ventricular assist device.

Factors Associated With FTR, AE, and Catastrophic AE Rates

Hierarchical models were calculated for each major outcome (Table 4). These models demonstrated that the adjusted risk of FTR was significantly lower at higher volume hospitals (odds ratio [OR] per 300 cases: 0.68; 95% CI, 0.55–0.84; P=0.0003). The risk of FTR was lower at hospitals with a high proportion of cases in adults (OR: 0.16; 95% CI, 0.05–0.55; P=0.003). Preprocedural receipt of inotropes (OR: 3.05; 95% CI, 2.43–3.83; P<0.0001), renal insufficiency (OR: 2.42; 95% CI, 1.54–3.79; P=0.0001), and low mixed venous saturation (OR: 1.39; 95% CI, 1.09–1.79; P=0.009) were positively associated with FTR, whereas surgery in the prior 30 days (OR: 0.16; 95% CI, 0.1–0.25; P<0.0001) was negatively associated with FTR. Procedure risk categories 2 and 3 were negatively associated with FTR relative to risk category 1 (OR: 0.70; 95% CI, 0.51–0.95; P=0.02; and OR: 0.72; 95% CI, 0.52–0.99; P=0.04, respectively).

Table 4

Multivariable Models for Outcomes of Interest

	FTR		All AEs		Catastrophic AEs
	OR (95% CI)	P Value	OR (95% CI)	P Value	OR (95% CI)	P Value
Age group
<30 d	0.82 (0.56–1.18)	0.29	2.62 (2.27–3.03)	<0.0001	2.25 (1.71–2.96)	<0.0001
30 d to 1 y	1.21 (0.90–1.62)	0.21	1.55 (1.39–1.73)	<0.0001	1.92 (1.55–2.39)	<0.0001
1–8 y	1 (NA)	NA	1 (NA)	NA	1 (NA)	NA
8–18 y	1.29 (0.97–1.73)	0.08	0.99 (0.89–1.10)	0.86	1.08 (0.87–1.34)	0.51
≥18 y	1.28 (0.80–2.06)	0.31	0.91 (0.76–1.09)	0.31	1.04 (0.73–1.48)	0.82
Male sex	0.89 (0.74–1.07)	0.20	1.05 (0.98–1.13)	0.14	0.96 (0.84–1.09)	0.52
Race
White	1 (NA)	NA	1 (NA)	NA	1 (NA)	NA
Black	1.04 (0.81–1.35)	0.75	1.02 (0.98–1.13)	0.14	1.06 (0.88–1.26)	0.56
Other	1.21 (0.9–1.62)	0.22	1.00 (0.89–1.12)	0.95	1.21 (0.98–1.49)	0.08
Surgery (previous 30 d)	0.16 (0.1–0.25)	<0.0001	10.15 (8.80–11.69)	<0.0001	1.05 (0.70–1.56)	0.82
Catheterization (previous 30 d)	1.24 (0.71–2.15)	0.45	0.65 (0.53–0.80)	<0.0001	1.00 (0.69–1.46)	0.98
Preprocedure medications
Inotrope	3.05 (2.43–3.83)	<0.0001	2.37 (2.14–2.62)	<0.0001	4.94 (4.20–5.82)	<0.0001
Vasodilators	0.96 (0.72–1.27)	0.77	1.02 (0.91–1.14)	0.79	1.08 (0.89–1.31)	0.45
Preprocedure conditions
Sepsis	2.19 (0.94–5.07)	0.07	1.16 (0.78–1.72)	0.47	1.62 (0.94–2.79)	0.08
Chronic lung disease	0.63 (0.43–0.90)	0.01	0.98 (0.86–1.12)	0.79	0.8 (0.61–1.03)	0.09
Genetic syndrome	0.76 (0.58–0.99)	0.05	1.22 (1.11–1.35)	<0.0001	1.01 (0.83–1.23)	0.91
Renal insufficiency	2.42 (1.54–3.79)	0.0001	1.28 (1.05–1.58)	0.02	1.98 (1.47–2.68)	<0.0001
Systemic arterial saturation
Low vs normal	0.95 (0.77–1.19)	0.68	1.14 (1.05–1.24)	0.003	1.08 (0.92–1.27)	0.36
Low vs missing	1.17 (0.8–1.7)	0.42	0.97 (0.83–1.13)	0.69	1.07 (0.82–1.40)	0.63
Mixed venous saturation
Low vs normal	1.39 (1.09–1.79)	0.009	1.53 (1.38–1.69)	<0.0001	1.85 (1.54–2.21)	<0.0001
Low vs missing	1.22 (0.88–1.70)	0.22	1.20 (1.05–1.38)	0.006	1.39 (1.10–1.77)	0.007
Pulmonary artery pressure
High vs normal	1.00 (0.78–1.29)	1.00	1.41 (1.28–1.56)	<0.0001	1.56 (1.30–1.86)	<0.0001
High vs missing	1.11 (0.85–1.43)	0.44	1.35 (1.22–1.49)	<0.0001	1.53 (1.27–1.84)	<0.0001
Systemic ventricular end‐diastolic pressure
High vs normal	0.86 (0.57–1.29)	0.46	1.45 (1.23–1.69)	<0.0001	1.43 (1.09–1.88)	0.009
High vs missing	1.15 (0.93–1.43)	0.21	1.03 (0.94–1.11)	0.56	1.13 (0.97–1.32)	0.13
Procedure group
1	1 (NA)	NA	1 (NA)	NA	1 (NA)	NA
2	0.70 (0.51–0.95)	0.02	1.54 (1.37–1.73)	<0.0001	1.03 (0.81–1.30)	0.81
3	0.72 (0.52–0.99)	0.04	1.70 (1.51–1.92)	<0.0001	1.14 (0.9–1.45)	0.28
4	0.89 (0.62–1.29)	0.55	2.05 (1.78–2.36)	<0.0001	1.49 (1.14–1.96)	0.004
Trainee present	0.96 (0.74–1.23)	0.72	1.04 (0.93–1.16)	0.52	1.01 (0.84–1.22)	0.89
General anesthesia vs IV sedation	1.17 (0.81–1.7)	0.40	1.28 (1.12–1.46)	0.0004	1.41 (1.09–1.84)	0.01
Hospital characteristics
Adult cases >35% of total	0.16 (0.05–0.55)	0.003	1.37 (0.82–2.28)	0.23	0.73 (0.34–1.57)	0.42
University hospital	1.01 (0.72–1.41)	0.95	1.22 (0.97–1.52)	0.09	1.16 (0.87–1.53)	0.31
Annual catheterization volume (per 300 cases)	0.68 (0.55–0.84)	0.0003	0.95 (0.83–1.10)	0.53	0.79 (0.66–0.94)	0.009

AE indicates adverse event; FTR, failure to rescue; IV, intravenous; NA, not available; OR, odds ratio.

In contrast, composite AEs were not significantly associated with procedural volume (P=0.53) but were significantly associated with patient‐level factors (age, preprocedure inotropes, recent surgery or catheterization, genetic syndrome, renal insufficiency, and previously identified markers of hemodynamic vulnerability) and procedure‐level factors (procedure group and use of general anesthesia).

A similar pattern of positive association was seen between many of the previously identified patient‐level factors (age, inotropes, renal insufficiency, 3 of the 4 markers of hemodynamic vulnerability) and procedure‐level factors (procedure group 4 and receipt of general anesthesia) and catastrophic adverse outcome. Increasing PCCL volume was associated with reduced odds of catastrophic adverse outcome (OR: 0.79; 95% CI, 066–0.94; P=0.009).

Analogous models were calculated for AEs with deaths excluded and proximal AEs (Table S3), with both models resembling the model for all AEs. A sensitivity analysis was performed for the FTR model restricted to the first catheterization in the sample with no significant changes in the point estimates (Table S4).

To evaluate interactions between hospital factors, models including only procedural volume, adult volume >35%, and teaching hospital versus non–teaching hospital were calculated and demonstrated no differences from the primary model (Tables S5–S7).

Ranking Programs

For each hospital, standardized ratios of observed to expected events were calculated for FTR, all AEs, and catastrophic AEs. There was no correlation between hospital rankings by RSR for all AEs and RSR for FTR (Spearman r=−0.08, P=0.46; if deaths are excluded, Spearman r=−0.14, P=0.23). In contrast, hospital ranking based on RSR for FTR and RSR for catastrophic AEs were strongly correlated (Spearman r=0.65, P<0.05; Figure 2).

Figure 2

Correlation of standardized risk of all adverse events (AEs), failure to rescue (FTR), and catastrophic AEs. Scatter plots depict the association between hospital risk standardized ratio (RSR) for catastrophic AEs and FTR (A), all AEs and FTR (B), and all AEs with death excluded and FTR (C). Ranks of RSRs for FTR and catastrophic outcome demonstrate significant correlation (Spearman r=0.65, P<0.05), whereas there was no significant correlation between RSRs for all AEs and FTR, regardless of whether deaths were included (P=0.46) or excluded (0.23).

Rank correlations based on hospital catheterization volume and our outcomes were also performed. There was a significant association between the ranks of hospitals based on procedural volume and RSR for FTR (r=−0.28, P=0.01; Figure 3) but not between annual catheterization volume and RSR for all AEs (P=0.32). There was of note a significant association between procedural volume rank and RSR for catastrophic volume (r=−0.21, P=0.048; data not shown).

Figure 3

Correlation of hospital volume and risk standardized ratio (RSR) of failure to rescue (FTR). This scatterplot compares hospital annual volume (x‐axis) and RSR for FTR. There is a significant association between hospital rank in terms of volume and FTR (r=−0.28, P=0.01), with higher hospital volume associated with improved (ie, lower) risk of failure. No significant correlation is seen between ranks for hospital volume and RSR for all adverse events (P=0.32, not shown).

Discussion

In this retrospective multi‐center cohort study, we sought to evaluate a modified version of FTR as a potential quality metric for PCCL programs. We compared its properties with a composite outcome of pooled major AEs that has been the standard in previous risk‐adjustment models.6, 7, 27 As in previous studies, the odds of pooled AEs were associated with a number of patient‐ and procedure‐level factors but not hospital factors (eg, procedural volume). FTR, in contrast, occurs only after an AE, and we expected it to be less influenced by case mix and more affected by the quality of the care team. As expected, the odds of FTR was associated with a smaller number of patient factors (that appeared to independently affect the risk of progressing from a relatively minor AE to a catastrophic one). Rankings of hospitals using case‐mix–adjusted AEs and rankings based on FTR were not correlated. Finally, rankings of hospitals by RSR for FTR were correlated with PCCL volume (a potential marker of hospital quality), whereas those for AEs were not. These findings suggest that FTR provides information about hospital performance not captured by case‐mix–adjusted AE and should be included in future research and quality‐improvement projects.

From a conceptual standpoint, several properties of FTR make it well suited as a measurement of quality for PCCL programs. The case mix in PCCL programs is complex, not only because of a broad range of patient characteristics (cardiac diagnoses, noncardiac conditions, and hemodynamic instability) but also because of a broad range of procedure‐associated risk.3, 4, 5, 7 The risk of AEs is strongly associated with these factors. FTR is associated with a much narrower range of these factors but is associated with procedural volume, a potential marker of programmatic quality. As a factor relatively independent of case mix, FTR may provide better discrimination of programmatic quality. This idea is supported by the fact that rankings based on risk‐adjusted pooled AEs do not correlate with those based on FTR, but rankings based on catastrophic AEs do. A possible explanation is that a greater number of AEs included in the composite outcomes are relatively minor. These relatively minor AEs are associated with the specific patient and procedure but do not appear to be associated with hospital characteristics. Even after case‐mix adjustment, these AEs (and thus composite AEs) may continue to be more reflective of case mix than FTR or catastrophic adverse outcomes.

To date, there has been greater experience in using case‐mix–adjusted AEs as a metric for adult and pediatric or congenital cardiac surgery. However, several concerns have been raised about the use of risk‐adjustment models to evaluate and report the quality of congenital heart surgery programs. First, risk‐adjustment models that are derived using large data sets may have excellent test characteristics but perform less well in discriminating between individual programs.34, 35, 36 There is a possibility of a ceiling effect for the highest risk patients, who are not captured in even the best risk‐adjustment model.35, 36 Even detailed models may not be able to account for all possible risk factors (and combinations of risk factors that may interact). The proposed FTR metric excludes the highest risk patients (emergent and salvage procedures) to avoid some of these concerns. There are also concerns that risk models derived using large data sets will have unreliable results when event rates are small for individual procedures.34, 35, 36 Finally, judging hospitals that do not engage in the entire spectrum of risk based on their performance with less risky patients or procedures produces an illusory estimate of their quality, as excellent performance in a lower risk pool does not predict similar results for higher risk cases.34, 35, 36 The proposed FTR metric studies the progress from AEs to a catastrophic AE and (as demonstrated in the current study) is relatively independent of patient‐ and procedure‐defined risk profiles. Consequently, FTR may be less subject to differences in hospital case mix than pooled AE.

A normative benefit of using FTR as a quality metric is that it encourages accurate reporting of relatively minor AEs, which might otherwise be reported differentially. This is because FTR removes a disincentive for reporting these (potentially equivocal) events and, because they are in the denominator of FTR, actually incentivizes reporting them. Catastrophic AEs are less equivocal, and reporting of them is less likely to be subjective and affected by incentives imposed by sharing of outcomes and potential public reporting. More important, using FTR as an outcome also ameliorates the disincentive to take on high‐risk or complex interventions and/or medically complex patients, who are associated with AE risk out of proportion to that predicted by risk‐adjustment models. Use of an FTR‐based ranking system places the emphasis of the metric on whether patients can be successfully shepherded through the PCCL without an AE progressing to a more catastrophic one, which is in greater alignment with the values of the field as a whole.

Our modified FTR metric also addresses the issue of attributability of postprocedure mortality, which has been questioned in the literature.28 PCCL patients differ from adult general surgical patients both in terms of the severity of their medical disease and the fact that these patients frequently undergo cardiothoracic surgery. Evaluating attributability is possible with review of the chart, but prospective adjudication of these events is challenging. In the IMPACT Registry, most of the component events in the current pooled AE composite outcome are recorded without regard to attributability. To improve the attributability of FTR to events in the catheterization laboratory, the proposed FTR metric counts deaths only if they occur within 2 days of catheterization and incorporates severe AEs that occur in combination with a more minor event in the catheterization laboratory or that were explicitly judged to be due to an event in the catheterization laboratory. We acknowledge that this strategy does not solve the issue of attributability, but it is a potential improvement over the current approach.

A factor that limited our ability to test the relative merits of composite AEs, FTR, and catastrophic AEs as quality metrics was the lack of validated hospital‐ or program‐level markers of quality. In the absence of any gold standard measures of hospital quality, we sought to use hospital annual PCCL volume as a potential marker of quality because procedural volume has been associated with superior outcomes across a broad swath of medical fields.29 Studies evaluating the volume–outcome relationship in PCCL have had equivocal results. A significant association between increasing procedural volume and reduced risk of catastrophic adverse outcomes has been demonstrated in several studies,25, 26 but when the composite outcome used included a broader range of less severe outcomes, the association was not seen.27 To clarify this, in the current study, we separately analyzed the associations between volume and risks of (1) FTR, (2) all AEs, and (3) catastrophic AEs. A significant association was demonstrated between procedural volume and the risk of FTR or catastrophic outcomes but not all AEs. Furthermore, there was a significant correlation of the ranks of hospitals by procedural volume with standardized RSR for FTR (and no such association was seen with RSR for all AEs). This finding by itself does not demonstrate that FTR is a superior outcome measure to all case‐mix–adjusted AEs. However, the association suggests that FTR has a stronger association with programmatic quality because it appears to be influenced by center‐level factors rather than patient‐ and procedure‐level factors. As the field progresses, in addition to comparing programs, future research should identify programmatic practices associated with improved outcomes. Future research evaluating other aspects of experience (eg, operator experience) would be useful but are not feasible at this time. At this point, we would recommend that both FTR and composite AEs be used in these studies.

The finding that hospitals with large proportions of adult patients were associated with reduced FTR was not anticipated. These hospitals are likely general hospitals with catheterization laboratory volume that includes coronary and structural heart cases (which are not recorded in IMPACT) on top of their recorded PCCL (underestimating their “true” volume). The observed association may be a correction for this underestimation of volume or some other factor (eg structures or processes present in institutions that perform other structural interventions such as transcatheter aortic valve replacement). Further research is necessary to clarify the factors from this association arises.

Further research is also necessary to evaluate the relative merits of FTR and traditional metrics of quality. Several other large data sets contain detailed information about PCCL procedures. Reevaluating this model in those databases may be a useful step in demonstrating its validity.

This study has several other limitations. It was an observational retrospective study, and causal inference is limited in this design. Identified associations do not imply causation. Moreover, there is always the risk of unmeasured confounding. Another concern regarding the implementation of FTR as a quality metric is that FTR is a relatively rare event, making quarterly reporting challenging. However, we would contend that for measuring the quality of a hospital program, it is paramount to find a metric with maximum utility and then to measure it over an adequate period to measure performance precisely. In addition, using catastrophic AEs (instead of FTR) does not increase statistical power (which depends on the number of events, which is similar between catastrophic AEs and FTR, not the rate of occurrence). As for quality‐improvement efforts, low event rates would make it a less agile measure. At the level of an individual catheterization laboratory, consideration of both FTR and a carefully selected pool of significant AEs may be the best means of maintaining quality. Finally, the study was performed using the IMPACT Registry, and the definitions of exposures, outcomes, and covariates are defined by those available in the database. As new versions of the registry are developed, updating the FTR metric to reflect available data (especially about outcomes) will be important.

Despite these limitations, we conclude that, as an outcome metric, FTR provides additional information that is not contained in case‐mix–adjusted AEs. FTR is complementary to case‐mix–adjusted AEs, offering a distinct perspective on identifying high‐ and low‐performing centers. We conclude that FTR should be included along with pooled AEs in future research and quality‐improvement efforts.

Sources of Funding

O'Byrne receives research support from the National Institutes of Health, National Heart, Lung, and Blood Institute (HL130420‐01), which also provided direct funding for this project. The funding agencies had no role in drafting the article or influencing its content. The proposed project and article were reviewed by the IMPACT Research and Publications Committee. The article does not represent the opinion of the funding agency, the IMPACT Research and Publications Committee, the American College of Cardiology Foundation, or the National Cardiovascular Data Registry. It is solely reflective of the views of the study authors.

Disclosures

O'Byrne has received honoraria from Gore Medical (Newark, Delaware). The remaining authors have no relevant disclosures to report.

Table S1. Unplanned Interventions and Preceding Other Events

Table S2. Individual Proximal Adverse Events

Table S3. Other Multivariable Models of Adverse Events

Table S4. Failure to Rescue Model Restricted to First Catheterization per Individual (n=53 056)

Table S5. Multivariable Model for Failure to Rescue With Hospital Volume as the Only Hospital Characteristic

Table S6. Multivariable Model for Failure to Rescue With University Hospital as the Only Hospital Characteristic

Table S7. Multivariable Model for Failure to Rescue With Large Volume Adult Hospital as the Only Hospital Characteristic

Click here for additional data file.