Derivation and validation of a clinical model to predict death or cardiac hospitalizations while on the cardiac surgery waitlist.

BACKGROUND: Waitlist management is a global challenge. For patients with severe cardiovascular diseases awaiting cardiac surgery, prolonged wait times are associated with unplanned hospitalizations. To facilitate evidence-based resource allocation, we derived and validated a clinical risk model to predict the composite outcome of death and cardiac hospitalization of patients on the waitlist for cardiac surgery.
METHODS: We used the CorHealth Ontario Registry and linked ICES health care administrative databases, which have information on all Ontario residents. We included patients 18 years or older who waited at home for coronary artery bypass grafting, valvular or thoracic aorta surgeries between 2008 and 2019. The primary outcome was death or an unplanned cardiac hospitalizaton, defined as nonelective admission for heart failure, myocardial infarction, unstable angina or endocarditis. We randomly divided two-thirds of these patients into derivation and one-third into validation data sets. We derived the model using a multivariable Cox proportional hazard model with backward stepwise variable selection.
RESULTS: Among 62 375 patients, 41 729 patients were part of the derivation data set and 20 583 were part of the validation data set. Of the total, 3033 (4.9%) died or had an unplanned cardiac hospitalization while waiting for surgery. The area under the curve of our model at 15, 30, 60 and 89 days was 0.85, 0.82, 0.81 and 0.80, respectively, in the derivation cohort and 0.83, 0.80, 0.78 and 0.78, respctively, in the validation cohort. The model calibrated well at all time points.
INTERPRETATION: We derived and validated a clinical risk model that provides accurate prediction of the risk of death and unplanned cardiac hospitalization for patients on the cardiac surgery waitlist. Our model could be used for quality benchmarking and data-driven decision support for managing access to cardiac surgery.

Waitlist management is an ongoing challenge for publicly funded health care systems because available resources are finite. This challenge has become more pervasive since the onset of the COVID-19 pandemic, as many nonemergent procedures have been postponed to preserve system capacity for patients with COVID-19.1

The decision to triage patients booked for cardiac surgery balances the likelihood of disease decompensation and the availability of operating room and intensive care unit resources. Our group has previously developed and validated the CardiOttawa Length of Stay Score to estimate the likelihood of high or low needs for postoperative intensive care unit resources,2 and the CardiOttawa Waitlist Mortality Score to support evidence-based prioritization for cardiac surgeries. These risk models have been combined into a single triage decision support tool that is used on a daily basis at our institution (available with sign up at https://cardiottawa.ottawaheart.ca). No models are available to predict unplanned hospitalizations for patients on the waitlist for definitive surgical interventions. We therefore conducted a population-based study in Ontario, Canada to derive and validate a clinical model to predict the composite outcome of death or unplanned cardiac hospitalizations in patients on the waitlist for cardiac surgery.

Methods

Study design and population

We conducted a population-based, retrospective cohort study of adult patients 18 years or older who were on the waitlist for aortic, mitral or tricuspid valve surgery, coronary artery bypass grafting (CABG) or surgery on the thoracic aorta in Ontario between Oct. 1, 2008, and Sept. 30, 2019. We restricted our study to patients who waited for surgery at home, and excluded those who were accepted to and removed from the waitlist on the same date.

Data source

We used the clinical registry of CorHealth Ontario and population-level administrative health care databases from ICES. ICES holds multiple population-based health databases of Ontario residents. CorHealth Ontario maintains a prospective registry of all patients who receive invasive cardiac procedures in Ontario; the data undergo selected chart audits and core laboratory validation.3

Using unique confidential identifiers, we deterministically linked the following data sets and analyzed them at ICES: the CorHealth Ontario registry (for detailed cardiac surgery waitlist data, operative priority, date and type of cardiac procedures, and physiologic and comorbidity data), the Canadian Institute for Health Information Discharge Abstract Database (for comorbidities, hospital admissions and in-hospital procedures), the Ontario Health Insurance Plan database (for physician service claims) and the Registered Persons Database (for vital statistics). These administrative databases have been validated for many outcomes, exposures and comorbidities, including heart failure, chronic obstructive pulmonary disease, asthma, hypertension, and diabetes.4–7 Ontario is the most populous province in Canada, with about 14.6 million residents, and is ethnically diverse.

Potential covariates included demographic, physiologic, anatomic and comorbidity data, as well as information about the planned procedure (i.e., operative priority status, recommended surgical wait time, redo sternotomy, type of surgery and acceptance to the waitlist during an inpatient encounter). We obtained data on height, weight and operative priority, and information pertaining to left ventricular ejection fraction, valvular disease and coronary anatomy from the CorHealth Ontario registry. In addition, we identified comorbidities from the CorHealth Ontario registry, which we supplemented with data from the Discharge Abstract Database and the Ontario Health Insurance Plan database using International Classification of Diseases and Related Health Problems, 10th Revision (ICD-10-CA) codes8 within 5 years before waitlist entry, according to validated algorithms.4,6,9,10

Outcome

The primary outcome was the composite of death or unplanned cardiac hospitalization, defined as nonelective admission for heart failure, myocardial infarction, unstable angina or endocarditis between the dates of waitlist entry and removal.

Statistical analysis

We compared the baseline characteristics of patients who died or had an unplanned cardiac hospitalization while on the waitlist with those of patients without these outcomes. We compared continuous variables with a 2-sample Student t test or Wilcoxon rank sum test, where appropriate, and categorical variables with a χ2 test. We followed patients from the date of entry to the waitlist until surgery or loss of eligibility for the Ontario Health Insurance Plan, whichever occurred first. We removed patients from the waitlist when they died or underwent their planned cardiac surgery.

We imputed missing values once using the SAS “proc MI” framework, where they were predicted by drawing on all candidate covariates, using predictive mean matching for continuous variables and logistic regression for categorical variables.11

We split the cohort into derivation and validation data sets by random selection, such that two-thirds of the cohort was used to derive the model.12 We predicted death or unplanned cardiac hospitalization using a Cox proportional hazard model.13 We selected predictor variables using a backward stepwise algorithm, with a significance threshold of p < 0.1 for entry and p < 0.05 for retention in the model.14 For continuous variables, we examined their association with the composite outcome using cubic spline analyses, with 5 knots at percentiles 5, 27.5, 50, 72.5 and 95. As there was no violation of the linearity assumption for any of these variables, we entered them into the model as continuous values. We validated the model on the remaining one-third of the cohort.

In both derivation and validation cohorts, we evaluated model discrimination in a time-dependent fashion using the area under the receiver-operating characteristic curve (AUC) at 15, 30, 60 and 89 days. We assessed calibration using the Brier score,15 as well as time-dependent plots of observed versus predicted mortality rates within vigintiles of predicted risk in the validation cohort.

We performed the analysis using SAS 9.4 (SAS Institute, Cary, NC) and defined statistical significance by a 2-sided p value of < 0.05.

Ethics approval

The use of data was authorized under section 45 of Ontario’s Personal Health Information Protection Act, which does not require review by a research ethics board.3

Results

Among 62 375 patients, 3033 (4.9%) died or were nonelectively hospitalized for cardiac causes while on the waitlist. These events occurred in 2091 (6.4%) of all 32 572 patients awaiting isolated CABG, 500 (2.6%) of 19 281 patients awaiting valve procedures, 415 (5.2%) of 7933 patients awaiting combined CABG and valve procedures, and 27 (1.0%) of 2589 patients awaiting thoracic aorta procedures. The characteristics of patients who did or did not have an event are presented in Table 1.

Table 1:

Baseline characteristics of patients who died or had unplanned cardiac hospitalizations while on the waitlist for cardiac surgery, compared with those who did not

Variable	No. (%) of patients		Standardized differences
	Eventn = 3033	No eventn = 59 342
Demographics
Age, yr, mean ± SD	67.1 ± 10.3	66.3 ± 10.9	0.07
Age, yr, median (IQR)	68 (60–74)	67 (60–74)	0.05
Sex, female	756 (24.9)	15 626 (26.3)	0.03
BMI, kg/m2, mean ± SD,	28.56 ± 5.38	28.95 ± 5.56	0.07
BMI, kg/m2, median (IQR)	28 (25–31)	28 (25–32)	0.07
Rural residence	2619 (86.4)	50 619 (85.3)	0.03
Hospital type
Community	379 (12.5)	14 953 (25.2)	0.33
Teaching	2654 (87.5)	44 389 (74.8)
Waitlisted during inpatient encounter	770 (25.4)	2913 (4.9)	0.6
Comorbidities
Hypertension	2662 (87.8)	49 488 (83.4)	0.12
Atrial fibrillation	418 (13.8)	7212 (12.2)	0.05
Recent myocardial infarction	437 (14.4)	2119 (3.6)	0.39
CCS classification
0	575 (19.0)	21 386 (36.0)	0.39
1	483 (15.9)	8805 (14.8)	0.03
2	569 (18.8)	14 989 (25.3)	0.16
3	603 (19.9)	11 828 (19.9)	0
4	163 (5.4)	1112 (1.9)	0.19
Low-risk acute coronary syndrome	360 (11.9)	858 (1.4)	0.43
Intermediate-risk acute coronary syndrome	260 (8.6)	339 (0.6)	0.39
High-risk acute coronary syndrome	20 (0.7)	25 (0.0)	0.1
Left main or left main equivalent disease	1311 (43.2)	18 319 (30.9)	0.26
Proximal LAD artery disease	1346 (44.4)	19 571 (33.0)	0.24
Previous percutaneous coronary intervention	375 (12.4)	6047 (10.2)	0.07
Left ventricular ejection fraction, %
≥ 50	2108 (69.5)	46 417 (78.2)	0.2
35–49	611 (20.1)	9367 (15.8)	0.11
20–35	273 (9.0)	3057 (5.2)	0.15
< 20	41 (1.4)	501 (0.8)	0.05
NYHA classification
1	2038 (67.2)	35 438 (59.7)	0.16
2	409 (13.5)	12 920 (21.8)	0.22
3	460 (15.2)	10 251 (17.3)	0.06
4	126 (4.2)	733 (1.2)	0.18
Heart failure	968 (31.9)	13 843 (23.3)	0.19
Moderate-severe mitral regurgitation	220 (7.3)	6951 (11.7)	0.15
Moderate-severe aortic regurgitation	69 (2.3)	2278 (3.8)	0.09
Severe aortic stenosis	687 (22.7)	18 980 (32.0)	0.21
Endocarditis
None	3007 (99.1)	58 852 (99.2)	0
Acute	13 (0.4)	154 (0.3)	0.03
Subacute	13 (0.4)	336 (0.6)	0.02
Cerebrovascular disease	314 (10.4)	5451 (9.2)	0.04
Peripheral arterial disease	412 (13.6)	7942 (13.4)	0.01
Smoking status
Never	1448 (47.7)	29 074 (49.0)	0.03
Current	604 (19.9)	9129 (15.4)	0.12
Former	981 (32.3)	21 139 (35.6)	0.07
Chronic obstructive pulmonary disease	795 (26.2)	13 111 (22.1)	0.1
Diabetes	1435 (47.3)	23 204 (39.1)	0.17
Dyslipidemia	2116 (69.8)	39 600 (66.7)	0.07
GFR, mL/min/1.73 m2, mean ± SD	82.0 ± 35.2	86.2 ± 34.1	0.12
GFR, mL/min/1.73 m2, median (IQR)	79 (58–103)	82 (62–105)	0.11
Dialysis	90 (3.0)	1074 (1.8)	0.08
Anemia	197 (6.5)	2344 (3.9)	0.11
Liver disease	37 (1.2)	561 (0.9)	0.03
Alcohol abuse	43 (1.4)	509 (0.9)	0.05
Dementia	47 (1.5)	656 (1.1)	0.04
Depression	51 (1.7)	415 (0.7)	0.09
Psychosis	6 (0.2)	70 (0.1)	0.02
Primary cancer	150 (4.9)	2887 (4.9)	0
Metastatic cancer	18 (0.6)	287 (0.5)	0.02
Operative characteristics
Surgery type
CABG	2091 (68.9)	30 481 (51.4)	0.36
Valve	500 (16.5)	18 781 (31.6)	0.36
CABG + valve	415 (13.7)	7518 (12.7)	0.03
Thoracic vorta	27 (0.9)	2562 (4.3)	0.22
Redo sternotomy	131 (4.3)	2047 (3.4)	0.05
Cardiogenic shock	1–5†	56 (0.1)	0.01
Operative priority
Urgent	988 (32.6)	20 296 (34.2)	0.03
Semiurgent	844 (27.8)	11 180 (18.8)	0.21
Elective	1201 (39.6)	27 866 (47.0)	0.15
Wait time characteristics
Recommend maximum wait time, d, mean ± SD	41.3 ± 31.0	43.7 ± 34.2	0.07
Recommend maximum wait time, d, median (IQR)	31 (14–62)	40 (14–71)	0.05
Adherence to recommended wait time‡	1999 (65.9)	31 126 (52.5)	0.28
All-cause ED visits while on the waitlist, mean ± SD	0.3 ± 0.6	0.1 ± 0.5	0.28
All-cause ED visits while on the waitlist, median (IQR)	0 (0–0)	0 (0–0)	0.35
All-cause outpatient physician visits while on the waitlist, mean ± SD	1.0 (1.5)	2.1 (1.8)	0.63
All-cause outpatient physician visits while on the waitlist, median (IQR)	1 (0–2)	2 (1–3)	0.75

Note: BMI = body mass index, CABG = coronary artery bypass grafting, CCS = Canadian Cardiovascular Society, ED = emergency department, GFR = glomerular filtration rate, IQR = interquartile range, LAD = left anterior descending, NYHA = New York Heart Association, SD = standard deviation.

Unless indicated otherwise.

Data suppressed because of small cell size.

Adherence is defined as adhering to the procedure-specific wait times recommended by the Canadian Cardiovascular Society Access to Care Working Group.1

The median time from waitlist entry to the composite outcome was 33 (interquartile range [IQR] 16–57) days in all patients, and 27 (IQR 13–48) days for patients waiting for CABG, 41 (IQR 22–71) days for patients waiting for valve surgery, 35 (IQR 20–59) days for patients waiting for combined CABG and valve surgery, and 46 (IQR 27–75) days for patients waiting for thoracic aorta procedures.

Of 41 792 patients in the derivation cohort, 2035 (4.9%) died or had unplanned cardiac hospitalizations while awaiting surgery. Of 20 583 patients in the validation cohort, 998 (4.8%) had an event while on the waitlist.

Predictors of death or unplanned cardiac hospitalization on the waitlist

The final predictive model, based on the derivation cohort, consisted of 16 variables (Table 2): body mass index, entry to the waitlist during an inpatient encounter, urban residence, treatment at a teaching hospital, myocardial infarction within 30 days, Canadian Cardiovascular Society and New York Heart Association classifications, history of heart failure, atrial fibrillation, diabetes, glomerular filtration rate, proximal left anterior descending artery disease, aortic stenosis, endocarditis, operative priority at the time of waitlist entry and type of planned surgery.

Table 2:

Multivariable predictors of patient death or unplanned cardiac hospitalization while on the waitlist for cardiac surgery

Variable	HR (95% CI)	p value
Demographics
BMI	0.99 (0.98–1.00)	0.004
Teaching v. community hospital	1.79 (1.56–2.05)	< 0.001
Waitlisted during inpatient encounter	5.05 (4.48–5.70)	< 0.001
Rural residence	0.86 (0.75–0.97)	0.02
Comorbidities
CCS classification
0	Reference
1	1.59 (1.36–1.87)	< 0.001
2	1.20 (1.02–1.40)	0.03
3	1.49 (1.27–1.75)	< 0.001
4	2.74 (2.18–3.45)	< 0.001
Low-risk acute coronary syndrome	6.01 (4.96–7.29)	< 0.001
Intermediate-risk acute coronary syndrome	8.37 (6.68–10.49)	< 0.001
High-risk acute coronary syndrome	8.08 (4.81–13.56)	< 0.001
Recent myocardial infarction	0.84 (0.72–0.97)	0.02
NYHA classification
1	Reference
2	0.72 (0.62–0.84)	< 0.001
3	0.94 (0.8–1.09)	0.4
4	1.78 (1.4–2.26)	< 0.001
Heart failure	1.4 (1.25–1.56)	< 0.0001
Atrial fibrillation	1.26 (1.1–1.44)	0.0007
Diabetes	1.13 (1.03–1.24)	0.008
GFR, per 10 unit increase	0.98 (0.96–0.99)	0.003
Proximal LAD > 70% stenosis	1.11 (1.01–1.23)	0.03
Severe aortic stenosis	1.28 (1.08–1.52)	0.004
Endocarditis
None	Reference
Acute	2.17 (1.16–4.08)	0.02
Subacute	0.79 (0.37–1.66)	0.5
Operative characteristics
Surgery type
CABG	Reference
Valve	0.33 (0.27–0.41)	< 0.001
CABG + valve	0.61 (0.49–0.76)	< 0.001
Thoracic aorta	0.16 (0.1–0.27)	< 0.001
Operative priority
Urgent	1.45 (1.24–1.69)	< 0.001
Semiurgent	1.50 (1.34–1.68)	< 0.001
Elective	Reference

Note: BMI = body mass index, CABG = coronary artery bypass grafting, CCS = Canadian Cardiovascular Society, CI = confidence interval, GFR = glomerular filtration rate, HR = hazard ratio, LAD = left anterior descending, NYHA = New York Heart Association.

Model performance

In the derivation cohort, the AUCs of the model were 0.85, 0.82, 0.81 and 0.80 at 15, 30, 60 and 89 days, respectively. In the validation cohort, the AUCs were 0.83, 0.80, 0.78 and 0.78 (Figure 1), and the Brier Scores were 0.027, 0.037, 0.05, and 0.06 at 15, 30, 60 and 89 days, respectively. Figure 2 shows the calibration plots of observed and predicted deaths or unplanned cardiac hospitalizations on the waitlist, according to vigintiles of risk at each of the 4 time points. The observed and predicted numbers of events in the validation cohort were similar across all patients except among those in the highest probability vigintiles, where the predicted risk underestimated the likelihood of events in the 4 highest risk vigintiles at 15 and 30 days, overestimated the likelihood of events in the top vigintile at 60 and 89 days. The model has been coded into an online application (available with sign up at https://cardiottawa.ottawaheart.ca/).

Figure 1:

Continuous plot of area under the receiver-operating characteristic curve values against time, with 95% confidence interval (CI).

Figure 2:

Calibration plots of observed (dots) versus predicted (dashed line) death or cardiac hospitalization on the waitlist at (A) 15, (B) 30, (C) 60 and (D) 90 days, according to vigintiles of risk.

Characteristics of patients at high risk of death or unplanned cardiac hospitalization

As the risk of death or cardiac hospitalization was particularly high in the 2 highest probability vigintiles, we conducted a post hoc analysis to examine the baseline characteristics of these high-risk patients compared with other patients in the validation cohort (Appendix 1, Supplemental Table 1, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.210170/tab-related-content). Patients who were male, were urban dwellers, were treated at teaching hospitals, had more severe cardiac symptoms (i.e., higher CCS and NYHA classifications), were scheduled to undergo CABG surgery, had proximal left anterior descending artery or left main stem coronary artery stenosis, had reduced left ventricular ejection fraction, had reduced renal function or had a higher comorbidity burden, were more likely to be at highest risk of death or cardiac hospitalization.

Discussion

We have provided an algorithm for predicting accurate, time-dependent risk estimates of the composite outcome of death and unplanned cardiac hospitalizations while on the waitlist, derived from and externally validated on a large and ethnically diverse population. The variables included in our model are readily available at the time of referral for surgery.

Waitlist management is challenging in publicly funded health care systems. Prolonged wait times may lead to increased deaths, hospitalizations and procedure urgency while patients await their surgery,16 as well as greater postoperative mortality, morbidity and disability rates.17,18 Existing guidance for waitlist management for cardiac surgery are based on expert opinion, as few real-world data are available to inform evidence-based decision-making.19,20

To date, efforts to risk stratify patients on the waitlist have been limited by small data sets, mainly in patients waiting for CABG and heart transplantation.21–25 The only contemporary, population-based study that included most cardiac surgeries evaluated mortality rates and did not address other important complications while on the waitlist.19 Given the medical complexity of these patients and the risk of decompensation while waiting, a data-driven approach is needed to individualize wait times and facilitate safe and timely triaging.

We developed this waitlist model to prioritize high-risk patients needing definitive surgery, improve patient outcomes and reduce use of health care resources. Our model can be used to provide decision support for referring physicians and the surgery–anesthesiology team, as well as health care administrators, through time-dependent, individualized risk prediction. It could also be used in quality benchmarking and to compare wait-time metrics across centres. In addition, we have identified features of patients who are at the highest risk of death or unplanned cardiac hospitalization while on the waitlist, which could be used to identify those who are most likely to benefit from expedited surgery.

Although little literature exists on death and unplanned cardiac hospitalization while on the waitlist for cardiac surgery, the risk factors identified from our model are similar to those from other studies of death while on the waitlist. In a 2017 systematic review, Head and colleagues identified angiographic complexity, angina severity, symptoms on stress testing and left ventricular dysfunction as risk factors for death while on the CABG waitlist.25 In a population-based study of the cardiac surgery waitlist in Alberta, Canada, Senaratne and colleagues identified age, aortic surgery, reduced left ventricular ejection fraction, urgent surgery, prior myocardial infarction, hemodynamic instability during cardiac catheterization, hypertension and dyslipidemia as independent risk factors for death.19 The differences in risk factors between this and our study likely reflect our use of a composite end point instead of death alone, as well as differences in patient inclusion criteria. Specifically, Senaratne included patients awaiting emergent surgery and those awaiting their procedures as inpatients, whereas we excluded these patients from our study.

Current guidance on waitlist management is based primarily on anatomic factors, such as coronary and valvular disease. Our predictor variables include anatomic factors, patient comorbidities, severity of cardiac symptoms, patient demographics, hospital characteristics, as well as characteristics of the proposed procedure. Adherence to the Canadian Cardiovascular Society’s recommended wait times for cardiac procedures may be insufficient to ensure patient safety,19 whereas our model has the potential to reduce adverse outcomes and use of health care resources through accurate, individualized risk stratification. It could be combined with the CardiOttawa Length of Stay Score2 (available with sign up at https://cardiottawa.ottawaheart.ca) to personalize wait times and optimize capacity in the intensive care unit. Like all decision-support tools, these models are intended to assist clinicians and should be used along with clinical judgment when making decisions.

Limitations

We were unable to include information about medical therapy because publicly funded drug coverage is available only to Ontario residents 65 years and older. However, model performance was excellent based on information that is available. Some physiologic measures, such as brain natriuretic peptide, were not available in the databases we used. However, brain natriuretic peptide is not routinely measured in the perioperative setting and the inclusion of such parameters might have limited the applicability of the model. We were unable to determine the cause of death on the waitlist, but it is likely that most deaths were from cardiac causes. We limited our model to commonly performed cardiac surgeries and it may not perform similarly in different patients, such as those waiting for heart transplantation or ventricular assist devices.

Conclusion

Our model accurately predicts the risk of death or unplanned cardiac hospitalization in patients on the waitlist for cardiac surgery. The model performed well in a large population-based sample. It can be combined with the CardiOttawa Length of Stay tool to benchmark quality and performance across centres, as well as provide triage decision support for clinicians, hospital administrators and policy-makers to improve access to cardiac care and conserve system capacity. This model should be evaluated in other health care jurisdictions.