Derivation and Validation of a Clinical Model to Predict Intensive Care Unit Length of Stay After Cardiac Surgery.

Background Across the globe, elective surgeries have been postponed to limit infectious exposure and preserve hospital capacity for coronavirus disease 2019 (COVID-19). However, the ramp down in cardiac surgery volumes may result in unintended harm to patients who are at high risk of mortality if their conditions are left untreated. To help optimize triage decisions, we derived and ambispectively validated a clinical score to predict intensive care unit length of stay after cardiac surgery. Methods and Results Following ethics approval, we derived and performed multicenter valida tion of clinical models to predict the likelihood of short (≤2 days) and prolonged intensive care unit length of stay (≥7 days) in patients aged ≥18 years, who underwent coronary artery bypass grafting and/or aortic, mitral, and tricuspid value surgery in Ontario, Canada. Multivariable logistic regression with backward variable selection was used, along with clinical judgment, in the modeling process. For the model that predicted short intensive care unit stay, the c-statistic was 0.78 in the derivation cohort and 0.71 in the validation cohort. For the model that predicted prolonged stay, c-statistic was 0.85 in the derivation and 0.78 in the validation cohort. The models, together termed the CardiOttawa LOS Score, demonstrated a high degree of accuracy during prospective testing. Conclusions Clinical judgment alone has been shown to be inaccurate in predicting postoperative intensive care unit length of stay. The CardiOttawa LOS Score performed well in prospective validation and will complement the clinician's gestalt in making more efficient resource allocation during the COVID-19 period and beyond.

Canadian Cardiovascular Society

cardiac surgery intensive care unit

Discharge Abstract Database

Institute for Clinical Evaluative Sciences

New York Heart Association

Ontario Health Insurance Plan

University of Ottawa Heart Institute

Clinical Perspective

What Is New?

We derived and ambispectively validated CardiOttawa, a bimodal score for predicting short (≤2 days) and prolonged (≥7 days) intensive care unit length of stay after cardiac surgery.

What Are the Clinical Implications?

The CardiOttawa Score will complement the clinician's gestalt in making more efficient resource allocation and may be used as quality benchmark during the coronavirus disease 2019 (COVID‐19) period and beyond.

Since having been declared a Public Health Emergency of International Concern by the World Health Organization on January 30, 2020, the coronavirus disease 2019 (COVID‐19) outbreak has rapidly redefined societal norms and challenged healthcare systems across the globe. COVID‐19 was declared a pandemic on March 11, 2020. By then, the availability of intensive care unit (ICU) resources had already begun to fall short of the increasing number of critically ill patients in some regions. Amidst this crisis, surgical patients continue to require lifesaving ICU resources. Although elective surgical procedures have been universally postponed, a significant number of patients with advanced, symptomatic cardiac diseases continue to require cardiac surgery on an urgent basis to prevent disease decompensation and death. This need challenges system capacity, given the complex comorbidities that often coexist with cardiac surgical disease, as well as the demand for ICU monitoring after cardiac surgery.

The current paradigm of triage decision‐making is primarily driven by clinicians' judgment and experience, which has been shown to be highly inaccurate in predicting prolonged cardiac surgical ICU (CSICU) length of stay (LOS). Although several objective clinical CSICU LOS models have been proposed, they are all built on small single‐center data sets, lack multicenter external validation, and rely on intraoperative and postoperative data to achieve modest discrimination. With a goal to save more lives while maintaining an efficient and adaptable allocation of critical care resources during this crisis, we derived and ambispectively validated a pair of clinical prediction models to provide individualized predictions of CSICU LOS after major cardiac surgery.

Methods

The data set from this study is held securely in coded form at the Institute for Clinical Evaluative Sciences (ICES). While data‐sharing agreements prohibit ICES from making the data set publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at www.ices.on.ca/DAS.

Design and Selection Criteria

This is an ambispective study, where we began by deriving models to predict low and high ICU resource use after cardiac surgery (defined by CSICU LOS of ≤2 and ≥7 days, respectively), using data available at the University of Ottawa Heart Institute (UOHI). We validated these models using a concurrent cohort of non‐UOHI cardiac surgery patients in Ontario. We then tested these models prospectively at the UOHI.

Included were adult patients aged ≥18 years who underwent coronary artery bypass grafting, and/or aortic, mitral, and tricuspid valve surgery. Excluded were patients who underwent procedures requiring circulatory arrest, as well as cardiac transplantation and ventricular assist devices. For patients with multiple cardiac procedures during the study period, only the index procedure was included in the analyses.

Patient Population and Data Sources

Derivation Cohort

The UOHI research ethics board approved the study and waived the need for individual patient consent. All 6625 patients who underwent cardiac surgery at the UOHI between November 1, 2009, and March 31, 2015, and met the selection criteria were included in the derivation cohort. We used prospectively collected clinical data from Cardiocore, a multimodular data repository that captures detailed demographics, comorbidities, procedural details, and outcomes of all patients who underwent cardiac surgical procedures at the UOHI, a university‐affiliated tertiary referral center that performs the full scope of cardiac operations. Cardiocore is managed by a multidisciplinary committee and undergoes regularly scheduled quality audits. , ,

Validation Cohort

The validation cohort consisted of cardiac surgical patients from 7 other cardiac care centers in Ontario who met the selection criteria between October 1, 2008 and December 31, 2018. Ontario is the most populous province in Canada, with 13 million residents and one of the most ethnically diverse jurisdictions in the world. The use of data was authorized under section 45 of Ontario's Personal Health Information Protection Act, which does not require review by a research ethics board. We used the clinical registry data from CorHealth Ontario, and population‐level administrative healthcare databases available at ICES. CorHealth maintains a detailed prospective registry of all patients who undergo invasive cardiac procedures in Ontario, including demographic, comorbidity, and procedural‐related information. CorHealth data undergo selected chart audits and core laboratory validation.

Using unique confidential identifiers, we linked the CorHealth Ontario registry (date and type of cardiac procedures, physiologic, and comorbidity data) with the Canadian Institute for Health Information Discharge Abstract Database (DAD; comorbidities and hospital admissions), the Ontario Health Insurance Plan (OHIP) database (physician service claims), and the Registered Persons Database (vital statistics). These administrative databases have been validated for many outcomes, exposures, and comorbidities, including heart failure, chronic obstructive pulmonary disease, asthma, hypertension, myocardial infarction, and diabetes mellitus. , , ,

Potential covariates considered in the analyses are detailed in Table 1 and included age, sex, body mass index, smoking, hypertension, left ventricular ejection fraction, myocardial infarction within 30 days before surgery, Canadian Cardiovascular Society (CCS) angina class, New York Heart Association class, atrial fibrillation, endocarditis, stroke, peripheral arterial disease, glomerular filtration rate, dialysis, diabetes mellitus treated with oral hypoglycemics and/or insulin, anemia, emergent operative status, preoperative cardiogenic shock, redo sternotomy, and type of surgery. The definitions for these variables are provided in Table S1.

Table 1

Baseline Characteristics of the Derivation and Validation Cohorts

Variable	Derivation (n=6625)	Validation (n=79 196)
Demographic
Age, median (IQR), y	59 (67–75)	67 (60–75)
Age, n (%), y
≤40	188 (2.8)	850 (1.3)
41–64	2596 (39.2)	25 315 (38.7)
65–74	2163 (32.7)	22 690 (34.7)
75–84	1507 (22.8)	15 993 (24.5)
≥85	171 (2.6)	1629 (2.5)
Female sex, n (%)	1851 (27.9)	15 993 (24.5)
Body mass index, n (%), kg/m2
<18.0	61 (0.9)	0
18.1–24.9	1779 (26.9)	17 059 (26.1)
25.0–29.9	2577 (38.9)	25 769 (39.4)
30.0–34.9	1446 (21.8)	14 896 (22.8)
≥35.0	762 (11.5)	7686 (11.8)
Comorbidities, n (%)
Hypertension	4855 (73.3)	56 521 (86.4)
Myocardial infarction within 30 d of surgery	1407 (21.2)	16 185 (24.7)
Canadian Cardiovascular Society classification
0	2751 (41.5)	12 620 (19.3)
1	492 (7.4)	5583 (8.5)
2	1070 (16.2)	10 574 (16.2)
3	1100 (16.6)	10 963 (16.8)
4	1212 (18.3)	25 670 (39.2)
NYHA classification
0	2497 (37.7)	17 365 (26.5)
1	765 (11.6)	28 849 (44.1)
2	1430 (21.6)	8839 (13.5)
3	1526 (23.0)	8386 (12.8)
4	407 (6.1)	1971 (3.0)
Left ventricular ejection fraction, %
≥50	4914 (74.2)	44 844 (68.6)
35–49	1009 (15.2)	14 228 (21.8)
20–35	474 (7.2)	5421 (8.3)
<20	228 (3.4)	917 (1.4)
Atrial fibrillation	1117 (16.9)	4704 (7.2)
Endocarditis	128 (1.9)	847 (1.3)
Smoker (active or former)	4186 (63.2)	11 726 (17.9)
Stroke	748 (11.3)	6759 (10.3)
Peripheral arterial disease	718 (10.8)	8220 (12.6)
Diabetes mellitus on medications	1761 (26.6)	20 652 (31.6)
Anemia	2244 (33.9)	6912 (10.6)
GFR, mL/min per 1.73 m2
≥60	4921 (74.3)	49 260 (75.3)
30–59	1486 (22.4)	13 995 (21.4)
<30	218 (3.3)	2155 (3.3)
Dialysis	102 (1.5)	1432 (2.2)
Operative characteristics, n (%)
Emergent procedure	531 (8.0)	9930 (15.2)
Preoperative cardiogenic shock	244 (3.7)	2700 (4.1)
Redo sternotomy	539 (8.1)	2110 (3.2)
Type of surgery
CABG	2908 (43.9)	47 136 (72.1)
Single valve	1176 (17.8)	9245 (14.1)
Valve(s)±CABG	2541 (38.4)	9029 (13.8)

CABG indicates coronary artery bypass grafting; GFR, glomerular filtration rate; IQR, interquartile range; and NYHA, New York Heart Association.

As with our previous studies, height and weight were identified from the CorHealth registry, and procedural urgency was ascertained from CorHealth and OHIP using an established algorithm. , , , In addition, comorbidities were identified from the CorHealth Ontario registry and supplemented with data from DAD and OHIP using International Classification of Diseases, Tenth Revision (ICD‐10) (Canada), codes within 5 years before the index procedure, according to validated algorithms. , , ,

Outcomes

The primary outcome was the total length of CSICU stay during the index surgical admission. Specifically, short CSICU LOS was defined as ≤2 days and prolonged LOS was defined as ≥7 days.

Statistical Analysis

Continuous variables were compared with a 2‐sample t test or Wilcoxon rank sum test for non‐normally distributed data. Categorical variables were compared with a chi‐square test.

Model Development

In the derivation set, we developed separate logistic regression models to predict the probabilities of CSICU LOS of ≤2 days and ≥7 days, respectively. For each model, we used univariate logistic regression to examine the association of potential predictors that were available at the time of triage and were routinely reported to CorHealth Ontario, with CSICU LOS. According to methods described by Harrell et al, potential predictors of LOS with univariate P values <0.25 were considered for entry into a multivariable logistic regression model based on both clinical and statistical significance. We used a backward variable selection algorithm, retaining in the final multivariable model covariates with P values <0.05, as well as those deemed to be clinically important. The final LOS prediction models were termed the CardiOttawa LOS Score.

Model Evaluation

Model discrimination in both the derivation and validation data sets was assessed using the c‐statistic. We assessed calibration using the Hosmer‐Lemeshow chi‐square statistic and by comparing the number of observed versus expected events in each risk quintile. We assessed model performance using the Brier score. For each of the LOS models, we constructed a predictiveness curve in the validation data set by plotting ordered risk percentile on the x‐axis, and the probabilities of LOS ≤2 and ≥7 days, respectively, on the y axis. Other measures of model performance, such as sensitivity and specificity and positive and negative predictive values, were determined by examining LOS in higher or lower risk groups at the optimal cutoff value.

We prospectively tested these predictive models at our institution between April 6 to 20, 2020, and descriptive statistics for the testing period are presented below. Analyses were performed using SAS version 9.4 (SAS Institute Inc), with statistical significance defined by a 2‐sided P value <0.05.

Results

Derivation and Validation Cohorts

Among the 6625 patients in the derivation cohort, 4201 (63.4%) stayed in the CSICU for ≤2 days and 692 (10.4%) for ≥7 days. Among 65 410 patients in the validation cohort, 50 442 (77.1%) stayed in the CSICU for ≤2 days and 3364 (5.1%) for ≥7 days. The baseline characteristics of both cohorts were similar, with the exception that patients in the derivation cohort were younger and more likely to undergo complex surgery, smoke, and have atrial fibrillation and anemia. Patients in the validation cohort were more likely to have CCS angina class 4 symptoms and undergo isolated coronary artery bypass grafting (Table 1).

Predictors of LOS

The multivariable predictors of short and prolonged CSICU LOS are presented in Tables 2 and 3, respectively. Of the candidate covariates evaluated, younger age; female sex; lower body mass index, CCS angina class, and New York Heart Association class; higher left ventricular ejection fraction; and absence of atrial fibrillation, endocarditis, stroke, peripheral arterial disease, anemia, higher glomerular filtration rate, emergent operative status, preoperative cardiogenic shock, redo sternotomy, and procedure type, were predictors of short CSICU LOS.

Table 2

Multivariate Analysis of Patients With Cardiac Surgical ICU LOS of ≤2 Days Versus >2 Days

Variable	Model β Coefficient	OR (95% CI)	Wald Chi‐Square	P Value
Demographic
Age, y
≤40	NA	Reference	Reference	NA
41–64	−0.192	0.83 (0.56–1.23)	0.92	0.339
65–74	−0.404	0.67 (0.45–1.00)	3.95	0.047
75–84	−0.515	0.60 (0.40–0.90)	6.09	0.014
≥85	−0.795	0.46 (0.27–0.77)	8.99	0.003
Female sex	−0.169	0.84 (0.74–0.97)	6.18	0.013
Body mass index, kg/m2
<18.0	−0.0408	0.96 (0.54–1.72)	0.019	0.891
18.0–24.9	NA	Reference	Reference	NA
25.0–29.9	−0.194	0.82 (0.71–0.96)	6.12	0.013
30.0–34.9	−0.461	0.63 (0.53–0.75)	26.09	<0.0001
≥35.0	−0.703	0.50 (0.40–0.61)	43.05	<0.0001
Comorbidities
CCS classification
0	NA	Reference	Reference	NA
1	−0.0087	0.99 (0.78–1.26)	0.0051	0.94
2	0.147	1.16 (0.96–1.41)	2.25	0.13
3	0.0341	1.04 (0.86–1.25)	0.12	0.73
4	−0.197	0.82 (0.67–1.00)	3.72	0.05
NYHA classification
0	NA	Reference	Reference	NA
1	−0.0656	0.94 (0.76–1.16)	0.38	0.54
2	−0.208	0.81 (0.69–0.96)	5.94	0.01
3	−0.538	0.59 (0.50–0.69)	40.11	<0.0001
4	−1.288	0.28 (0.20–0.38)	60.57	<0.0001
LVEF, %
≥50	NA	Reference	Reference	NA
35–49	−0.386	0.68 (0.68–0.80)	21.69	<0.0001
20–35	−1.043	0.35 (0.28–0.44)	80.81	<0.0001
<20	−1.479	0.23 (0.16–0.34)	57.81	<0.0001
Atrial fibrillation	−0.302	0.74 (0.63–0.87)	14.25	0.0002
Endocarditis	−0.660	0.52 (0.33–0.81)	8.66	0.003
Stroke	−0.250	0.78 (0.65–0.93)	7.40	0.007
Peripheral arterial disease	−0.194	0.82 (0.69–0.99)	4.28	0.04
Anemia	−0.373	0.69 (0.61–0.79)	31.65	<0.0001
GFR, mL/min 1.73 m2
≥60	NA	Reference	Reference	NA
30–59	−0.463	0.63 (0.54–0.74)	32.45	<0.0001
<30	−0.805	0.45 (0.32–0.63)	21.72	<0.0001
Operative characteristics
Emergent procedure	−0.914	0.40 (0.31–0.52)	48.40	<0.0001
Preoperative cardiogenic shock	−1.218	0.30 (0.18–0.48)	24.59	<0.0001
Redo sternotomy	−0.539	0.58 (0.47–0.72)	25.27	<0.0001
Type of surgery
CABG	NA	Reference	Reference	NA
Single valve	0.0131	1.01 (0.82–1.25)	0.015	0.90
Valve(s)±CABG	−0.785	0.46 (0.39–0.54)	88.88	<0.0001

CABG indicates coronary artery bypass grafting; CCS, Canadian Cardiovascular Society; GFR, glomerular filtration rate; ICU, intensive care unit; LOS, length of stay; LVEF, left ventricular ejection fraction; NA, not available; NYHA, New York Heart Association; and OR, odds ratio.

Table 3

Multivariate Analysis of Patients With Cardiac Surgical ICU LOS of ≥7 Days Versus <7 Days

Variable	Model β Coefficient	OR (95% CI)	Wald Chi‐Square	P Value
Demographic
Age, y
≤40	NA	Reference	Reference	NA
41–64	−0.156	0.86 (0.48–1.53)	0.28	0.60
65–74	0.197	1.22 (0.67–2.21)	0.42	0.52
75–84	0.545	1.72 (0.94–3.17)	3.08	0.08
>85	0.552	1.74 (0.83–3.62)	2.17	0.14
Female sex	0.119	1.13 (0.92–1.38)	1.34	0.25
Body mass index, kg/m2
<18.0	0.039	1.04 (0.45–2.43)	0.008	0.93
18.0–24.9	NA	Reference	Reference	NA
25.0–29.9	0.300	1.35 (1.07–1.70)	6.42	0.011
30.0–34.9	0.427	1.53 (1.16–2.02)	9.25	0.0023
≥35.0	0.674	1.96 (1.41–2.72)	16.23	<0.0001
Comorbidities
NYHA classification
0	NA	Reference	Reference	NA
1	−0.069	0.93 (0.62–1.42)	0.10	0.75
2	0.330	1.39 (1.03–1.88)	4.65	0.031
3	0.763	2.14 (1.63–2.82)	29.91	<0.0001
4	1.307	3.70 (2.61–5.24)	54.08	<0.0001
LVEF, %
≥50	NA	Reference	Reference	NA
35–49	0.377	1.46 (1.14–1.87)	9.03	0.0027
20–34	0.788	2.20 (1.64–2.96)	27.32	<0.0001
<20	1.390	4.02 (2.76–5.84)	52.70	<0.0001
Hypertension	0.380	1.46 (1.16–1.85)	9.98	0.0016
Atrial fibrillation	0.358	1.43 (1.16–1.76)	11.24	0.0008
Endocarditis	0.941	2.56 (1.57–4.18)	14.30	0.0002
Anemia	0.333	1.40 (1.15–1.70)	11.20	0.0008
GFR, mL/min 1.73 m2
≥60	NA	Reference	Reference	NA
30–59	0.466	1.59 (1.27–2.00)	16.27	<0.0001
<30	0.807	2.24 (1.50–3.34)	15.68	<0.0001
Operative characteristics
Emergent procedure	1.059	2.88 (2.17–3.84)	52.80	<0.0001
Preoperative cardiogenic shock	1.062	2.89 (2.01–4.17)	32.64	<0.0001
Redo sternotomy	0.590	1.80 (1.38–2.35)	19.05	<0.0001
Type of surgery
CABG	NA	Reference	Reference	NA
Single valve	0.0999	1.11 (0.79–1.55)	0.33	0.57
Valve(s)±CABG	0.936	2.55 (2.01–3.24)	58.95	<0.0001

CABG indicates coronary artery bypass grafting; GFR, glomerular filtration rate; ICU, intensive care unit; LOS, length of stay; LVEF, left ventricular ejection fraction; NYHA, New York Heart Association; and OR, odds ratio.

Age and sex were forced into the prolonged LOS model on the basis of clinical significance. Other multivariable predictors of prolonged CSICU LOS were body mass index, New York Heart Association class, left ventricular ejection fraction, hypertension, atrial fibrillation, endocarditis, anemia, glomerular filtration rate, emergent operative status, preoperative cardiogenic shock, redo sternotomy, and procedure type.

Multivariable Analysis

Short Stay Model

In the derivation data set, the c‐statistic of the multivariable model was 0.78 and the Hosmer‐Lemeshow chi‐square statistic was 12.71 (P=0.12). In the validation data set, the c‐statistic of the multivariable model was 0.71 and the Hosmer‐Lemeshow chi‐square statistic was 626.9 (P<0.001). The Brier score was 0.16.

Table 4 shows the observed rates of short CSICU LOS according to each risk quintile. The observed and predicted numbers of patients having an LOS ≤2 days were similar among all except the lowest probability quintile, where the model tended to underestimate (observed rate 53.4%, predicted 44.3%). On examining the predictiveness curve (Figure—Panel A), 60% of patients had predicted probabilities exceeding the average rate of short stay. The optimal cutoff point on the receiver operating characteristic curve was at a predicted probability of 76.3%, with the following characteristics: sensitivity, 69.8%; specificity, 60.8%; positive predictive value, 85.7%; and negative predictive value, 37.4%.

Table 4

Observed Versus Predicted Number of Patients With a Cardiac Surgical ICU LOS of ≤2 Days in the Validation Cohort

Risk Quintile	Observed		Predicted		OR (95% CI)
	No.	Rate (95% CI)	No.	Rate (95% CI)
1 (Low likelihood)	6988	0.53 (0.52–0.54)	5792.6	0.44 (0.44–0.45)	Reference
2 (Low‐moderate)	9672	0.74 (0.73–0.75)	9379.3	0.72 (0.72–0.72)	2.47 (2.35–2.61)
3 (Moderate)	10 614	0.81 (0.80–0.82)	10 659.1	0.82 (0.81–0.82)	3.75 (3.55–3.97)
4 (Moderate‐high)	11 356	0.87 (0.86–0.87)	11 437.9	0.87 (0.87–0.87)	5.58 (5.25–2.93)
5 (High)	11 812	0.91 (0.90–0.91)	11 903.7	0.91 (0.91–0.91)	8.44 (7.88–9.04)

The 95% CIs were obtained through 200 bootstraps with replacement. ICU indicates intensive care unit; LOS, length of stay; and OR, odds ratio.

Figure 1

Predictiveness of the CardiOttawa LOS Score.

A, Predictiveness of the CardiOttawa LOS Score showing ordered distribution of the probability of short stay. The solid line represents the predicted probability. The dotted line represents the average probability of short stay. B, Predictiveness of the CardiOttawa LOS Score showing ordered distribution of the probability of prolonged stay. The solid line represents the predicted probability. The dotted line represents the average probability of prolonged stay.

Long Stay Model

In the derivation data set, the c‐statistic of the multivariable model was 0.85 and the Hosmer‐Lemeshow chi‐square statistic was 18.54 (P=0.02). In the validation data set, the c‐statistic of the multivariable model was 0.78 and the Hosmer‐Lemeshow chi‐square statistic was 131.43 (P<0.001). The Brier score was 0.047.

Table 5 is a calibration table showing the rates of prolonged CSICU LOS according to each risk quintile. The number of observed patients with an LOS ≥7 days was similar to that predicted among all quintiles. Specifically, the average observed probability of short stay was 0.8% in quintile 1 (predicted probability 0.9%), 1.7% in quintile 2 (predicted 1.6%), 3.0% in quintile 3 (predicted 2.5%), 5.5% in quintile 4 (predicted 4.6%), and 14.8% in quintile 5 (predicted probability 17.2%). On examining the predictiveness curve (Figure—Panel B), 22% of patients had predicted probabilities that exceeded the average rate of prolonged stay. The optimal cutoff point on the receiver operating characteristic curve was at a predicted risk of 3.9% (sensitivity, 73.2%; specificity, 68.8%; positive predictive value, 11.3%; negative predictive value, 97.9%). At the 25th, 50th, and 75th percentiles of risk, sensitivities were 95.6%, 85.3%, and 64.1%, respectively, whereas negative predictive values were 99.1%, 98.5%, and 97.5%, respectively.

Table 5

Observed Versus Predicted Number of Patients With a Cardiac Surgical ICU LOS of ≥7 days in the Validation Cohort

Risk Quintile	Observed		Predicted		OR (95% CI)
	No.	Rate (95% CI)	No.	Rate (95% CI)
1 (Low likelihood)	111	0.008 (0.007–0.01)	128.5	0.009 (0.009–0.009)	Reference
2 (Low‐moderate)	207	0.017 (0.014–0.019)	194.9	0.016 (0.016–0.016)	2.06 (1.63–2.60)
3 (Moderate)	400	0.030 (0.027–0.033)	330.2	0.025 (0.025–0.025)	3.83 (3.10–4.73)
4 (Moderate‐high)	710	0.055 (0.050–0.058)	594.4	0.046 (0.045–0.046)	7.08 (5.79–8.66)
5 (High)	1936	0.15 (0.14–0.15)	2253.2	0.17 (0.17–0.18)	21.26 (17.53–25.78)

The 95% CIs were obtained through 200 bootstraps with replacement. ICU indicates intensive care unit; LOS, length of stay; and OR, odds ratio.

Sensitivity Analysis

A small number of patients died before postoperative day 7, amounting to 24 (0.56%) of the derivation cohort and 583 (0.89%) of the validation cohort. As perioperative death and prolonged ICU LOS are highly correlated, we tested the ability of the CardiOttawa to predict death before postoperative day 7 or ICU LOS ≥7 days as a composite outcome in the validation data set. Model performance for this composite outcome was mostly unchanged as compared with that of prolonged ICU LOS alone. Specifically, the c‐statistic was 0.79, Hosmer‐Lemeshow P<0.001, and Brier Score 0.051.

CardiOttawa LOS Score

The β coefficients for the logistic models are presented in Tables 2 and 3 (online calculator available at https://cardi​ottawa.ottaw​aheart.ca/).

Prospective Testing

During the beta testing period from April 6 to 20, 2020, a total of 42 patients who were evaluated with the CardiOttawa LOS Score proceeded to have surgery on an urgent basis. Using a predictive threshold of ≥70%, 35 of 38 (92.1%) patients who were predicted to have CSICU LOS of ≤2 days actually did. One patient was predicted to have an LOS of ≥7 days but had intraoperative death. The remaining 3 patients were classified as “indeterminate” (ie, had predicted probabilities of ≤50% for both short and prolonged LOS). Of these patients, 2 had an LOS of between 2 and 7 days and 1 had an LOS of ≥7 days.

Discussion

Triaging decisions for cardiac surgery may be improved with the aid of objective evidence to more efficiently allocate ICU resources. However, evidence‐based decision‐support tools are lacking for this patient group. We developed and ambispectively validated clinical models to predict the likelihood of low and high CSICU resource use as defined by short (≤2 days) and prolonged (≥7 days) LOS, using variables that are readily available at the time of surgical referral. Our models predicted well during prospective testing.

Unlike previous models that were developed using small data sets and monotonically focused on predicting prolonged LOS, the CardiOttawa LOS Score demonstrated excellent performance in Ontario, which is the most populous and ethnically diverse province in Canada. An online calculator for these logistic models is available at https://cardi​ottawa.ottaw​aheart.ca/. The CardiOttawa LOS Score may help to optimize daily operative planning, whereby scheduling of cases with varying postoperative resource requirements could be staggered to maximize the number of urgent cases performed, while minimizing non‐COVID ICU bed occupancy at any given time.

To our knowledge, the multicenter CardiOttawa LOS Score is thus far the best performing model, and the only validated model to provide bimodal LOS prediction after cardiac surgery. Previous models have focused on predicting prolonged CSICU LOS, which has been inconsistently defined in the literature (ranging between ≥1 and ≥10 days). In a decade‐old study that systematically reviewed published CSICU LOS models and externally validated them using single‐center data (n=11 395), the areas under the receiver‐operating characteristic curve of 6 general cardiac surgery models ranged between 0.57 and 0.75 for predicting LOS of ≥2 days. Of these, the Parsonnet and the European System for Cardiac Operative Risk Evaluation (EuroSCORE), which were originally intended for the prediction of mortality, had the highest discrimination (area under the receiver operating characteristic curve 0.75 and 0.71, respectively). In the original single‐center study that evaluated the performance of the EuroSCORE in LOS prediction (n=1562), the additive EuroSCORE was found to have areas under the receiver operating characteristic curve of 0.76 and 0.67 for predicting CSICU LOS of ≥7 and 2 days, respectively. The logistic EuroSCORE performed similarly in predicting these end points. The CardiOttawa LOS Score is calibrated to modern practices and outcome patterns. It is comparably as parsimonious as the Parsonnet score and is simpler than the EuroSCORE while retaining elements of importance to triage decision‐making, such as the presence of endocarditis and disease symptom severity. It predicts CSICU LOS of ≤2 days with an area under the receiver operating characteristic curve of 0.71 and ≥7 days with an area under the receiver operating characteristic curve of 0.78 in a large representative validation cohort of >65 000 patients.

The CardiOttawa predictor variables are consistent with those identified in the literature and capture important information on patient demographics, comorbid conditions, and the urgency and complexity of the scheduled procedure. Triaging decisions for cardiac surgery have traditionally been driven by clinical judgment, which may be no better than a coin toss in predicting the exact CSICU LOS after surgery. In an era when the importance of ICU and hospital resource management cannot be overemphasized, it is worth noting that although clinicians are adept at identifying patients who will require a short CSICU LOS, they are only able to correctly identify those requiring a prolonged LOS 39% of the time. Thus, our high‐performing prolonged LOS model is well suited to complement the clinician's gestalt in the decision‐making process.

The implications of the CardiOttawa LOS Score relate to its ability to support triaging decisions by complementing the physician's assessment of disease acuity and clinical factors with real‐world data. The potential impact of the CardiOttawa LOS Score depends on the average CSICU LOS durations specific to each institution. At institutions with lower CSICU LOS durations after cardiac surgery, this score may help to identify the high resource users, whereas at institutions with longer CSICU LOS durations, this score may identify patients who are likely to have a rapid transition through the CSICU. Given its robust performance in prospective validation, the CardiOttawa LOS calculator could be used to benchmark the predicted versus observed CSICU LOS as a quality metric. It could also be used to identify patients who may benefit the most from preoperative optimization (ie, those who are most likely to require a prolonged LOS). Prospective studies are needed to examine whether a risk‐stratified approach to optimizing conditions such as anemia and glycemic control could reduce CSICU LOS, while carefully balancing the potential benefits of optimization against the risk of delaying the procedure. The caveat that applies to all decision‐support tools is pertinent, because the CardiOttawa LOS Score is intended to assist the clinician, who should ultimately synthesize the predictive score with clinical judgment in making decisions.

Strengths and Limitations

Major strengths of the CardiOttawa LOS Score are its generalizability in the broad cardiac surgery population, its suitability for use at the time of surgical referral, and its bimodal LOS prediction. As these models are intended to guide decisions regarding the timing of surgery based on disease acuity and anticipated ICU resource needs at a system level, it is important for model validation to be performed in a patient sample that is representative of the population they are intended to serve.

Our study has some limitations. First, as universal drug coverage is only available to Ontarians ≥65 years, we were unable to include information on prescription medications for all patients in the modeling process. However, medications have not routinely been incorporated in cardiac surgical risk models to date, and decision‐support tools require a balance between variable inclusiveness and model simplicity, limiting the incorporation of an exhaustive list of potential factors. Second, we lack certain detailed physiologic measures such as brain natriuretic peptide in the databases used. However, brain natriuretic peptide is not routinely performed in the perioperative setting. Third, we lack certain procedure‐related details such as the use of minimally invasive surgical techniques. However, such information is usually unavailable at the time of triage, before assignment of surgical staff and operative consultation by the attending surgeon.

Future Directions

More recent, a number of artificial intelligence algorithms have emerged to assist with CSICU LOS prediction, with some demonstrating promising results. However, these algorithms are still in the development phase and suffer from similar limitations as published statistical models (eg, single center with even smaller sample sizes, lack of external reproducibility, and a practical means of implementation). , Further work is needed before they can be launched into clinical practice.

Conclusions

The CardiOttawa LOS Score is a set of simple clinical risk models that predict the likelihood of a short (≤2 days) postoperative CSICU LOS with moderate accuracy, and a prolonged (≥7 days) LOS with a high degree of accuracy. The importance of these predictive models is underscored by the inclusion of a population‐based patient sample, its bimodal LOS prediction, and its utility in guiding triaging decisions in the COVID‐19 era and beyond. The care and outcomes of all patients requiring ICU resources may be substantially improved if clinical judgment is supported by objective quantification in the planning of care.

Sources of Funding

This study was supported by an operating grant from the Canadian Institutes of Health Research. Sun was named National New Investigator by the Heart and Stroke Foundation of Canada and is supported by the Ottawa Heart Institute Research Corporation and the Tier 2 Clinical Research Chair in Big Data and Cardiovascular Outcomes at the University of Ottawa. Ruel and Mesana are supported by endowed research chairs at the UOHI. This study is supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health and Long‐Term Care (MOHLTC). The authors acknowledge that the clinical registry data used in this analysis are from participating hospitals through CorHealth Ontario, which serves as an advisory body to the MOHLTC, is funded by the MOHLTC, and is dedicated to improving the quality, efficiency, access, and equity in the delivery of the continuum of adult cardiac and stroke care in Ontario, Canada.

Disclosures

None.

Table S1

References ,

Click here for additional data file.