Severity of illness scores at presentation predict ICU admission and mortality in COVID-19.

BACKGROUND: The COVID-19 pandemic has overwhelmed hospital systems in multiple countries and necessitated caring for patients in atypical healthcare settings. The goal of this study was to ascertain if the conventional critical care severity scores qSOFA, SOFA, APACHE-II, and SAPS-II could predict which patients admitted to the hospital from an emergency department would eventually require intensive care.
METHODS: This single-center, retrospective cohort study enrolled patients admitted to Vanderbilt University Hospital from the emergency room with symptomatic, confirmed COVID-19 infection between March 8, 2020 through May 15, 2020. Clinical phenotyping was performed by chart abstraction, and the correlation of the qSOFA, SOFA, APACHE-II, and SAPS-II scores for the primary endpoint of ICU admission and secondary endpoint of in-hospital mortality was evaluated.
RESULTS: During the study period, 128 patients were admitted to Vanderbilt University Hospital from the emergency room with COVID-19. Of these, 39 patients eventually required intensive care; the remaining 89 were discharged from the medical ward. All severity of illness scores demonstrated at least moderate ability to identify patients who would die or require ICU admission. Of the three severity of illness scores assessed, the APACHE-II score performed best with an AUC of 0.851 (95% CI: 0.786 to 0.917) for identifying patient that would require ICU admission. No patient with an APACHE-II score at the time of presentation less than 8 or qSOFA of 0 required intensive care unit (ICU) admission. All patients with an APACHE-II score less than 10 or qSOFA score of 0 survived to hospital discharge.
CONCLUSIONS: The APACHE-II score accurately predicts the eventual need for ICU admission. This may allow for risk-stratification of patients safe to treat in alternative health care settings and prognostic enrichment to accelerate clinical trials of COVID-19 therapies.

Introduction

The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has led to over 7.5 million global cases and 425,000 deaths due to coronavirus disease 2019 (COVID-19) as of June 13, 2020. An emerging challenge in the COVID-19 pandemic is identifying, among the patients presenting to the hospital, which patients are likely to require intensive care unit (ICU) and which can be managed without such intensive resources. Predicting ICU admission would help allocate clinical resources and allow prognostic enrichment in clinical trials. Various severity of illness scores has been developed to predict the risk of mortality at the time of ICU admission, including the sequential organ failure assessment (SOFA) (1), simplified acute physiology score (SAPS) II (2), and acute physiology and chronic health evaluation (APACHE) II (3). These scores have also demonstrated utility in evaluating disease severity outside of the ICU setting. SOFA score at the time of emergency department presentation predicts outcomes in severe sepsis (4). The APACHE-II predicts mortality of acute pancreatitis at initial presentation (5) and the long-term mortality of patients admitted with a COPD exacerbation outside of the ICU (6). Higher APACHE-II and SAPS-II scores also correlated with subsequent ICU admission in patients admitted to a tertiary intermediate care unit (7). The APACHE-II score was recently shown to predict ICU mortality in COVID-19 (8), but its performance at the time of hospital presentation and its ability to predict ICU admission for patients with COVID-19 remain unknown. Further, the performance of these risk scores, relative to newly proposed COVID-specific markers of severity of illness such as the neutrophil:lymphocyte ratio, have yet to be evaluated (9).

The performance of the SOFA, SAPS-II, APACHE-II severity of illness scores and the neutrophil:lymphocyte ratio were evaluated for their ability to predict ICU in a retrospective cohort of patients hospitalized for COVID-19 at a large academic medical center. We present the following article in accordance with the STROBE reporting checklist (available at http://dx.doi.org/10.21037/jeccm-20-92).

Methods

Setting and participants

This retrospective cohort study was conducted in accordance with the Declaration of Helsinki (as revised in 2013) and was approved by the Vanderbilt University institutional review board with wavier of informed consent (IRB#200537). Between March 8th and May15th, 2020, we enrolled all adult patients (18 years or older) who were admitted from the emergency department (ED) to Vanderbilt University Medical Center in Nashville, TN with symptomatic COVID-19, confirmed by SARS-CoV-2 testing. Patient outcomes were followed until June 12th at 5pm. Patients readmitted within 72 hours of discharge were analyzed as a single admission.

Variables and data sources

Study personnel reviewed electronic health records to collect baseline patient characteristics including age, gender, race, ethnicity, body mass index, home medications, comorbidities, and smoking history. Active malignancy was defined as receipt of chemotherapy, radiotherapy, or surgery for malignancy within 45 days of admission. Vital signs and laboratory values were recorded. Missing data was analyzed by pairwise deletion. The primary outcome was ICU admission at any point during the inpatient hospitalization. Secondary outcomes were mortality and hospital length of stay. Patients who expired without ICU transfer based on limitations in care (e.g., a patient who expired after transfer from the hospital ward to inpatient hospice) were analyzed as not experiencing ICU transfer. Patients discharged to hospice were analyzed as having died. Patients who remained hospitalized at the end of the follow-up period were censored for analyses of mortality.

Measurement of severity of illness scores

The SOFA (1), SAPS-II (2), APACHE-II (3), and qSOFA (10), were calculated using data collected within 24 hours of ED presentation. Calculation of SOFA scores substituted the oxygen saturation to fraction of inspired oxygen (S/F) ratio (11) in place of the PaO2 to fraction of inspired oxygen (P/F) ratio. APACHE-II scores were calculated without inclusion of the PaO2 term (12) and using bicarbonate instead of pH (13). Scores were calculated blinded to patient outcome.

Statistical Analysis

Sample size was estimated by the precision analysis. With the proposed sample size of 110, and an estimated ICU admission rate of 30%, the half-width of the two-sided 95% confidence interval (CI) for receiver operating characteristic (ROC) curve’s area under the curve (AUC) was less than 12%. Continuous variables were reported as median and interquartile range (IQR). Categorical variables were reported as frequencies and proportions. Between-group comparisons were made with the Mann-Whitney rank-sum test for continuous variables and the Fishers exact test for categorical variables. Comparisons between AUCs were made using Delong’s test. A two-sided P value <0.05 was used to indicate statistical significance without adjustment for multiple testing. ROC curves with 95% CI: for qSOFA score, SOFA score, SAPS-II score, APACHE-II score, and neutrophil:lymphocyte ratio were generated using R version 4.0 (14) and package pROC 1.16.2 (15).

Results

Patient cohort

A patient flow diagram is shown in Figure 1. One hundred twenty-eight patients with laboratory confirmed SARS-CoV-2 were admitted from the ED during the study period from March 8, 2020 to May 15, 2020. At the end of the follow-up period on June 12, 2020, 125 patients were discharged and three patients remained hospitalized, all of whom had already required ICU admission.

Figure 1

Patient flow diagram. 128 patients were admitted from an emergency department (ED) with COVID-19. At the end of follow-up, 86 patients recovered without requiring intensive care unit (ICU) care, 25 patients recovered after requiring ICU admission, 3 patients died after declining ICU care, 11 patients died after receiving ICU care, and 3 patients remained hospitalized.

ICU admissions

Overall, 39/128 (30.4%) of patients received ICU care. Table 1 shows baseline patient characteristics for patients who did and did not require ICU admission. The most common indication for ICU admission was hypoxemic respiratory failure (n=32, 82.1%). Other indications included altered mental status (n=2, 5.1%), arrhythmia (n=1, 2.6%) and increased nursing needs (n=4, 10.3%). Twenty-four patients (61.5%) were admitted directly to the ICU, while 15 (38.4%) patients were initially admitted to the ward and subsequently transferred to the ICU. The median time from presentation to ICU transfer was 3.0 days with IQR from 1.0 to 5.5 days.

Table 1

Clinical characteristics

Characteristic	All patients, n=128	No intensive care unitadmission, n=89	Intensive care unitadmission, n=39	P value
Age (years)	56.0 [45.4, 67.8]	53.4 [42.8, 67.5]	57.3 [49.9, 69.6]	0.15
Male sex	75 (58.6%)	48 (53.9%)	27 (69.2%)	0.12
Race/ethnicity
Caucasian race	49 (38.3%)	31 (34.8%)	18 (46.2%)	0.24
African American race	33 (25.8%)	26 (29.2%)	7 (17.9%)	0.20
Hispanic/Latino ethnicity	26 (20.3%)	18 (20.2%)	8 (20.5%)	1
Asian American	9 (7.0%)	5 (5.6%)	4 (10.6%)	0.45
Body mass index[†]	30.0 [26.4, 35.4]	30.2 [25.7, 35.9]	29.6 [27.5, 32.1]	0.84
Smoking status
Current smoker	9 (7.0%)	6 (6.7%)	3 (7.7%)	1
Former smoker	20 (15.6%)	11 (12.4%)	9 (23.1%)	0.18
Never smoker	93 (72.7%)	66 (74.2%)	27 (69.2%)	0.67
Medical comorbidities
Immunocompromised	7 (5.5%)	4 (4.5%)	3 (7.7%)	0.43
Hypertension	66 (51.6%)	46 (51.7%)	20 (51.3%)	1
Diabetes mellitus	32 (25.0%)	17 (19.1%)	15 (38.5%)	0.03
Asthma	11 (8.6%)	9 (10.1%)	2 (5.1%)	0.50
Chronic obstructive pulmonary disease	9 (7.0%)	6 (6.7%)	3 (7.7%)	1
Active malignancy	5 (3.9%)	1 (1.1%)	4 (10.3%)	0.03
Home medications
Angiotensin converting enzyme inhibitors or angiotensin receptor blockers	41 (32.0%)	26 (29.2%)	15 (38.5%)	0.31
Oral hypoglycemic agents	16 (12.5%)	10 (11.2%)	6 (15.4%)	0.57
Insulin	11 (8.6%)	7 (7.9%)	4 (10.3%)	0.73
Duration of symptoms at presentation[‡] (days)	7.0 [3.5,9.0]	7.0 [3.0,9.5]	6.5 [4.3,8.8]	0.93
Supplemental oxygen at admission	67 (52.3%)	36 (40.4%)	31 (79.5%)	<0.001
Severity of illness measure during first 24 hours
Sequential organ failure assessment (SOFA)	2 [2, 4]	2 [1, 3]	3 [2, 5]	<0.001
Simplified acute physiology score-II (SAPS-II)	20 [16, 28]	18 [14, 23]	25 [30, 35.8]	<0.001
Acute physiology and chronic health evaluation (APACHE)-II	10 [7, 13]	8 [6, 11]	13 [11, 19.8]	<0.001
Quick SOFA (qSOFA)	1 [1, 2]	1 [1, 1]	2 [1, 2]	<0.001
Neutrophil:lymphocyte ratio (NLR)	4.1 [2.5, 6.7]	3.6 [2.2, 5.5]	6.5 [3.8, 10.5]	<0.001
Oxygen saturation to percent inspired oxygen ratio (S/F)	351.9 [258.3, 438.1]	387.5 [340.7, 442.9]	182.0 [117.3, 340.7]	<0.001

Values are presented as the median [IQR] or No. (%).

body mass index was missing for 6 patients (4.2%); 5 in the non-ICU group and 1 in the ICU group.

estimated duration of symptoms missing for 11 patients, 8 in the non-ICU and 3 ICU patients. ICU, intensive care

In univariate analysis, patients who required ICU admission were more likely to have diabetes mellitus (38.5% vs. 19.1%, P=0.03), active malignancy (10.3% vs. 1.1%, P=0.03), or require supplemental oxygen at admission (79.5% vs. 40.4%, P<0.001). Gender and race were not significantly associated with risk of ICU admission.

Figure 2A shows the ROC curves for the three severity of illness scores. The C-index for the APACHE-II score (AUC =0.851, 95% CI: 0.786 to 0.917) was higher than the SAPS-II (AUC =0.758, 95% CI: 0.671 to 0.844, P=0.009), SOFA score (AUC =0.730, 95% CI: 0.642 to 0.817, P=0.003), and qSOFA score (AUC 0.713, 95% CI: 0.630 to 0.797, P=0.004). The neutrophil:lymphocyte ratio alone had a C-index of 0.756, but the addition of the neutrophil:lymphocyte ratio to the APACHE-II did not significantly improve predictive ability of the APACHE-II score (P=0.445). No patient with an APACHE-II score <8 experienced ICU admission during hospitalization, and 18.2% of patients with an APACHE-II score between 8 and 10 experienced ICU admission. Similarly, no patient with qSOFA score 0 experienced ICU admission during hospitalization, and 26.4% of patients with qSOFA score 1 experienced ICU admission. All patients who did not require supplemental oxygen on admission but still experienced critical illness had APACHE-II scores greater than 10. Figure 2B,C display the risk of ICU admission stratified by APACHE-II and qSOFA score, respectively. More patients had an APACHE-II score <8 than qSOFA score of 0 (35/128 vs. 19/128, P=0.21).

Figure 2

Severity of illness scores and intensive care unit (ICU) admission. For patients with COVID-19 presenting to the emergency department (ED), (A) displays the receiver operating characteristic curves for the outcome of ICU admission for the acute physiology and chronic health evaluation (APACHE) II score (AUC 0.851), simplified acute physiology score (SAPS) II (AUC 0.758), sequential organ failure assessment (SOFA) score (AUC 0.730), and quick SOFA (qSOFA) score (AUC 0.713); (B) displays the percent of patients who experienced ICU admission by APACHE-II score at presentation to the ED; (C) displays the percent of patients who experienced ICU admission by qSOFA score at presentation to the ED.

Patient outcomes

111 patients survived to hospital discharge, and 14 patients died. Three critically ill patients remained hospitalized at the end of follow-up. At the end of follow-up, the overall mortality was 10.9% for all patients and 30.5% for ICU patients.

All three models predicted in-hospital mortality. The SAPS-II score had the highest AUC (0.911, 95% CI: 0.856 to 0.966). The SAPS-II score did not perform better than either the APACHE-II (AUC =0.851, 95% CI: 0.766 to 0.936, P=0.072) or SOFA (AUC =0.826, 95% CI: 0.732 to 0.919, P=0.068), but did have a better AUC than the qSOFA (AUC =0.801, 95% CI: 0.692 to 0.911, P=0.028). The ROC curve of all three severity of illness scores is shown in Figure 3A. Figure 3B,C display the mortality risk stratified by APACHE-II and qSOFA score, respectively. No patient with an APACHE-II score less than 10 or qSOFA score of 0 died.

Figure 3

Severity of illness scores and mortality. For patients with COVID-19 presenting to the emergency department (ED), (A) displays the receiver operating characteristic curves for the outcome of death for the acute physiology and chronic health evaluation (APACHE) II score (AUC 0.851), simplified acute physiology score (SAPS) II (AUC 0.911), sequential organ failure assessment (SOFA) score (AUC 0.823), and quick SOFA (qSOFA) score (AUC 0.801); (B) displays the percent of patients who died by APACHE-II score at presentation to the ED; (C) displays the percent of patients who died by qSOFA score at presentation to the ED.

Discussion

This single-center cohort study found that, among all patients with COVID-19 admitted from the ED, the APACHE-II score more accurately predicted subsequent ICU admission than the SOFA, SAPS-II, or qSOFA score. Notably, both the APACHE-II and SOFA performed well using previously described modifications that obviated the need for frequently unavailable arterial blood gas measurements. We substituted S/F for the SOFA P/F value (11), eliminated the PaO2 term from the APACHE-II (12), and used a serum bicarbonate substitution for the APACHE-II pH term (13). In this cohort, patients with either a low APACHE-II score (<8) or qSOFA score (0) did not require ICU admission. All scores accurately predicted mortality. As previously reported, advanced age and history of diabetes mellitus were associated with poor outcomes. While this cohort did not identify non-Caucasian race as an independent risk factor for ICU admission or death among this cohort of patients hospitalized with COVID-19, the proportion of non-Caucasian requiring hospitalization was twice as high as observed in historical, institutional cohorts (16,17). The finding that APACHE-II scores can accurately predict subsequent ICU admission has two basic applications.

First, when making decisions regarding allocation of hospital resources, the APACHE-II score or qSOFA may identify patients with COVID-19 at low risk for ICU admission and death, who might be safely treated in lower acuity environments. Conversely, patients with COVID-19 and high APACHE-II scores presenting to facilities with limited ICU services might benefit from early transfer to a tertiary facility. APACHE-II score may have performed better than qSOFA, SOFA, and SAPS-II because APACHE-II incorporates a broader assessment of chronic comorbidities, which may influence both the severity of COVID-19 and an individual’s physiologic reserve. It is notable that, while hypoxemia at the time of presentation is highly associated with the need for ICU admission, six patients on room air during the first 24 hours of admission required ICU care during their hospitalizations. The APACHE-II score predicted an increased risk of ICU admission in all six cases.

Second, a significant challenge for designing randomized therapeutic trials in COVID-19 is that the majority of patients will improve without treatment, which threatens to dilute outcome events, increases the required sample size, and increases the number of patients exposed to toxicities without potential benefit. Clinical trial outcomes such as death and mechanical ventilation occur predominantly in ICU patients. Thus, enrolling COVID-19 patients at risk for ICU admission into early treatment trials may prognostically enrich trials by increasing event rates, allowing smaller sample sizes, and improving the benefit/risk ratio for participants.

This study has several considerations. All studies are prone to bias, but this was mitigated through the use of objective inclusion criteria (selection bias), data collection while blinded to patient outcome (outcome bias), pre-specified modeling approaches (over-fitting bias), and sample size determination to avoid type II error. Conduct at a single tertiary care center, small sample size, and lack of a validation cohort may, however, limit generalizability. Additionally, nearly 50% of patients did not have a C-reactive protein, ferritin, or d-dimer measured at admission, which precluded additional biomarker analysis. If validated in larger cohorts, the ability of APACHE-II scores to predict ICU admission and mortality could have significant implications for patient care and clinical trials during the ongoing COVID-19 pandemic.

Conclusions

The APACHE-II score calculated within 24 hours of admission accurately predicts the eventual need for ICU care, and no patients with APACHE-II score <8 or qSOFA score of zero required ICU care. This finding could allow for safe triaging of patients to alternative care sites at times of high healthcare resource utilization and prognostically enrich future therapeutic clinical trials.