Accuracy of Emergency Severity Index in older adults.

BACKGROUND AND IMPORTANCE: Emergency Severity Index is a five-level triage tool in the emergency department that predicts the need for emergency department resources and the degree of emergency. However, it is unknown whether this is valid in patients aged greater than or equal to 65 years.
OBJECTIVE: The aim of the study was to compare the accuracy of the Emergency Severity Index triage system between emergency department patients aged 18-64 and greater than or equal to 65 years. DESIGN, SETTINGS, AND PARTICIPANTS: This was a retrospective observational cohort study of adults who presented to a Finnish emergency department between 1 February 2018 and 28 February 2018. All data were collected from electronic health records. OUTCOME MEASURES AND ANALYSIS: The primary outcome was 3-day mortality. The secondary outcomes were 30-day mortality, hospital admission, high dependency unit or ICU admission, and emergency department length of stay. The area under the receiver operating characteristic curve and cutoff performances were used to investigate significant associations between triage categories and outcomes. The results of the two age groups were compared. MAIN
RESULTS: There were 3141 emergency department patients aged 18-64 years and 2370 patients aged greater than or equal to 65 years. The 3-day mortality area under the curve in patients aged greater than or equal to 65 years was greater than that in patients aged 18-64 years. The Emergency Severity Index was associated with high dependency unit/ICU admissions in both groups, with moderate sensitivity [18-64 years: 61.8% (50.9-71.9%); greater than or equal to 65 years: 73.3% (63.5-81.6%)] and high specificity [18-64 years: 93.0% (92.0-93.8%); greater than or equal to 65 years: 90.9% (90.0-92.1%)]. The sensitivity was high and specificity was low for 30-day mortality and hospital admission in both age groups. The emergency department length of stay was the longest in Emergency Severity Index category 3 for both age groups. There was no significant difference in accuracy between age groups for any outcome.
CONCLUSION: Emergency Severity Index performed well in predicting high dependency unit/ICU admission rates for both 18-64 years and greater than or equal to 65-year-old patients. It predicted the 3-day mortality for patients aged greater than or equal to 65 years with high accuracy. It was inaccurate in predicting 30-day mortality and hospital admission for both age groups.

Introduction

The world population is aging rapidly, and the number of older emergency department (ED) patients is increasing [1-3]; however, acuity assessment for older adults remains an unsolved challenge [4].

Emergency Severity Index (ESI) is a five-level triage instrument that ranges from level one, with patients who require an immediate life-saving intervention, to level five, with less acute patients who are not estimated to require any ED resources [5]. ESI has been reported to perform at least as well as two other widely used triage instruments: the Canadian Triage Acuity Scale and the Manchester Triage Scale [6,7].

Only a few studies have focused on triage accuracy in older patients with ED [6]. ESI has been associated with mortality, hospital admission, ED length of stay (EDLOS), and resource utilization in older ED patients, although there is a risk of undertriage [8-11]. Two studies have reported on the association between ED outcomes and other five-level triage tools for older adults [12,13].

The predictive capacity of early warning scores and vital signs in older ED patients is equivocal [14-17]. Other screening tools have failed to predict ED outcomes. Several reviews on the Triage Risk Stratification Tool and Identification of Seniors at Risk have demonstrated a modest predictive value [18-21].

Considering the small number of studies on the subject and the ambiguity of results, there is still a need to explore the acuity assessment of older adults. The aim of this study was to determine whether the accuracy of ESI was similar for patients under and over 65 years. The primary outcome was 3-day mortality, and secondary outcomes were 30-day mortality, hospital admission to high dependency unit (HDU) or ICU, and EDLOS. To our knowledge, this is the first study to assess ESI in Northern Europe.

Methods

This was a retrospective observational study performed in the Tampere University Hospital ED between 1 February 2018 and 28 February 2018 (annual census: 100 000 patients). The ethical board of the University of Helsinki (HUS/2678/2017), the Helsinki University Hospital (HUS/280/2019), and the Tampere University Hospital (RI8602) approved the study protocol. The need for informed consent from patients was waived because this was an observational study.

Data collection

The data of every adult patient presenting to the ED during the study period were collected from electronic health records. The inclusion criteria for the study were as follows: (1) age greater than or equal to 18 years, (2) examined by an ED physician, and (3) arrived alive.

On arrival at the ED, a triage nurse assessed each patient. The majority of patients were triaged using ESI before being examined by a physician. A small number of patients presenting with minor complaints were discharged home at triage or were seen by an emergency nurse or a psychiatric team. These patients were not seen by an ED physician and were triaged using ESI. The presenting complaints for an emergency nurse visit were typically minor complaints, such as common cold or wound care.

Patients who were discharged home immediately after triage or who were triaged to be seen by a nurse or the psychiatric team were excluded from the analysis as they were not triaged using ESI or seen by an ED physician. Residents of the hospital district who die at home outside of office hours are transported to the ED to be examined by the ED physician. Patients who were dead upon arrival were excluded from the analysis (Fig. 1).

Fig. 1.

Patient selection flowchart. ED, emergency department; ESI, Emergency Severity Index.

The collected data elements included the following: date of birth, sex, time and date of arrival and departure, date of death (within 30 days from visit), triage category, and hospital and HDU/ICU admission. The date of death was obtained from the Digital and Population Data Services Agency, Finland [22]. No other follow-up data were collected.

Patient age was calculated as the difference between the date of arrival and the date of birth. The 3-day mortality was calculated as the difference between the date of death and date of arrival and recorded as an event if the difference was 3 days or less. The 30-day mortality was calculated in a similar manner. Admission to any hospital ward was documented as hospital admission. ICU and HDU admissions were combined into a single variable. EDLOS was calculated as the time between arrival and departure.

Analysis

The data were analyzed using the Statistical Package for the Social Sciences software version 25 [23] and the MedCalc software [24]. The patients were divided into two age groups: 18–64 years and greater than or equal to 65 years. Areas under the receiver operating characteristic curves were calculated for binary outcomes. All patients who were admitted and discharged were included in the mortality analysis. EDLOS was described using median and interquartile range (IQR).

To analyze the sensitivity of the test, a cutoff value of ESI 2 for 3-day mortality and HDU/ICU admission and a cutoff value of ESI 3 for 30-day mortality and hospital admission was used. The cutoff values were selected to be consistent with two recent reviews on this topic [6,7]. Sensitivity, specificity, and positive and negative predictive values were calculated for the outcomes.

Results

A total of 5909 adult patients presented to the ED during the study period. A total of 363 patients were not seen by an ED physician and were thus excluded from the analysis; two of them died on arrival. In addition, 35 dead-on-arrival patients who were triaged to category 5 were excluded from the analysis. The 3- and 30-day mortality rates were 0/361 for the excluded patients who arrived alive; the admission rate was 1/361 (one patient was admitted to the ophthalmology ward). The patient selection flowchart is presented in Fig. 1.

A total of 5511 patients were included in this study: 3141 ED patients aged 18–64 years [median age: 41 years; 1506/3141 (47.9%) were male] and 2370 ED patients aged greater than or equal to 65 years [median age: 78 years; 1052/2370 (44.4%) were male]. The events in each triage category are listed in Table 1.

Table 1.

Events in each triage category stratified by age

	ESI	N	3-day mortality	30-day mortality	Hospital admission[a]	HDU/ICU admission[a],[b]
Adults18–64 years	1	8 (0.3%)	1 (12.5%)	4 (50.0%)	7 (87.5%)	5 (62.5%)
	2	262 (8.3%)	2 (0.8%)	10 (3.8%)	176 (67.2%)	50 (19.1%)
	3	2142 (68.2%)	1 (0.05%)	13 (0.6%)	621 (29.0%)	34 (1.6%)
	4	634 (20.2%)	1 (0.2%)	3 (0.5%)	51 (8.0%)	0 (0.0%)
	5	95 (3.0%)	1 (1.1%)	1 (1.1%)	4 (4.2%)	0 (0.0%)
	Total	3141	6 (0.2%)	31 (1.0%)	859 (27.3%)	89 (2.8%)
Adults≥65 years	1	8 (0.3%)	4 (50.0%)	7 (87.5%)	7 (100.0%)	2 (28.6%)
	2	279 (11.8%)	10 (3.6%)	35 (12.5%)	251 (91.3%)	72 (26.5%)
	3	1899 (80.1%)	6 (0.3%)	77 (4.1%)	1117 (58.9%)	25 (1.3%)
	4	167 (7.1%)	0 (0.0%)	0 (0.0%)	35 (21.0%)	2 (1.2%)
	5	17 (0.7%)	0 (0.0%)	0 (0.0%)	2 (11.8%)	0 (0.0%)
	Total	2370	20 (0.8%)	119 (5.0%)	1412 (59.7%)	101 (4.3%)

ED, emergency department; ESI, Emergency Severity Index; HDU, high dependency unit.

Six older adults died in the ED and were excluded from admission analysis.

Data missing for five patients: one in the younger adult group and four in the older adult group.

Patients aged greater than or equal to 65 years were triaged more often [287/2370 (12.1%)] into high acuity (ESI 1–2) triage categories compared to 18–64-year-old patients [270/3141 (8.6%); P < 0.001]. The same applied to medium acuity triage categories: 2186/2370 (92.2%) of greater than or equal to 65-year-old patients and 2412/3141 (76.8%) of 18–64-year-old patients were triaged into ESI categories 1–3 (P < 0.001).

Mortality

The 3-day mortality was higher for patients aged greater than or equal to 65 years in every triage category (Table 1). There were few deaths in the 18–64-year-old patient group, which led to a nonsignificant result. ESI performed well in predicting 3-day mortality in patients aged greater than or equal to 65 years [area under the curve (AUC): 0.82; 95% confidence interval (CI): 0.70–0.93]. The age-stratified outcomes for mortality are shown in Table 2. With the cutoff value at ESI 2, specificity was high and sensitivity was low in both age groups.

Table 2.

Age-stratified emergency department outcomes for Emergency Severity Index

		3-day mortality	30-day mortality	Hospital admission	HDU/ICU admission
	Cutoff category	ESI 2	ESI 3	ESI 3	ESI 2
Adults 18–64 years	AUC (95% CI)	0.61 (0.28–0.94);P = 0.373	0.69 (0.58–0.81);P < 0.001	0.67 (0.65–0.69);P < 0.001	0.82 (0.77–0.87);P < 0.001
	Sensitivity (95% CI)	50.0% (11.8–88.2%)	87.1% (70.2–96.4%),	93.6% (91.8–95.1%)	61.8% (50.9–71.9%)
	Specificity (95% CI)	91.5% (90.5–92.4%)	23.3% (21.8–24.8%)	29.5% (27.7–31.5%)	93.0% (92.0–93.8%)
	NPV (95% CI)	99.9% (99.8–100.0%)	99.5% (98.6–99.8%)	92.5% (90.4–94.1%)	98.8% (98.5–99.1%)
	PPV (95% CI)	1.1% (90.4–92.4%)	1.1% (1.0–1.3%)	33.3% (32.6–34.0%)	20.4% (17.2–24.0%)
Adults ≥65 years	AUC (95% CI)	0.82 (0.70–0.93);P < 0.001	0.65 (0.60–0.71);P < 0.001	0.63 (0.61–0.65);P < 0.001	0.82 (0.77–0.87);P < 0.001
	Sensitivity (95% CI)	70.0% (45.7–88.1%)	100.0% (97.0–100.0%)	97.4% (96.4–98.2%)	73.3% (63.5–81.6%)
	Specificity (95% CI)	88.4% (87.0–89.7%)	8.2% (7.1–9.4%)	15.4% (13.2–17.9%)	90.9% (90.0–92.1%)
	NPV (95% CI)	99.7% (99.4–99.9%)	100.0%[a]	79.9% (73.7–85.0%)	98.7% (98.2–99.1%)
	PPV (95% CI)	4.9% (3.6–6.5%)	5.4% (5.4–5.5%)	63.1% (62.4–66.3%)	26.5% (23.2–30.1%)

CI not available due to zero false-negative cases.

AUC, area under the curve; CI, confidence interval; ED, emergency department; ESI, Emergency Severity Index; HDU, high dependency unit; NPV, negative predictive value; PPV, positive predictive value.

The 30-day mortality was higher in patients aged greater than or equal to 65 years compared to those aged 18–64 years in higher triage categories (Table 1). ESI had low accuracy with 30-day mortality in both groups. With the cutoff value at ESI 3, sensitivity was high for patients aged greater than or equal to 65 years and moderate for patients aged 18–64 years. The specificity was low in both groups.

Hospital admission

In every triage category, patients aged greater than or equal to 65 years were admitted to the hospital more often than those aged 18–64 years (Table 1). ESI had poor accuracy in both age groups. With the cutoff value at ESI 3, sensitivity was high and specificity was low in both age groups. Age-stratified outcomes are presented in Table 2.

High dependency unit/ICU admission

Patients aged 18–64 years were admitted to an HDU/ICU facility more often than those aged greater than or equal to 65 years in triage category 1 (Table 1). Two greater than or equal to 65-year-old patients triaged to category 4 required HDU/ICU admission. ESI had good accuracy in predicting HDU/ICU admission in both age groups. With the cutoff at ESI 2, sensitivity was low and specificity was high in both age groups. Age-stratified outcomes for HDU/ICU admission are presented in Table 2.

Emergency department length of stay

The median LOS for all patients was 240 (IQR: 156–349) min. The LOS for patients aged 18–64 years was 208 (IQR: 129–308) min, and for those aged greater than or equal to 65 years, it was 281 (IQR: 197–395) min. In both groups, EDLOS was the shortest in categories 1 and 5 and longest in category 3. The EDLOS according to the triage category in each group is shown in Table 3.

Table 3.

Emergency department length of stay (minutes) in each Emergency Severity Index category by age group

Adults 18–64 years			Adults ≥65 years
ESI category	N	Median (IQR)	ESI category	N	Median (IQR)
1	8	81 (62–221)	1	8	122 (95–209)
2	262	199 (126–302)	2	279	234 (147–340)
3	2142	239 (157–337)	3	1899	301 (207–408)
4	634	139 (85–209)	4	167	172 (109–247)
5	95	107 (56–199)	5	17	163 (66–215)

ED, emergency department; ESI, Emergency Severity Index; IQR, interquartile range.

Discussion

According to our results, ESI accurately predicted the 3-day mortality and HDU/ICU admissions for ED patients aged greater than or equal to 65 years. ESI predicted the 30-day mortality and hospital admission poorly for patients aged greater than or equal to 65 years. The performance of ESI in regards to 30-day mortality, hospital admission, and HDU/ICU admission rates was similar for patients aged 18–64 years.

Our results support two previous studies that found ESI to be an accurate triage tool for patients aged greater than or equal to 65 years [8,9]. Grossmann et al. found that ESI was associated with in-hospital mortality in older adults, and the predictive ability of ESI for ICU admission was similar to that of our study. However, both previous studies reported markedly better accuracy for hospital admission prediction compared to our findings.

Our results showed that patients aged greater than or equal to 65 years were triaged in the medium and high acuity categories more often. However, their HDU/ICU admission rates were lower in category 1 and higher in category 2 when compared to 18–64-year-old patients. In addition, two greater than or equal to 65-year-old patients in triage category 4 required HDU/ICU admission. These findings indicate undertriage, which has been reported in previous studies [8,11].

In our study, a cutoff age of 65 years was used for older adults. Choosing a higher cutoff age might have resulted in lower ESI accuracy in older adults. However, the selected cutoff age is consistent with previous studies on the topic and facilitates the comparison of results [8-13]. There is no established standard for reporting mortality among triage studies on older adults. Previous studies have used different measures, ranging from ED mortality to 1-year mortality. Triage studies concerning the general adult population have usually reported ED or in-hospital mortality; these studies typically have a larger number of study participants [6].

ESI was not associated with LOS in the study population. We hypothesized that the most unwell patients in the higher triage categories are seen and moved to HDU/ICU beds quickly, whereas patients in the lower triage categories are suitable for quick ‘fast track’ assessment and home discharge. The longest LOS was found in triage category 3, which may be due to crowding and long waiting times for hospital beds.

Our results support the use of ESI in acuity assessment for patients aged greater than or equal to 65 years. While the ESI is an imperfect tool, its performance appears to be comparable to that of the general adult population. However, the number of studies on this topic is still small, and there is variability in hospital admission prediction. Confirmatory studies are warranted to verify the performance of ESI in older ED patients.

Strengths and limitations

Although this was a single-center study, the number of included patients was relatively large. Both age groups were well represented. The number of patients in triage categories 1 and 5 was small, which contributed to a nonsignificant result for 3-day mortality in the younger adult group. The number of male patients in the oldest adult group was lower, which reflects the general sex distribution in Finland (44% of greater than or equal to 65-year-old adults are male). Data from electronic health records were comprehensive for all outcomes.

Conclusion

In this cohort study, ESI performed well in predicting HDU/ICU admission rates for both 18–64 and ≥ 65-year-old patients. It predicted the 3-day mortality for patients aged greater than or equal to 65 years with high accuracy. It had poor accuracy in predicting the 30-day mortality and hospital admission for both age groups.

Acknowledgements

This work was supported by Helsinki University Hospital and the Finnish Association for Emergency Medicine.

Steve Kemp contributed to language editing and Sami Mustajoki contributed to acquisition of the data.

The datasets generated and analyzed are not publicly available due to national juridical restrictions protecting pseudonymized research data. The study permission acquired from the ethical board does not allow sharing pseudonymized data. Further description or analysis of data are available from the authors upon reasonable request.

Reporting guidelines: The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE).

Conflicts of interest

There are no conflicts of interest.