Development and Validation of Simple Age-Adjusted Objectified Korean Triage and Acuity Scale for Adult Patients Visiting the Emergency Department.

PURPOSE: The study aimed to develop an objectified Korean Triage and Acuity Scale (OTAS) that can objectively and quickly classify severity, as well as a simple age-adjusted OTAS (S-OTAS) that reflects age and evaluate its usefulness.
MATERIALS AND METHODS: A retrospective analysis was performed of all adult patients who had visited the emergency department at three teaching hospitals. Sex, systolic blood pressure, diastolic blood pressure, pulse rate, respiratory rate, body temperature, O2 saturation, and consciousness level were collected from medical records. The OTAS was developed with objective criterion and minimal OTAS level, and S-OTAS was developed by adding the age variable. For usefulness evaluation, the 30-day mortality, the rates of computed tomography scan and emergency procedures were compared between Korean Triage and Acuity Scale (KTAS) and OTAS.
RESULTS: A total of 44402 patients were analyzed. For 30-day mortality, S-OTAS showed a higher area under the curve (AUC) compared to KTAS (0.751 vs. 0.812 for KTAS and S-OTAS, respectively, p<0.001). Regarding the rates of emergency procedures, AUC was significantly higher in S-OTAS, compared to KTAS (0.807 vs. 0.830, for KTAS and S-OTAS, respectively, p=0.013).
CONCLUSION: S-OTAS showed comparative usefulness for adult patients visiting the emergency department as a triage tool compared to KTAS. © Copyright: Yonsei University College of Medicine 2022.

INTRODUCTION

Emergency department (ED) overcrowding is a worldwide problem and something that affects the prognosis of patients.123 Accordingly, a triage system has been developed and used in order to select severely ill patients and efficiently utilize medical resources.4 In Australia, the Australian Triage Scale (ATS) was developed, and then in Canada, the Canadian Triage and Acuity Scale (CTAS) was developed by revising the research results of the ATS according to Canada. In addition, the Manchester Triage System (MTS) in the United Kingdom and the Emergency Severity Index (ESI) in the United States have been developed and used.5

In South Korea, the interest in triage system has also increased. In 2012, based on CTAS, the Korean Triage and Acuity Scale (KTAS), which encompasses the pre-hospital and hospital stages, was developed. KTAS is a triage system linked to the National Health Insurance, and has been applied by all EDs in Korea since 2016.6 Like CTAS, KTAS consists of a combination of objective items (consciousness level, fever, heart rate, etc.), subjective items felt by patients (pain, anxiety, etc.), and subjective items evaluated by medical staff (breathing effort, paleness, etc.). Classification is carried out through four stages: medical department, main symptoms, first order modifiers, and second order modifiers.

The Korean Society of Emergency Medicine provides provider qualifications after KTAS training for medical personnel with more than a year of experience working in the ED. Therefore, the current KTAS system requires specialized personnel, time and cost for training, and periodical training to maintain qualifications. However, even with such efforts, the results may vary depending on the classifier’s experience and ability due to the various subjective items. Furthermore, pain, a subjective factor of the patient, may cause overtriage.6 In addition, it takes time to classify patients as the first order modifiers and second order modifiers must be applied sequentially according to the main symptoms; and when the patient complains of two or more main symptoms, the time required for classification increases as the same sequence must be repeated for each main symptom.

Therefore, we developed an objectified KTAS (OTAS) with a shorter classification time without being affected by the experience and ability of the classifier by excluding the subjective elements of first and second order modifiers from KTAS and using only objective information. Subsequently, a simple age-adjusted objectified Korean Triage and Acuity Scale (S-OTAS) was developed to reflect age. The purpose of this study was to evaluate the usefulness of OTAS and S-OTAS compared to KTAS.

MATERIALS AND METHODS

Study design, population, and setting

This study performed a retrospective analysis of all adult patients who had visited the EDs of two urban tertiary teaching hospitals (Seoul National University Bundang Hospital, Dongguk University Ilsan Hospital) and one rural secondary teaching hospital (Hallym University Chuncheon Sacred Heart Hospital) from January to December 2019. All adult patients aged 18 years or older with a valid record of vital signs were eligible. Death on arrival, defined as patients who were declared dead upon arrival, was not eligible. Exclusion criteria included patients who had an invalid KTAS level value or discharge outcome, had visited the ED with a non-medical reason, had been transferred from another hospital, and/or had transferred to another hospital.

The scale of each hospital was as follows: Seoul National University Bundang Hospital 1328 beds, Hallym University Chuncheon Sacred Heart Hospital 402 beds, and Dongguk University Ilsan Hospital 654 beds. About 100000 patients visited Seoul National University Bundang Hospital annually, while about 40000 patients visited Hallym University Chuncheon Sacred Heart Hospital, and about 40000 patients visited Dongguk University Ilsan Hospital. This study was conducted and reported in line with the Transparent Reporting of a multivariate prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines,7 and was approved by the Institutional Review Board of these three institutions (Seoul National University Bundang Hospital: B-2004-606-002, Hallym University Chuncheon Sacred Heart Hospital: 2021-04-027-001, Dongguk University Ilsan Hospital: DUIH 2021-04-038). Informed consent was waived.

Data collection and processing

Age, sex, systolic blood pressure (SBP), diastolic blood pressure (DBP), pulse rate (PR), respiratory rate (RR), body temperature (BT), O2 saturation (Sat), level of consciousness, and KTAS level were investigated via electronic medical records of each hospital. Level of consciousness was classified by the Alert/Verbal/Painful/Unresponsive (AVPU) scale. Alert was defined as the patient being fully awake (this patient would spontaneously have their eyes open, respond to vocal stimulation/commands, and have motor functions). Verbal response was defined as the patient making some kind of response when being talked to, which could be in any of the three component measures of eyes, voice, or motor. Pain response was defined as the patient making a response to any of the three component measures on the application of pain stimuli, like a sternal rub, or a peripheral stimulus, such as squeezing of fingers. Unresponsive was defined as sometimes being seen noted as “unconscious” (this outcome was recorded if the patient did not give any eye, voice, or motor response to voice or pain stimulations/commands).8

When a patient visited the ED, KTAS level was implemented by medical personnel including doctors, nurses, and emergency medical technicians who were certificated as KTAS providers. KTAS levels were determined by investigating the patient’s history, SBP, DBP, BT, Sat, pain scores, and trauma mechanisms.

The primary outcome was a 30-day mortality (death within 30 days of visiting the ED), and the secondary outcomes were the 7-day mortality (death within 7 days of visiting the ED), ED mortality (death in the ED), intensive care unit (ICU) admission, and the rates of computed tomography (CT) scan and emergency procedures (endotracheal intubation, defibrillation, central line catheterization, and transcutaneous pacing). Patients who were discharged with improved condition were assumed to survive. If the patient was discharged for other reasons, such as discharge against medical advice or discharge without significant improvement, the records of outpatient visits after discharge were checked. If there was no outpatient record, the discharge result was processed as missing. For severity comparison, hospital mortality and ICU admission were used, and the rates of CT scan and emergency procedures were used for acuity comparison.8910

Simple age-adjusted objectified Korean Triage and Acuity Scale

While maintaining the basic characteristics of KTAS, we have developed a new triage method that excludes subjective factors and reflects only the main symptoms and objective factors. The main symptoms used by KTAS, which were complied with the Canadian Emergency Department Information System Presenting Complaint List Version 1.1, were used.1112

First, the contents that involve subjectivity, such as breathing effort (use of accessory muscles, breathing depth), skin color (pale), and pain score in the first order modifiers, were excluded. Then, an objective criterion was made. For SBP, Dugas, et al.13 defined hypotension as an SBP less than 100 mm Hg. Jones, et al.14 reported that an SBP less than 90 mm Hg showed a high odds ratio (OR) of mortality, ICU admission, and vasopressor use, and that the OR was even higher when an SBP was less than 80 mm Hg. For PR, Considine, et al.15 reported that the patients who had abnormal PR, which was defined as slower than 60 beats/min or faster than 100 beats/min, showed higher incidence of emergency calls compared to the patients with normal PR. Based on these studies, SBP < 80 mm Hg was defined as OTAS level 1, 80 mm Hg ≤ SBP < 100 mm Hg & (PR > 100 beats/min or PR < 60 beats/min) was defined as OTAS level 2, and SBP ≥ 100 mm Hg & (PR > 100 beats/min or PR < 60 beats/min) or 80 mm Hg ≤ SBP < 100 mm Hg & 60 beats/min ≤ PR ≤ 100 beat/min was defined as OTAS level 3. In the case of consciousness level, KTAS defines both a response to Pain and Glasgow Coma Scale (GCS) 3–8 as level 1; however, in a study comparing a GCS and AVPU scale, a response to pain and GCS 3–8 showed a big difference.16 With reference to this study, we defined the case of unresponsive among the AVPU scale as OTAS level 1 and the case of pain response or verbal response as OTAS level 2. In addition, systemic inflammatory response syndrome criteria and Sat, which already have objective criteria in KTAS, were used as they are (Table 1). Likewise, in the second order modifiers, all subjective factors, such as dehydration (reduction of skin elasticity, dry mucous membrane), were excluded (Supplementary Table 1, only online).

Table 1

KTAS & OTAS Triage Rules

First order modifiers	Level	KTAS	OTAS
Respiratory distress	1	Fatigue from excessive work of breathing, cyanosis, single-word speech, unable to speak, upper airway obstruction, lethargic or confused, or SpO2 <90	SpO2 <90
	2	Increased work of breathing, speaking phrases or clipped sentences, significant or worsening stridor but the airway protected, or SpO2 <92	90 ≤ SpO2 < 92
	3	Dyspnea, tachypnea, shortness of breath on exertion, no obvious increased work of breathing, able to speak in sentences, stridor without any obvious airway obstruction, or SpO2 <94	92 ≤ SpO2 < 94
Hemodynamic stability	1	Evidence of severe end-organ hypoperfusion: marked pallor, cool skin, diaphoresis, weak or thready pulse, hypotension, postural syncope, significant tachycardia or bradycardia, ineffective ventilation or oxygenation, decreased level of consciousness. Can also appear as flushed, febrile, toxic, as in septic shock	SBP <80
	2	Evidence of borderline perfusion: pale, history of diaphoresis, unexplained tachycardia, postural hypotension (by history), feeling faint on sitting and standing, or suspected hypotension (lower than normal blood pressure or expected blood pressure for a given patient)	80 ≤SBP <100 & (PR >100 or PR <60)
	3	Vital signs at the upper and lower ends of normal as they relate to the presenting complaint, especially if they differ from the usual values for the specific patient	SBP ≥100 & (PR >100 or PR <60)
Level of consciousness	1	Unable to protect airway, response to pain or loud noise only and without purpose, continuous seizure or progressive deterioration in level of consciousness, or GCS 3-8	Unresponsive
	2	Response inappropriate to verbal stimuli, loss of orientation to person, place or time, new impairment of recent memory, or altered behavior	Verbal or painful response
Temperature SIRS [PR >90, RR >20, (BT <36 or BT >38)]	2	Neutropenia (or suspected), chemotherapy or immunosuppressive drugs including steroids, or patient has evidence of infection, have 3 positive SIRS criteria, or show evidence of hemodynamic compromise, moderate respiratory distress or altered level of consciousness	3 positive SIRS [PR >90, RR >20, (BT <36 or BT >38)]
	3	Patient has <3 positive SIRS criteria but appear ill (i.e. flushed, lethargic, anxious, or agitated)	(BT <36 or BT >38) & (PR >90 or RR >20)
	4	Patient has fever as their only positive SIRS criteria and appear to be comfortable and in no distress	BT <36 or BT >38

KTAS, Korean Triage and Acuity Scale; OTAS, objectified Korean Triage and Acuity Scale; SpO2, peripheral capillary oxygen saturation; SBP, systolic blood pressure; GCS, Glasgow Coma Scale; SIRS, systemic inflammatory response syndrome; PR, pulse rate; RR, respiratory rate; BT, body temperature.

Second, minimal OTAS levels according to the main symptoms were established (Supplementary Table 2, only online). For example, in the diarrhea section (KTAS 5), only chronic diarrhea was an indicator of KTAS values. We hypothesized that there should be at least a mild acute exacerbation even if the patient’s symptoms were chronic diarrhea. Therefore, the minimal OTAS level was defined as level 4 in this case. Also, “bleeding disease” that requires listening to the patient’s medical history, and the “mechanism of the accident,” which requires accurate information in the pre-hospital stage, were excluded for rapid triage.

Finally, the highest level (lowest number) was selected by comparing the minimal OTAS level according to the main symptom and the OTAS level according to the objective criterion. The calculation of each OTAS level according to objective criteria, the selection of OTAS level according to main symptoms, and the process of selecting the highest value among them were all automated.

Moreover, since the prognosis of patients admitted to the ED varied according to age, for patients aged 65 years or older, the OTAS value was increased by 1 to obtain the S-OTAS.9 In addition, to reflect age as a continuous variable, a logistic regression age-adjusted objectified Korean Triage and Acuity Scale (LR-OTAS) model was made using logistic regression. Accordingly, OTAS, S-OTAS, and LR-OTAS were obtained using SBP, PR, Sat, RR, level of consciousness, BT, minimal OTAS level for each main symptom, and age.

Statistical analyses

Continuous variables were examined with the Shapiro-Wilk test to determine the normality of the distribution, and were then expressed as the median (interquartile range). Students’ t-tests or Wilcoxon rank-sum tests were performed depending on the normality of the variables’ distributions, as appropriate. Categorical variables were described as a number with a percentage and compared using the χ2-test or Fisher’s exact test, as appropriate.

A cross-table was used for the comparison between S-OTAS level and KTAS level. The rate of agreement between the S-OTAS level and KTAS level was expressed through the KTAS standard accuracy concordance. Scoring discrepancies were defined as the score differences between S-OTAS and KTAS, which were calculated by S-OTAS level minus KTAS level. A multivariable logistic regression model with the input method was used for obtaining LR-OTAS. The OTAS level and age were included for calculating the probability of 30-day mortality.

To investigate the relationship between death within 30 days and KTAS, OTAS, S-OTAS, and LR-OTAS, a generalized linear model was created with death within 30 days as the dependent variable and KTAS, OTAS, S-OTAS, and LR-OTAS as the independent variables. The area under the curve (AUC) of the receiver operating characteristic described and compared the overall performances of the three methods. The AUCs were calculated and compared according to the method of Hanley and McNeil.

All data processing and statistical analyses were performed using R software, version 4.0.3 (R Foundation for Statistical Computing, Vienna, Austria). A two-tailed p-value less than 0.05 was considered statistically significant.

RESULTS

Among the 167861 patients (Seoul National University Bundang Hospital 89728 patients, Chuncheon Sacred Heart Hospital 33741 patients, Dongguk University Ilsan Hospital 44392 patients) who visited the ED in 2019 from the three hospitals used for the study during the period, 49761 patients were eligible, after excluding 41220 patients under the age of 18, 287 patients who arrived in a state of death, and 76593 patients whose vital signs were missing. Among them, a total of 44402 patients were analyzed after excluding non-medical visit patients, patients who had been transferred from other hospitals after receiving the diagnosis and treatment at other hospitals which could affect the KTAS level, patients who had gone to other hospitals and thus the final results were unknown, and patients whose KTAS level or discharge results were missing (Fig. 1).

Fig. 1

Flow diagram showing patient inclusion. ED, emergency department; DOA, dead on arrival; KTAS, Korean Triage and Acuity Scale.

The characteristics of patients compared according to the S-OTAS level showed statistically significant differences between levels of all variables (Table 2). When comparing S-OTAS and KTAS, the overall agreement rate was 43.6% with KTAS as the reference value. For each score, level 1 was 71.9%, level 2 was 50.4%, level 3 was 46.5%, level 4 was 24.8%, and level 5 was 23.2%, showing a tendency that the lower the level, the lower the concordance rate. The scoring discrepancies between S-OTAS and KTAS ranged from -4 to 3 (Table 3).

Table 2

Clinical Characteristics of Patients according to the S-OTAS Level

Characteristics		Total (n=44402)	S-OTAS level					p value
			Level 1 (n=5077)	Level 2 (n=12977)	Level 3 (n=18375)	Level 4 (n=6525)	Level 5 (n=1448)
Age (yr)		57.3±19.9	73.7±14.5	70.0±16.1	45.6±15.2	54.7±19.2	44.4±13.3	<0.001
Male		21498 (48.8)	2752 (54.2)	6528 (50.3)	8735 (47.5)	3152 (48.3)	717 (49.5)	<0.001
SBP (mm Hg)		133.7±26.9	124.0±36.8	136.0±27.8	133.9±24.1	136.1±22.9	134.4±21.6	<0.001
DBP (mm Hg)		77.4±16.6	68.3±21.9	75.3±16.5	80.3±15.2	79.9±12.8	81.5±12.7	<0.001
PR (beats/min)		88.7±20.7	98.4±25.3	89.0±22.3	89.2±20.1	80.8±10.7	81.6±10.0	<0.001
RR (breaths/min)		20.0±2.9	22.9±5.4	20.5±2.6	19.8±1.6	17.7±0.9	17.7±0.9	<0.001
BT (°C)		36.9±1.0	36.9±1.7	36.9±1.1	36.8±0.8	36.7±0.7	36.7±0.6	<0.001
SpO2 (%)		97.3±3.4	93.1±7.5	97.2±2.2	98.1±1.6	98.1±1.5	98.2±1.4	<0.001
Mental status								<0.001
	Alert (%)	41752 (94.0)	2747 (54.1)	12657 (97.5)	18375 (100.0)	6525 (100.0)	1448 (100.0)
	Verbal response	1074 (2.4)	754 (14.9)	320 (2.5)	0 (0.0)	0 (0.0)	0 (0.0)
	Painful response	1251 (2.8)	1251 (24.6)	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)
	Unresponsive	325 (0.7)	325 (6.4)	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)
Outcomes
	30-day mortality	868 (2.0)	529 (10.4)	227 (1.7)	77 (0.4)	32 (0.5)	3 (0.2)	<0.001
	7-day mortality	533 (1.2)	362 (7.1)	127 (1.0)	31 (0.2)	12 (0.2)	1 (0.1)
	ED mortality	125 (0.3)	98 (1.9)	21 (0.2)	4 (0.0)	1 (0.0)	1 (0.1)
	ICU admission	3763 (8.5)	1523 (30.0)	1390 (10.7)	662 (3.6)	177 (2.7)	11 (0.8)
	CT scan	21707 (48.9)	3559 (70.1)	6662 (51.3)	8210 (44.7)	2955 (45.3)	321 (22.2)
	Procedure	762 (1.7)	490 (9.7)	189 (1.5)	64 (0.3)	18 (0.3)	1 (0.1)

S-OTAS, simple age-adjusted objectified Korean Triage and Acuity Scale; SBP, systolic blood pressure; DBP, diastolic blood pressure; PR, pulse rate; RR, respiratory rate; BT, body temperature; SpO2, peripheral capillary oxygen saturation; ED, emergency department; ICU, intensive care unit; CT, computed tomography.

Variables are expressed as mean±SD or n (%)

Table 3

S-OTAS Compared with KTAS

KTAS level	S-OTAS level					Concordance (%)
	Level 1	Level 2	Level 3	Level 4	Level 5
1	878	57	43	4	0	878/982 (89.4)
2	2051	2361	831	175	29	2361/5447 (43.3)
3	2010	9022	13806	4578	605	13806/30021 (46.0)
4	126	1340	3067	1674	532	1674/6739 (24.8)
5	12	197	628	94	282	282/1213 (23.2)
Total	5077	12977	18375	6525	1448	19001/44402 (42.8)

S-OTAS, simple age-adjusted objectified Korean Triage and Acuity Scale; KTAS, Korean Triage and Acuity Scale.

In distribution comparison between S-OTAS and KTAS, S-OTAS showed a sequential decrease from levels 1 to 5 regarding the rates of 30-day mortality, 7-day mortality, ED mortality, and ICU admission, while there was no increasing or decreasing tendencies in KTAS. The rate of CT scan was the highest at level 3 for both S-OTAS and KTAS, and the rates of CT scan at levels 1 and 2 were higher in S-OTAS than in KTAS. Regarding the rate of emergency procedures, it was the highest in level 1 in both S-OTAS and KTAS, and decreased sequentially according to the level. Compared to KTAS, S-OTAS showed a greater decrease according to the level (Fig. 2).

Fig. 2

Outcome distribution comparison between S-OTAS and KTAS. (A) 30-day mortality. (B) 7-day mortality. (C) ED mortality. (D) ICU admission. (E) CT scan. (F) Emergency procedure. S-OTAS, simple age-adjusted objectified Korean Triage and Acuity Scale; KTAS, Korean Triage and Acuity Scale; ED, emergency department; ICU, intensive care unit; CT, computed tomography.

The results of calculating AUC by applying the generalized linear model with KTAS, OTAS, S-OTAS, and LR-OTAS as independent variables with 30-day mortality as the outcome were as follows. In KTAS, AUC was 0.751, while AUC was 0.780 for OTAS, 0.812 for S-OTAS, and 0.837 for LR-OTAS, all showing improved results compared to KTAS, and showed a significant difference with p<0.001 (Fig. 3A). For 7-day mortality, AUCs were 0.783, 0.818, 0.846, and 0.869 for KTAS, OTAS, S-OTAS, and LR-OTAS, respectively. OTAS, S-OTAS, and LR-OTAS showed significant differences compared to KTAS (p<0.001) (Fig. 3B). In terms of ED mortality, AUCs were significantly different only between KTAS and LR-OTAS (0.848 and 0.907, respectively; p=0.001) (Fig. 3C). In the AUCs comparison of each group regarding ICU admission, there were significant differences between KTAS and OTAS (0.766 and 0.714, respectively; p<0.001) as well as KTAS and S-OTAS (0.766 and 0.749, respectively, p<0.001) (Fig. 3D). With regard to the rate of CT scan, AUCs were 0.571, 0.550, 0.583, and 0.623 for KTAS, OTAS, S-OTAS, and LR-OTAS, respectively, and there were statistically significant differences between S-OTAS and KTAS (p< 0.001) and LR-OTAS and KTAS (p<0.001) (Fig. 3E). S-OTAS and LR-OTAS also showed higher AUCs compared to KTAS in the rate of emergency procedures (Fig. 3F).

Fig. 3

AUC comparison between four triage and acuity scales. (A) 30-day mortality. (B) 7-day mortality. (C) ED mortality. (D) ICU admission. (E) CT scan. (F) Emergency procedure. AUC, area under the curve; KTAS, Korean Triage and Acuity Scale, OTAS, objectified Korean Triage and Acuity Scale; S-OTAS, simple age-adjusted objectified Korean Triage and Acuity Scale; LR-OTAS, logistic regression age-adjusted objectified Korean Triage and Acuity Scale; ED, emergency department; ICU, intensive care unit; CT, computed tomography.

DISCUSSION

In the present study, when comparing the AUCs of each model with 30-day mortality, with the rates of CT scan and emergency procedures as the outcomes, S-OTAS and LR-OTAS were higher than KTAS.

The effectiveness of current triage tools in the ED has been demonstrated in several studies. In the study by Lee, et al.,10 the CTAS score, severity of elderly patients (death and admission rate), and resource utilization (length of stay, cost, consultation, and the use of a CT scan) showed a strong correlation, and high predictability for immediate life-saving intervention was shown in the elderly. Also, in the study by Martins, et al.,17 MTS was associated with short-term mortality and was a powerful tool to distinguish patients who would stay in the ED for more than 24 hours. In the study by Cremonesi, et al.,18 the more severe the overcrowding of the ED, the longer the time to treatment for the non-urgent patients was, but the waiting time of urgent patients did not increase, proving that the triage priority system is effective in situations of overcrowding.

Contrary to these studies, there were studies showing the disadvantages of the triage system. Triage had various problems such as limited information, time pressure, various medical conditions, dependence on intuition, etc. As a result, more than 50% of patients who visited the ED were at level 3, an ambiguous intermediate stage.1419 In the study by Wuerz, et al.,20 the reliability between interrater and intrarater was low; and in the study by Mistry, et al.,21 the agreement between ESI score evaluated by the nurse and the reference standard was low. In the study by Han, et al.,22 the ambulance diversion and left-without-being-seen (LWBS) cases decreased and the ED length of stay decreased when triage was performed by physicians compared to the other cases. This means that the effectiveness of classification varies depending on the classifier. In the study by Zachariasse, et al.,23 there was a difference in MTS performance between age groups, as well as a difference in validity. In addition, there were differences in performance between evaluation agencies. In the KTAS-related study by Lee, et al.,6 there was an over-triage problem due to the subjective factor of pain, as well as a negative impact on the predictability of urgent patients.

Various efforts are being made to overcome the problems of this triage system. Canada developed a web-based triage decision support tool (eTRAIGE) based on CTAS. In recent years, the electronic Canadian Triage and Acuity Scale (eCTAS), a real-time electronic decision-support tool, has also been developed and used.82425 In the studies by McLeod, et al.,26 the effects before and after the use of eCTAS were compared, and the triage accuracy after eCTAS use showed significant improvement, as well as high interrater agreement and a remarkable decrease in the number of over-under-triages. However, the time required for classification increased, and the accuracy of classification at CTAS levels 4 and 5 decreased. In another eCTAS-related study, the use of eCTAS showed little effect on hospital admission, rate of LWBS, and time to physician initial assessment compared to before its usage.27 Recently, studies using machine learning are also actively progressing. Levin, et al.28 developed a machine-learning-based triage tool, which showed similar or improved results to ESI in identifying patients with critical outcomes. In the study by Goto, et al.,29 when machine learning approaches were applied to the ED triage, the discriminative ability to predict clinical and disposition outcomes was increased compared to the conventional triage approach, and high sensitivity was shown in the prediction of critical care outcomes. Yu, et al.30 combined machine learning with initial nurse assessment and compared it with KTAS, which was better in predicting ED death and ICU admission. However, despite the advantages of this triage system using machine learning, due to the property of machine learning, it is not possible to confirm which algorithm was classified, and it can be a decisive obstacle to application when considering the domestic situation in which emergency medical fees are set based on the results of KTAS classification. Similar to our study, some studies have classified patients using physiological data. However, these studies included small number of samples; and although they were effective in detecting patient deterioration, there was no significant benefit of use as a triage system.3132

S-OTAS, on the other hand, does not require an expert for triage; therefore, there is no cost or time spent on expert training and retraining. Also, since subjective factors are excluded, there would be no discrepancy between the evaluators or institutions. In addition, the basis for classification can be clarified, so it would be reasonable to apply it to the current Korean insurance system. Furthermore, in connection with the electronic medical records, the triage could be performed and displayed immediately by simply inputting the main symptom, vital signs, and level of consciousness without a separate procedure. Even when there are changes in vital signs or consciousness after entering the ED, the re-triage could be accomplished immediately by simply recording some variables in the electronic medical records.

In addition, S-OTAS showed a greater difference in AUC in 30-day mortality, 7-day mortality, CT scan, and emergency procedure compared to KTAS. Also, in the distribution of 30-day mortality, 7-day mortality, and ED mortality, S-OTAS showed the highest rate at level 1 and the rate decreased sequentially, but KTAS showed the highest at levels 2 and 3. In addition, there were more patients with levels 1 and 2, fewer in level 3, and similar numbers in levels 4 and 5 in S-OTAS compared to KTAS, which means that there were less ambiguous intermediate level 3s in S-OTAS. Regarding ICU admission, AUC was higher in KTAS than in S-OTAS. However, as for the distribution according to triage level, S-OTAS showed the highest rate in level 1 and a sequential decrease like mortality, whereas it was highest in levels 2 and 3 in KTAS. Given all of these, S-OTAS has a comparative advantage over KTAS as a triage tool for distinguishing severely ill patients.

Currently, KTAS recommends ED arrival to physician intervals as immediate, 10 minutes, 30 minutes, 60 minutes, and 120 minutes according to KTAS levels 1, 2, 3, 4, and 5, respectively. Compared with KTAS, the number of level 1 patients was about 5.2 times more and the number of level 2 patients was 2.4 times more in S-OTAS. Therefore, it might be difficult to treat S-OTAS level 1 patients immediately and treat S-OTAS level 2 patients within 10 minutes according to the current KTAS standards. However, the data analyzed in this study were from three university hospitals. Therefore, 5077 S-OTAS level 1 patients means there were 4–5 patients per day in each hospital, and 12977 S-OTAS level 2 patients means 11–12 patients per day in each hospital. Considering this, it might be possible to apply the current KTAS standard to S-OTAS. Additionally, if the physician fails to examine the patient within the recommended time, KTAS recommends continuous monitoring, reassessment every 10 minutes, 30 minutes, 60 minutes, and 120 minutes for the patients in the waiting room according to KTAS levels 1, 2, 3, 4, and 5, respectively. These recommendations can be also used in S-OTAS. Nevertheless, the conclusion could not be made regarding the ED arrival to physician intervals or reassess time in accordance with the S-OTAS level in this study. Further studies are needed to investigate these factors.

In the present study, age was an important factor in triage. To determine whether a strong influence of age, not due to objectification, influenced the outcome, the simple age-adjusted Korean Triage and Acuity Scale (S-KTAS) reflecting age in KTAS was made and compared with KTAS and S-OTAS. S-KTAS showed better classification results compared to KTAS, but the AUC for 30-day mortality was still higher in S-OTAS than in SKTAS (Supplementary Figs. 1 and 2, only online). In addition, LR-OTAS, which reflects the influence of age using logistic regression, showed better classification results compared to S-OTAS. However, there was a limit that could not be applied to the currently used five-level triage tool.

KTAS, which was used as a basic framework when developing S-OTAS, was developed based on CTAS. CTAS is being used not only in Canada but also in many other countries, including Saudi Arabia, Costa Rica, Hungary, Barbados, Trinidad, and Turks and Caicos.3334 In addition, several triage scales, such as the Japan Triage Acuity Scale and the Taiwan Triage Acuity Scale, have been developed based on CTAS.93435 Given those, objectification methods in this research could be applied to CTAS and other CTAS-based triage tools as well as KTAS.

This study had several limitations. First, many of the ED patients were not eligible due to missing values. In particular, 76190 patients accounted for most of the cases in which the oxygen saturation value was omitted among vital signs. This was because pulse oximetry was not applied to the patients who did not have any symptoms related to the respiration; however, there could be severely ill patients whose saturation was not detected. Considering the triage distribution results, it is possible that many patients with mild symptoms were excluded, which may be the cause of the increased proportion of patients with KTAS levels 1, 2, and 3.

Second, although data from two urban hospitals and one rural hospital were used, it may be difficult to apply the results of this study to other hospitals. In addition, since this study was conducted in three academic hospitals, the ratio of KTAS levels 1, 2, and 3 was about 71% (36450/44402 patients). This was similar to other studies conducted in academic hospitals.63637 However, it was different from the 44% of studies conducted nationwide.3839 Therefore, the results of this study can be interpreted as S-OTAS better reflecting the severity of patients with KTAS levels 1, 2, and 3.

Third, patients who were discharged with improved condition were assumed to survive in this study. However, there might be patients who died after discharge despite showing improvement before being discharged.

Fourth, in the results of the discrepancy between S-OTAS and KTAS, 878 (89.4%) of 982 patients with KTAS level 1 were S-OTAS level 1, whereas 878 (17.3%) of 5077 patients with S-OTAS level 1 had KTAS level 1, suggesting the possibility of overtriage, which may have increased the AUC value of S-OTAS for mortality.

Last, in the depression/suicidal or deliberate self-harm category, modifiers that depend on the subjective evaluation of the medical staff such as clear plan, active suicide intent, uncertain flight, and safety risk could not be reflected in S-OTAS.

In conclusion, in adult patients visiting the ED, the 30-day mortality AUC of S-OTAS was 0.812 and the 7-day mortality AUC was 0.846, which were significantly higher than those of KTAS (0.751; p<0.001, 0.783; p<0.001, respectively).