The effectiveness of a national early warning score as a triage tool for activating a rapid response system in an outpatient setting: A retrospective cohort study.

Rapid response system (RRS) efficacy and national early warning score (NEWS) performances have largely been reported in inpatient settings, with few such reports undertaken in outpatient settings.This study aimed to investigate NEWS validity in predicting poor clinical outcomes among outpatients who had activated the RRS using single-parameter criteria.A single-center retrospective cohort studyFrom April 1, 2014 to November 30, 2017 in an urban 350-bed referral hospital in JapanWe collected patient characteristics such as activation triggers, interventions, arrival times, dispositions, final diagnoses, and patient outcomes. Poor clinical outcomes were defined as unplanned intensive care unit transfers or deaths within 24 hours. Correlations between the NEWS and clinical outcomes at the time of deterioration and disposition were analyzed.Among 31 outpatients, the NEWS value decreased significantly after a medical emergency team intervention (median, 8 vs 4, P < .001). The difference in the NEWS at the time of deterioration and at disposition was significantly less in patients with poor clinical outcomes (median 3 vs 1.5, P = .03). The area under the curve (AUC) for the NEWS high-risk patient group at the time of deterioration for predicting hospital admission was 0.85 (95% confidence interval [CI], 0.67-1.0), while the AUC for the NEWS high-risk patient group at disposition for predicting poor clinical outcomes was 0.83 (95% CI, 0.62-1.0).The difference between the NEWS at the time of deterioration and at disposition might usefully predict admissions and poor clinical outcomes in RRS outpatient settings.

Introduction

In recent decades, the rapid response system (RRS) for timely identification and treatment of clinically deteriorating patients has been adopted globally.[,] Appropriate use of the RRS has been reported to decrease the incidence of cardiac arrest and mortality in inpatient settings outside of the intensive care unit (ICU).[,,] However, failure of healthcare staff to recognize or respond to patient deterioration and a delay in RRS activation have both been associated with increased hospital mortality, morbidity, and length of stay.

In the United States and in Australia, various RRS-activation criteria have been developed. Most criteria have been based on extremely abnormal specific vital sign values (eg, pulse rate, <40 or >120 beats/min). The standard single parameter for RRS activation includes ≥1 vital sign-based criteria (a single parameter criteria) in addition to staff concerns for patient care. Ideal RRS activation criteria require the highest discrimination in relation to patient outcome (“high sensitivity”) and the lowest trigger rate (“high specificity”), thereby minimizing the risk of overlooking the likelihood of serious outcomes and excessive staff workload.

Recently, the early warning score system (EWSS) has received attention as an accurate prediction tool to activate the RRS in situations of clinical deterioration. The EWSS was designed to allocate points for patients’ multiple vital signs in a quantitative manner and assess the sum of each point. In 2012, the Royal College of Physicians recommended standardizing the EWSS for the National Health Service, which became known as the national EWS (NEWS).

A recent systematic review showed that the NEWS could be a useful tool for predicting unplanned ICU transfer, cardiac arrest, and short-term mortality within 48 hours, in an inpatient setting.[,] In the Netherlands in 2015, the introduction of a nationwide implementation of the RRS and a modified early warning score resulted in a reduction in the composite endpoint of cardiopulmonary arrests, unplanned ICU admissions, and mortality of patients in general wards. The NEWS has been validated in both emergency room (ER) and prehospital settings, and has been shown to predict important clinical outcomes effectively.[,,,]

RRS efficacy and NEWS performance have largely been reported in inpatient settings, with very few reports concerning the effects of the RRS in an outpatient setting.[,,,] Therefore, we investigated an RRS-activated single parameter criteria outpatient population, and hypothesized that the NEWS could predict poor clinical outcomes.

Methods

This study was undertaken from April 1, 2014 to December 31, 2017 at a 350-bed community hospital with a 14-bed ICU, a 12-bed high dependency unit (HDU), a general ward, an ER, and an outpatient clinic. Our medical emergency team (MET) calling criteria (see Table, Supplemental Content1, which described MET calling criteria of our institution.) is based on the University of Pittsburgh Medical Center “Condition C” calling criteria. We reviewed MET records, and cases from the outpatient setting were identified. Patients who activated the MET in an outpatient setting were included. RRS-activated outpatients were evaluated by an emergency physician-led MET, and transferred to the ER. This study was approved by the Institutional Review Board.

Patient age and sex, and physiological measurements including respiratory rate, oxygen saturation, body temperature, systolic blood pressure, heart rate, and the level of consciousness graded using the alert; verbal, voice response present; pain, pain response present; unresponsive scale, as well as the activation trigger, intervention performed, time of arrival, final diagnosis, and disposition were recorded. Outcomes were unplanned ICU transfer, as well as 24-hour, 30-day, and 90-day mortality rates. We defined poor clinical outcomes as unplanned ICU transfers or death within 24 hours. The NEWS was calculated as reported by the Royal College of Physicians. The aggregated NEWS was categorized into 3 groups, as follows: a low-risk group (NEWS 0–4), a medium-risk group (NEWS 5–6), and a high-risk group (NEWS ≥7), according to NEWS threshold criteria. The data used for calculating the NEWS were obtained from patient vital signs recorded on 2 occasions, namely, at the time of deterioration and at disposition.

For data analysis, R (The R Foundation for Statistical Computing, Vienna, Austria, version 3.3.3) and EZR (Saitama Medical Center, Jichi Medical University, Saitama, Japan, version 1.3.5) were used. EZR is a graphical user interface for R. More precisely, it is a modified version of R commander designed to add statistical functions frequently used in biostatistics.

A Mann–Whitney U test was used for statistical comparison of continuous variables, and the Wilcoxon rank-sum test was used for comparison between the NEWS at the time of deterioration and the NEWS at disposition. A Fisher exact test was used to analyze data involving categorical variables.

Results

There were 31 (males, 23; females, 8) outpatients for whom the RRS had been activated during the study period. All patients were transferred to the ER and were evaluated by an emergency physician-led MET, according to our hospital RRS protocol. Patient characteristics are shown in Table 1. The median age was 68 years (interquartile range [IQR], 51–78 years). The median time interval from RRS activation to the arrival of the MET was 5 minutes (IQR, 2–5 minutes). Patient details including diagnosis, NEWS, and disposition were shown (see Table, Supplemental Content 2, that illustrated the patient list). The NEWS at the time of deterioration and at disposition is shown in Figures 1 and 2. The NEWS decreased significantly following the MET intervention (median score, 8 [IQR, 6–8] vs 4 [IQR, 2–6]; P < .001). The NEWS percentages in the low-risk, medium-risk, and high-risk groups were 19%, 15%, and 67% at the time of deterioration, respectively, and 56%, 30%, and 15% at disposition, respectively. Forty-two percent (13/31), 19% (6/31), and 13% (4/31) of the patients were admitted to a general ward, HDU, or ICU, respectively. Twenty-four-hour, 30-day, and 90-day mortality rates were 3%, 4%, and 13%, respectively. The leading triggers for MET activation were altered mental status (53%), tachypnea (40%), and hypoxemia (33%) (Table 1). Administration of intravenous fluid (87%) and oxygen (43%) were the most common interventions performed.

Table 1

Characteristics of outpatient RRS cases (n = 31).

Figure 1

NEWS value based on admission status. X axis: Admission status and time point, Y axis: NEWS value. National early warning score calculated at the time of deterioration, and at the disposition based on admission status (no admission vs admission). Boxes comprise 25th to 75th percentiles with median values shown, whiskers display 10th and 90th percentile. NEWS = national early warning score.

Figure 2

NEWS value based on poor outcome. X axis: Poor outcome and time point, Y axis: NEWS value. National early warning score calculated at the time of deterioration and at the disposition based on poor clinical outcome. Boxes comprise 25th to 75th percentiles with median values shown, whiskers display 10th and 90th percentile. NEWS = national early warning score.

The NEWS calculated at the time of deterioration and at disposition, based on admission status (admission vs discharge), is shown in Figure 1. The NEWS for patients who required admission at the time of deterioration and at disposition was significantly higher than for patients who were discharged (median score, 8 [IQR, 8–8] vs 4 [IQR, 4–6.25], P = .008; and median score, 5 [IQR, 2.25–6] vs 2 [IQR, 1–3], P = .04, respectively). The difference in the NEWS taken at the time of deterioration compared to disposition was not significantly different based on admission status (median score, 3 [IQR, 3–4] vs 2 [IQR, 1–4], P = .49). The NEWS at the time of deterioration and at disposition, based on short-term poor clinical outcomes (an unplanned ICU transfer or death within 24 hours) is shown in Figure 2. The NEWS of the patients requiring ICU admission or who died within 24 hours of disposition was significantly higher than that of patients who did not require ICU admission or die within 24 hours (median score, 6.5 [IQR, 5.5–8.75] vs 4, [IQR, 2–5] P = .04). The difference in the NEWS at the time of deterioration and at disposition was significantly lower among patients with a poor clinical outcome (median score, 3 [IQR, 3–4] vs 1.5 [IQR, 0.75–2]; P = .03).

Table 2 shows the validity of NEWS for clinical outcomes. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the curve (AUC) of the NEWS in the high-risk group at the time of deterioration for predicting hospital admissions were 76 (95% confidence interval [CI], 53–92); 67 (95% CI, 22–96); 89 (95% CI, 65–99); 44 (95% CI, 14–78); and 0.85 (95% CI, 0.67–1.0), respectively. The sensitivity, specificity, PPV, NPV, and AUC of the NEWS for high-risk group at disposition for predicting ICU admission or death within 24 hours were 50 (95% CI, 7–93); 91(95% CI, 72–99); 50 (95% CI, 7–93); 91(95% CI, 72–99); and 0.83 (95% CI, 0.62–1.0), respectively.

Table 2

Validity of NEWS among outpatient RRS cases.

Discussion

This is the first study evaluating the NEWS and mortality rates among RRS-activated outpatients according to single parameter criteria. Our results showed that the NEWS at the time of deterioration was associated with hospital admission, and the NEWS at disposition could predict poor clinical outcomes (cardiac arrest, death, and unplanned ICU admission) within 24 hours (Figs. 1 and 2). Despite the significant improvement in the NEWS after the MET intervention in the total RRS-activated population, little improvement in the NEWS was seen in the poor clinical outcome group (Fig. 2).

The RRS has been adopted internationally to detect impending critically ill patients and expedite timely intervention. Appropriate use of the RRS has been shown to decrease the incidence of cardiac arrest and mortality in an inpatient setting outside the ICU[,,]; however, very little RRS data in relation to an outpatient setting are available.[,,,] RRS activation in an outpatient setting accounted for between 8% and 13% of all RRS activations.[,,,] Seventy-four percent (23/31) of our patients were admitted to hospital, and 13% (4/31) were admitted to the ICU. In a previous report, between 26% and 54% of patients required hospital admission, and between 1% and 18% of patients had been admitted to the ICU.[,,,] We were not able to directly compare the severity of illness in patients using the percentage of admissions or ICU transfers because healthcare systems differ among countries. However, RRS activation and a rapid MET intervention are considered to be essential, even in an outpatient setting, as patients who had activated the RRS included critically ill patients who required ICU transfer, and delayed initial treatment may lead to an increased mortality or length of hospital stay.[,] In terms of interventions, most patients in our study received intravenous fluid and oxygen therapy. Only a few patients received more intensive interventions such as endotracheal intubation and cardiopulmonary resuscitation (Table 1). As in 3 previous studies, the most common MET interventions in an outpatient setting were intravenous fluid and oxygen administration.[,,] Conversely, most of the MET interventions in an inpatient setting occurred at a critical care level. In the MERIT study, 99.8% (2371/2376) of the patients received a critical care level intervention, and 19% (451/2376) of the patients were transferred to the ICU. Regarding activation triggers, the most common symptoms noted among our patients were neurological symptoms (55%). It is essential to rapidly assess patients with an altered mental status to rule out cardiac syncope or orthostatic syncope with hypovolemia. Syncope (including vasovagal reflex) was also a common final diagnosis for 13% of the study patients (Table 1). Similar to our results, syncope and near syncope were also the most common diagnoses (27%–29%) for the study patients in 2 previous outpatient RRS studies.[,] Patients with syncope often have a good prognosis, and 4 patients in our study with syncope were discharged. In our results, mortality rates at 24 hours, 30 days, and 90 days were 3%, 4%, and 13%, respectively. We understand our study is the first to report a long-term prognosis of RRS-activated outpatients.

The NEWS is based on the EWSS, and has been used in Europe and more widely in recent years. The NEWS is an aggregate scoring system in which scores between 0 and 3 points are allocated for each of 7 physiological measurements, with a total worst score of 20 points. The Royal College of Physicians has recommended that medium-risk group patients (NEWS 5-6 points) should be observed and monitored closely and assessed urgently by a clinician or team with core competencies in the care of acutely ill patients. It has also been recommended that high-risk patients (NEWS ≥7 points) should be assessed by a team with critical care competencies, including a practitioner with advanced airway management skills, and considered for transfer to an HDU or an ICU.

In our study, we evaluated the validity of the NEWS among patients who activated the RRS according to conventional criteria. Compared to conventional RRS-activation criteria based on a single parameter, the NEWS showed sufficient sensitivity and a higher specificity in the inpatient population. In a study that compared any single parameter component of the NEWS (3 points) with an aggregate NEWS (≥5 points) as the threshold for escalation of care, single parameter criteria were shown to increase physicians’ workload by 40% with only a 3% increase in detected adverse outcomes.

Previous studies have evaluated the NEWS only at the time of admission in an inpatient setting.[,] One study evaluated the validity of the NEWS using a different time point in an ER setting. The NEWS at ER arrival, 1 hour after arrival, and at disposition were significantly associated with hospital admission, unplanned ICU transfer, and death (n = 274, P < .001, P < .003, and P < .0001, respectively). In our study, among RRS-activated outpatients according to conventional criteria, in the NEWS high-risk group, scores taken at the time of deterioration could identify patients who required admission; however, it was not possible to determine the most critical patients who required ICU admission or those most likely to die within 24 hours. This was because most of the high-risk patients in our study only had transient reversible diseases, such as syncope, asthma attack, and gastrointestinal bleeding, and their NEWS improved shortly after MET intervention. In our population, patients in the high-risk group according to the NEWS at the time of deterioration accounted for 67% (18/27). However, a previous study reported that patients evaluated as NEWS high-risk comprised 12% (32/274) of the total ER population at ER arrival. The prediction of poor clinical outcomes using the NEWS at disposition was better than the NEWS at the time of deterioration (Table 2). The NEWS at disposition in our study used the same time points as the NEWS at admission in a previous study in an inpatient setting that had been validated and shown to predict important clinical outcomes effectively.[,]

Although the NEWS significantly improved after MET intervention in the total RRS-activated population, little improvement in the NEWS was seen in the poor clinical outcome group (Fig. 2). We suggest that the difference in the NEWS between the time of deterioration and at disposition might be important to predict poor clinical outcomes. To identify the most critically ill patients, we consider it is important to evaluate the responsiveness of treatment after MET intervention using changes in the NEWS.

The disposition of RRS-activated outpatients ranged from discharge home to admission to ICU, and most interventions were simple, rarely needing ICU-level interventions; therefore, we deemed an ER physician-led MET appropriate for RRS-activated outpatients. The NEWS may be useful as part of RRS activation criteria and as a prediction tool to identify the most critically ill patients in an outpatient setting. Further prospective studies to evaluate the validity of the NEWS across an entire outpatient population are needed.

Limitations

There are some limitations in our study. First, this was a single-facility retrospective study and the study population was exclusively comprised of RRS-activated outpatients; therefore, our results cannot be extrapolated to a general outpatient clinic population. A future prospective study is needed that includes all outpatients, and not only RRS-activated outpatients. Second, our patient sample size was small, despite having a long follow-up period. This may have been due to the possibility that not all cases had been examined. In this study, diagnostic quantities had large confidence intervals because of the small sample size. Third, the NEWS could not be determined in 13% of the patients because vital signs were missing from patient records.

Conclusion

In an outpatient setting, introducing the NEWS is likely to be useful for predicting admissions or poor outcomes for RRS-activated outpatients. The NEWS at the time of deterioration is also likely to be effective in identifying patients who require admission. The NEWS at disposition, and the difference between the NEWS at the time of deterioration and at disposition may additionally be important in predicting poor clinical outcomes. Future multicentered studies are needed to evaluate the validity of the NEWS prospectively in a general outpatient setting.

Acknowledgments

We wish to sincerely thank Dr. Osamu Takahashi for his support with the statistical analysis. We also thank Yuko Kudo for her support with data collection.

Author contributions

Conceptualization: Toru Yamada, Shigeki Fujitani.

Data curation: Hsiang-Chin Hsu.

Formal analysis: Jun Ehara.

Investigation: Jun Ehara, Toru Yamada.

Methodology: Jun Ehara, Hsiang-Chin Hsu, Toru Yamada, Yosuke Homma.

Project administration: Jun Ehara, Shigeki Fujitani.

Software: Shigeki Fujitani.

Supervision: Eiji Hiraoka, Shigeki Fujitani.

Validation: Yosuke Homma.

Writing – original draft: Jun Ehara.

Writing – review and editing: Eiji Hiraoka, Hsiang-Chin Hsu, Toru Yamada, Shigeki Fujitani.