The Accuracy of Rapid Emergency Medicine Score in Predicting Mortality in Non-Surgical Patients: A Systematic Review and Meta-Analysis.

Background: Emergency department (ED) physicians often need to quickly assess patients and determine vital signs to prioritize them by the severity of their condition and make optimal treatment decisions. Effective triage requires optimal scoring systems to accelerate and positively influence the treatment of trauma cases. To this end, a variety of scoring systems have been developed to enable rapid assessment of ED patients. The present systematic review and meta-analysis aimed to investigate the accuracy of the rapid emergency medicine score (REMS) system in predicting the mortality rate in non-surgical ED patients.
Methods: A systematic search of articles published between 1990 and 2020 was conducted using various scientific databases (Medline, Embase, Scopus, Web of Science, ProQuest, Cochrane Library, IranDOC, Magiran, and Scientific Information Database). Both cross-sectional and cohort studies assessing the REMS system to predict mortality in ED settings were considered. Two reviewers appraised the selected articles independently using the National Institutes of Health (NIH) quality assessment tool. The random-effects model was used for meta-analysis. I2 index and Q statistic were used to examine heterogeneity between the articles.
Results: The search resulted in 1,310 hits from which, 29 articles were eventually selected. Out of these, for 25 articles, the area under the curve value of REMS ranged from 0.52 to 0.986. The predictive power of REMS for the in-hospital mortality rate was high in 19 articles (67.85%) and low in nine articles (32.15%).
Conclusion: The results showed that the REMS system is an effective tool to predict mortality in non-surgical patients presented to the ED. However, further evidence using high-quality design studies is required to substantiate our findings. Copyright: © Iranian Journal of Medical Sciences.

What’s Known

Previous studies have shown that the rapid emergency medicine score (REMS) system could be a valuable predictor of long-term mortality in non-surgical emergency department (ED) patients.

REMS is reported to have good prognostic potential (AUC=0.815) to predict hospital mortality in severely injured patients.

What’s New

Results of our systematic review showed that most of the included studies confirmed the REMS system as an effective tool to predict mortality in ED patients.

REMS is recommended as a valuable tool to predict in-hospital mortality in non-surgical patients admitted to the ED.

Introduction

The emergency department (ED) plays a pivotal role in managing complex and acute patients. Triage in ED focuses on effective patient flow management, providing appropriate care, and preventing unnecessary interventions to improve medical outcome. Emergency physicians often need to quickly assess patients, determine vital signs for prioritization, and make optimal decisions. Effective triage requires optimal scoring systems to accelerate treatment and positively influence treatment outcomes.

During the past decades, a variety of scoring systems have been developed to assess patients upon admission. The core element in these systems is an objective assessment of disease severity based on deviations in various physiological variables. More recently, researchers such as Nguyen and Hyzy have developed new scoring systems for critically ill trauma patients. However, none of these systems are dedicated to non-surgical ED patients. The Acute Physiology and Chronic Health Evaluation II (APACHE II) system has been developed based on 12 physiological variables for use in the intensive care unit (ICU). However, APACHE II cannot be applied to ED patients due to the use of biochemical parameters. The Rapid Acute Physiology Score (RAPS), a shortened version of APACHE II, is one of the most appropriate scoring systems used in ED. It evaluates physiological parameters such as blood pressure, respiratory rate, pulse rate, and Glasgow coma scale (GCS). RAPS is further improved by including oxygen saturation and patient age, introducing a new system known as rapid emergency medicine score (REMS). The benefit of these additions is that oxygen saturation can be easily measured in the ED, and age is an independent risk factor for severe diseases and mortality. A previous study showed that REMS is a powerful predictor of patient outcomes in the ED versus other scoring systems. Another study reported that REMS could be a valuable predictor of long-term mortality in non-surgical ED patients. In contrast, Söyüncü and Bektaş indicated that other scoring systems are more reliable than REMS. Due to the lack of comprehensive data on the prognostic value of scoring systems, we performed a systematic review of the literature and meta-analysis to investigate the accuracy of REMS in predicting the mortality rate in non-surgical ED patients.

Materials and Methods

The study was approved by the Local Ethics Committee (code: IR.TBZMED.VCR.REC.1399.003). We conducted a systematic search from 1990 to 2020 using Medline (Ovid, PubMed), Embase, Scopus, Web of Science, ProQuest, and Cochrane Library. We also searched Iranian databases such as IranDOC, Magiran, and Scientific Information Database (SID). The search strategy included a combination of MeSH terms and free-text such as REMS, rapid emergency medicine score, rapid emergency medical score, and mortality (appendix 1-3). PICO (population, interventions, comparisons, outcomes) components were respectively non-surgical patients referred to ED, rapid emergency medicine score, other scoring systems, and mortality.

All identified citations were collated and uploaded into EndNote X9 (Clarivate Analytics, USA) followed by the exclusion of duplicate citations. Then, titles and abstracts were independently screened by two reviewers. The full texts of the screened articles were retrieved and assessed in detail. Inclusion criteria were using REMS as a predictive tool for mortality, studies conducted in ED, cross-sectional and cohort studies, non-surgical patients, and articles in English or Persian. Exclusion criteria were articles published before 1990, the use of languages other than English or Persian, and studies with patients discharged from ED or admitted to ED with cardiac arrest. The assessment was performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA). The quality of eligible articles was determined using the National Institutes of Health (NIH) quality assessment tool for observational cohort and cross-sectional studies. Disagreements between reviewers were resolved through discussion until consensus was reached.

Eligible articles were appraised independently by two reviewers for methodological quality using standard critical appraisal tools. Disagreements between reviewers were resolved through mutual discussion. Following a critical appraisal, based on the degree of study bias, articles not fulfilling the quality threshold (i.e., meeting at least two items from the checklist) were excluded. The extracted data from the selected articles were the name of first author, publication year, country, setting, type of study, sample size, age, sex, admission reasons, study period (months), length of hospital stay (days), number of deceased patients, REMS score for survivors and non-survivors, the area under the curve (AUC) value of REMS, and the predictive power of REMS.

Statistical Analysis

The data were analyzed using Comprehensive Meta-Analysis software, version 3.0 (BioStat Inc., USA). The random-effects model was used for meta-analysis. I2 index and Q statistic were used to examine heterogeneity between the articles. Subgroup analysis was conducted based on the age of patients. P values less than 0.05 were considered statistically significant.

Results

The search resulted in 1,310 hits, of which 497 duplicate articles were removed. From the remaining 813 articles, those that did not meet the inclusion criteria (n=755) were removed. The full texts of the remaining 58 were assessed for eligibility, resulting in the exclusion of a further 29 articles because of non-original type of research, different study settings, or using REMS for assessing patients for procedures other than non-surgical approaches. Subsequently, a total of 29 studies were included in our systematic review. As depicted in figure 1, the selection process was in accordance with the PRISMA checklist. Of the 29 included articles, eight were cross-sectional and 21 were cohort studies. A total of 550,966 patients were included in this study of which 324775 (58.95%), 226,191 (41.05%) were men and women, respectively. The mean age of the patients was 49.13 years (range: 6.2-90.8 years). The reported setting was ED and the patients were admitted because of sepsis, injuries, vibrio vulnificus infection, splenic abscess, hepatic portal venous gas; severe fever with thrombocytopenia syndrome, trauma, S. aureus bacteremia or other suspected infections, febrile; non-surgical, acute coronary syndrome, or internal diseases. The average study duration was 27.04±1.0 months (range: 5-183 months). More than 50% of the studies reported an average hospital stay of about six days. Most studies reported the number of deceased patients with an average mortality rate of 7.95% (table 1).

Figure 1

The search strategy for the systematic review is illustrated according to the PRISMA guidelines.

Table 1

Detailed characteristics of included articles retrieved from the data extraction form

Author	Publication year	Country	Setting	Type of study	Sample size (n)		Average age	Admission reasons	Study period (month)	Length of hospital stay (day)	Number of deceased patients	REMS score for survivors	REMS score for non-survivors	AUC of REMS	Predictive power of REMS
					Male	Female
Alter 21	2017	USA	A county-based advanced life support EMS agency	Cohort	28,035	33,311	51.9		12			4.3			High
Brabrand 22	2017	Denmark	Hospital	Cohort	2,917	2,867	67		5	1 (median)	193			0.77	Low
Bulut 23	2014	Turkey	Hospital	Cohort	1,039	961	61.41±18.92		6		153	5	7	0.589	High
Cardenete-Reyes 13	2017	Spain	Hospital	Cross-sectional	67	37	60.25±11.06	Acute coronary syndrome	12						High
Carugati 24	2018	Tanzania	Hospital	Cohort				Febrile	11		44			0.52	Low
Cattermole 14	2009	Hong Kong	Hospital	Cross-sectional	195	135	61.3±20.6				12			0.771	Low
Crowe 25	2010	USA	Hospital	Cohort	108	108		Severe sepsis or septic shock	13	1-45		10	11	0.62	Low
Dundar 15	2015	Turkey	Hospital	Cross-sectional	507	432	71 (median)	Geriatric patients	12		73	1	5	0.833	High
Ghanem-Zoubi 26	2011	Israel	Community based hospital	Cohort	582	490	74.7±16.1	Sepsis	15	8.77	387	8.4	11.9	0.77	High
Gok 16	2018	Turkey	Hospital	Cross-sectional	144	106	57.60±20.82	Internal diseases, surgery, and trauma	24					0.703	Low
Goodacre 27	2006	UK	Hospital	Cohort	3,222	2,361	63.4		55		744		8.4	0.74	High
Ha 17	2015	Vietnam	Hospital	Cross-sectional	806	940				7 (median)	172	6	9	0.712	High
Hilderink 28	2015	Netherlands	Hospital	Cohort	296	304	64.6	Sepsis	12		75			0.78	Low
Howell 29	2007	Israel	Tertiary care hospital	Cohort	1,020	1,112	61	Suspected infection	10		83	5	10	0.80	High
Hung 1	2017	Taiwan	Hospital	Cohort	77	37	56.33±16.12	Splenic abscess	183			0.10	0.16	0.67	Low
Imhoff 30	2014	USA	Level one trauma center	Cohort	2,718	962	36.5	Trauma	48	7.6	191	3.4	11.8	0.91	High
Kuo 31	2013	Taiwan	Hospital	Cohort	96	75	63.1±12.3	Vibrio vulnificus infection		16.8±14.6 (mean±SD)	43	5.4±2.3	9.7±2.6	0.895	High
Miller 32	2017	USA	Level one trauma center	Cohort	263,957	165,656		Blunt and/or penetrating injuries		5.2 (mean)	3,382	2.9	17.7	0.967	High
Nakhjavan-Shahraki 18	2017	Iran	Hospital	Cross-sectional	1,623	525	39.50±17.27	Trauma			123			0.92	High
Nakhjavan-Shahraki 19	2017	Iran	Hospital	Cross-sectional	605	209	11.65±5.36	Trauma	6		26			0.986	High
Olsson 33	2003	Sweden	Hospital	Cohort	513	513	70±18.1			5	116			0.911	High
Olsson 9	2004	Sweden	Hospital	Cohort	5,663	6,087	61.9±20.7	Non-surgical disorders	12	3.2					Predictor of long-term mortality
Olsson 7	2004	Sweden	Hospital	Cohort	5,663	6,087	61.9±20.7	Non-surgical disorders	12	3.2	285	5.5	10.5	0.852	High
Park 34	2017	South Korea	Hospital	Cohort	4,298	2,607	57.42±18.51	Trauma	60	24.95	212	4.31	9.71	0.9	High
Polita 35	2014	Brazil	Hospital	Cohort	131	32	38±18	Trauma	5		17	4.9		0.761	Low
Seak 36	2017	Taiwan	Hospital	Cohort	36	30	69.23±16.64	Hepatic portal venous gas			38	6.86	14.21	0.9286	High
Sharma 37	2013	USA	Tertiary care community hospital	Cohort	241		56.95±17.62	S. aureus bacteremia	17		55	5.24	9.58	0.806	High
Yang 38	2017	China	Hospital	Cohort	62	61	59±12	Severe fever with thrombocytopenia syndrome	38		31	8.55	12.45	0.746	Low
Ala 20	2020	Iran	Hospital	Cross-sectional	154	146	59.21±19.86	Non-surgical disorders		30	40				High

REMS: Rapid emergency medicine score; AUC: Area under the curve

Methodological Quality Assessment

The quality of the included articles was assessed by two reviewers independently using the NIH quality assessment tool for observational cohort and cross-sectional studies. All articles were judged to be fair or good (table 2). Since most of the articles used secondary data and were retrospective studies, three questions in the Critical Appraisal Skills Programme (CASP) checklist (numbers 8, 10, and 12) were deemed not applicable and therefore omitted (table 3).

Table 2

The quality rating of included articles using the National Institutes of Health quality assessment tool for observational cohort and cross-sectional studies

No.	Author	Publication year	Quality rating (reviewer 1)	Quality rating (reviewer 2)
1	Alter 21	2017	Good	Good
2	Brabrand 22	2017	Fair	Fair
3	Bulut 23	2014	Fair	Fair
4	Cattermole 14	2009	Fair	Fair
5	Carugati24	2018	Good	Good
6	Crowe 25	2010	Good	Good
7	Dundar 15	2015	Fair	Fair
8	Ghanem-Zoubi 26	2011	Fair	Fair
9	Gok 16	2018	Fair	Fair
10	Goodacre 27	2006	Fair	Fair
11	Ha 17	2015	Fair	Fair
12	Hilderink 28	2015	Fair	Fair
13	Howell 29	2007	Fair	Fair
14	Hung1	2017	Fair	Fair
15	Imhoff 30	2014	Fair	Fair
16	Kuo 31	2013	Good	Good
17	Miller 32	2017	Fair	Fair
18	Nakhjavan-Shahraki 18	2017	Fair	Fair
19	Nakhjavan-Shahraki 19	2017	Fair	Fair
20	Olsson 33	2003	Fair	Fair
21	Olsson 9	2004	Fair	Fair
22	Olsson 7	2004	Fair	Fair
23	Park 34	2017	Fair	Fair
24	Polita 35	2014	Fair	Fair
25	Cardenete-Reyes 13	2017	Fair	Fair
26	Seak 36	2017	Fair	Fair
27	Sharma 37	2013	Good	Good
28	Yang 38	2017	Good	Good
29	Ala 20	2020	Good	Good

Table 3

Methodological quality assessment of included articles using the Critical Appraisal Skills Programme (CASP) checklist

Author	1: Objective	2: Population definition	3: Participation rate	4: Selection criteria	5: Sample size	6: Exposure assessment	7: Timeframe	9: Exposure measures	11: Outcome measures	13: Loss to follow up	14: Statistical analysis
Alter 21	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Brabrand 22	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Bulut 23	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Cattermole 14	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Carugati 24	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Crowe 25	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Dundar 15	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Ghanem-Zoubi 26	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Gok 16	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Goodacre 27	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Ha 17	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	Yes	Yes
Hilderink 28	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Howell 29	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Hung 1	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Imhoff 30	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Kuo 31	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Miller 32	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Nakhjavan-Shahraki 18	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Nakhjavan-Shahraki 19	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Olsson, et al 33	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Olsson 9	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Olsson 7	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Park 34	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Polita 35	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Cardenete-Reyes 13	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Seak 36	Yes	Yes	CD	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Sharma 37	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Yang 38	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	CD	Yes
Ala 20	Yes	Yes	CD	Yes	No	Yes	yes	No	No	CD	Yes

CD: Could not be determined

Predictive Power of REMS

Almost all articles reported the average REMS score for survivors (5.10) and non-survivors (9.88). Except for four articles, average AUC values (0.79; range: 0.52-0.986) were reported. In these articles, REMS was considered independently or in comparison with other scoring systems. The predictive power of REMS for in-hospital mortality rate was high in 19 articles (67.85%) and low in nine articles (32.15%). Only one study reported that REMS was a good predictor of long-term mortality (4.7 years).

Meta-analysis

Twenty-two articles reported the percentage of mortality by surveying 477,186 ED cases. Publication bias was assessed using the funnel plot and Egger’s regression test. The results showed that diffusion between the articles was not statistically significant (t=0.59, df=20, P=0.281). Furthermore, the funnel plot showed symmetry between the articles (figure 2). Heterogeneity between the articles was significant (Q=11,340.14, df=21, I2=99.81, P<0.001), and the percentage of mortality was 8.69% (pooled death=0.0869, 95% CI: 4.50-16.11, P<0.001). The forest plot of the result of our meta-analysis is shown in figure 3.

Figure 2

Funnel plot illustrates bias in the results of the meta-analysis.

Figure 3

Forest plot depicted the mortality rates, which is extracted from the reviewed articles.

Subgroups Analysis

Subgroup analysis was conducted based on the age of the patients and the predictive power of REMS (table 4, figures 4 and 5). The results showed that the mortality rate in patients under versus above 60 years was 8.5% and 10.44%, respectively. Moreover, studies that evaluated the predictive power of REMS reported high and low levels of mortality rates at 8.72% and 8.59%, respectively.

Table 4

Tabular presentation of the results of subgroups analysis

Subgroup		Effect size and 95% interval				Null hypothesis		Heterogeneity
		Number Studies	Proportion of patient deaths	Lower limit	Upper limit	Z-value	P value	Q-value	df	P value	I2
Predictive power of REMS	High	17	0.0872	0.0407	0.1771	-5.67	<0.001	10,881.68	16	<0.001	99.85
	Low	5	0.0859	0.0204	0.2973	-3.08	0.002	184.62	4	<0.001	97.83
Age	≤60	8	0.0857	0.0398	0.1749	-5.69	<0.001	308.67	7	<0.001	97.73
	>60	12	0.1044	0.0566	0.1847	-6.34	<0.001	1,856.35	11	<0.001	99.41

Figure 4

Forest plot depicted the reported mortality rates in patients aged below and above 60 years.

Figure 5

Forest plot indicated the reported mortality rates in terms of the high and low predictive power of REMS.

Discussion

The results of the present systematic review showed that the AUC value of REMS was 0.79. The majority of the included studies (67.85%) reported that the REMS system has a high or good predictive value for mortality. In contrast, a previous study reported the lack of sufficient evidence to conclude on the accuracy of prognostic models in patients with suspected infection admitted to the ED. The results of another systematic review aimed at validating 10 different scoring systems, including REMS, reported that none of the systems could accurately predict the risk of in-hospital mortality and admission to the ICU. However, they found that REMS had an acceptable discriminatory power but poor calibration.

In the present study, we mainly focused on the ED setting, whereas some other review studies focused on other healthcare settings. Nonetheless, their findings on the predictive power of REMS were in line with our study. El-Sarnagawy and Hafez assessed different scoring systems, including REMS, in predicting the need for mechanical ventilation in patients with a drug overdose. They reported that REMS had a 100% positive predictive value and recommended this scoring system as an appropriate tool. In contrast with our study, Yu and colleagues compared REMS with other scoring systems in terms of its predictive ability to detect clinical deterioration in non-ICU patients diagnosed with an infection. They measured each score serially to characterize how these scores changed with time. They reported that REMS had an AUC value of 0.70 and lacked adequate predictive value that other systems. Ji and colleagues conducted a study in the ED and coronary care unit (CCU) of a hospital and showed that REMS did not have adequate predictive value for short-term risk of death in patients with acute myocardial infarction (AMI). After comparing REMS with Global Registry of Acute Coronary Events (GRACE) and APACHE II risk scores, they reported that the AUC value of REMS for predicting mortality in AMI patients within 30 days was 0.615. In the present study, the average AUC value of REMS for non-surgical patients was 0.79, which is an acceptable predictive value.

We also compared the findings of the studies included in our systematic review with the results of other studies. One of the included articles reported that REMS was a good predictor of long-term in-hospital mortality (4.7 years). Similarly, Olsson and colleagues showed that while REMS can be a predictor of long-term mortality, it cannot independently predict short-term (three-day, seven-day) mortality in non-surgical ED patients. Seven studies in our systematic review were conducted in traumatic patients, five of which reported that REMS could accurately predict in-hospital mortality. Lee and colleagues also reported that REMS had a good prognostic ability (AUC=0.815) to predict hospital mortality in severely injured patients. Although most of the studies in our systematic review assessed patients with infectious diseases, the reported overall AUC>0.70 was in line with other studies conducted on traumatic patients. Furthermore, three studies were conducted on ED patients diagnosed with sepsis in the ED setting. Among these, one study reported the high predictive power of REMS for in-hospital mortality. In line with our findings, another study reported that REMS had a good prognostic ability (AUC=0.72) to predict mortality in adult ED patients diagnosed with sepsis.

In the present systematic review, we selected studies that specifically focused on non-surgical patients. It is recommended that future studies include other categories of patients to further confirm the high prognostic ability of REMS to predict mortality. The main limitation of our systematic review was related to poor quality or lack of access to the full text of some of the selected articles, as well as the exclusion of studies published in languages other than English and Persian.

Conclusion

The results of the present systematic review and meta-analysis showed that the REMS system is an effective tool to predict hospital mortality in non-surgical patients admitted to ED. The use of the REMS system is recommended in ED to predict mortality and serve as a basis for developing an efficient care plan. However, further evidence using high-quality design studies is required to substantiate our findings.

Acknowledgement

The authors would like to express their gratitude for the financial support from the Vice-Chancellor for Research of Tabriz University of Medical Sciences, Tabriz, Iran.

Authors’ Contribution

A.Gh: Acquisition and analysis of data, Drafting and critical revision of the manuscript for important intellectual content; N.V: Systematic search, analysis of data, Drafting and critical revision of the manuscript for important intellectual content; S.Sh.V: Study design, Critical reviews, Drafting of the manuscript; A.A: Study concept and design, Drafting and critical revision of the manuscript for important intellectual content; M.J: Study concept and design, Critical reviews, Acquisition of Data, Drafting and critical revision of the manuscript for important intellectual content; All authors have read and approved the final manuscript and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Conflict of Interest: None declared.