Inter-Rater Agreement of Emergency Nurses and Physicians in Emergency Severity Index (ESI) Triage.

INTRODUCTION: Triage is one of the most important systems in patients prioritizing at the time of arrival to hospital. Based on the severity of the injury and the need for treatment, this system manages patients in the least time, which could lead to rotation of patients with high reliability and safety. Currently, the most accepted method for triage is emergency severity index (ESI) system, considered as five-level triage method, too. This method were implemented in Al Zahra Hospital of Isfahan by trained nurses since March to May 2010. This study was aimed to evaluate the accuracy of emergency nursing triage using ESI.
METHODS: This prospective cross sectional study was carried out on 601 patients referred to Al-Zahra hospital of Isfahan through May 2010. The patients' triage level were determined by physicians and nurses separately and the results compared. To define the level of agreement between two groups (inter-rater agreement), the kappa index was evaluated. To specify the association between the time interval of initial triage and patient final status, Chi-Square test was applied using SPSS 18 statistical software.
RESULTS: There was no significant difference between results of nurses and physicians triage (P<0/0001). The agreement level (kappa index) between two groups was 94% (95% CI: 0.931-0.957). Of 601 patients, 44.1% ones were hospitalized at the emergency department, 52.6% discharged and 3.3% died. The average of time interval between nursing triage and physician visit was 9.55 minutes at the level one triage, 21.64 minutes at level two, 26.03 minutes at level three, 26.93 minutes at level four, and 11.70 minutes at level five.
CONCLUSION: It seems that there is an acceptable inter-rater agreement between emergency nurses and physicians regarding patients' triage in terms of ESI system.

Introduction:

The rate of emergency department (ED) visits is increasing and need for accurate and reliable tools is inevitable for classification of patients based on severity of emergency (1-5). Various methods have been designed for this purpose such as Traffic director, Manchester system, Canadian system, spot –check, and comprehensive triage (6-11). Through these methods, a five-level severity index or emergency severity index (ESI) recognized as a valid and accurate system which is not only prioritize patients, but also follow their treatment process for better access to medical facilities and services (12, 13). In comparison with the three-level triage, this system has higher validity and reliability, and successfulness in use of financial, human and time resources (14, 15). ESI first was designed in United States of America in 1990 and now has been recognized as the gold standard of triage in many countries, such as Australia, Canada, and United Kingdom (1, 12, 16-20). Because of simplicity, ease of learning, operational and conceptual approach, this triage method seems to be the most appropriate system for our country (Iran), too. Since the knowledge and experience of nurses are very important in accurate triage, several studies examined the role of their knowledge and awareness in prioritizing of patients (2, 21). ESI triage system was implemented by trained nurses through May 2010 in Alzahra Hospital of Isfahan. In the present study, the accuracy of ESI triage by emergency nurses was evaluated.

Methods:

Six hundred and one patients referred to the ED of Alzahra hospital, Isfahan, Iran, were triaged based on ESI version four recommendation by physicians and trained nurses during March to April 2008. They triaged patients separately while were unaware to the results of each other. The study protocol was approved by the ethics committee of Medical University of Isfahan. A checklist was fulfilled for each patients, contained demographic data (age, sex, date, and cause of admission), nurses and physicians triage results, time interval between the initial triage and first visit by physicians, the outcome of the patient, and the time of discharge from the ED. Hospitalization, discharging from ED and mortality was considered as the outcomes. The sample size was calculated considering to p=0.50, d=0.1, and α=0.05 (n=96). All data were analyzed with chi-square and Pearson correlation tests using SPSS version 18 statistical software. P<0.05 was considered as significant. The weighted kappa index (κ) was used for assessment of Inter-rater reliability between the triage of nurse and physician (17).The weighted kappa with 95% confidence interval was reported. Kappa index less than 0.2, 0.2-0.4, 0.6-0.8 and more than 0.8 were considered as week, moderate, good, and excellent, respectively (22, 23).

Results:

There was no significant difference between Nurses and physicians triage. Total calculated weighted Kappa was 94% (95% CI: 0.931-0.957; p<0.0001). Table 1 shows the percentage of agreement between two groups in different levels of triage. Of 601 patients, 316 (52.6%) were discharged from ED, 265 (44.1%) hospitalized, and 20 (3.3%) died (Table 2). The time interval between triage and first physician visit was 0.0 minutes in level one, 12.64±5.0 in level two, 26.03± 9.6 in level three, 62.93±17.3 in level four, and 110.70 ±26.8 in level five (Table 3 ).

Table 1

The number (%) of agreements between two groups in different levels of emergency severity index (ESI) triage

ESI level	1	2	3	4	5	Total
1	11 (100)	0	0	0	0	11
2	0	138 (95)	3	0	0	141
3	0	6	404 (98)	5	0	415
4	0	0	2	22 (81)	0	24
5	0	0	0	0	10 (100)	10
Total	11	144	409	27	10	601

ESI: Emergency severity index

Table 2

Final outcomes of patients with different levels of triage after 24 hours

Triage levels	Outcome of patients (%)
	Hospitalized		Discharged from ED		Died
1	3 (27.3)		2 (18.2)		6 (54.5)
2	80 (55.6)		54 (37.5)		10 (6.9)
3	175 (42.8)		232 (56.7)		2 (0.5)
4	5 (18.5)		20 (74.1)		2 (7.4)
5	2 (20)		8 (80)		0 (0)
Total	265 (44.1)		316 (52.6)		20 (3.3)

ED: Emergency department

Table 3

The average time interval between initial nursing triage and physician examination (minute)

ESI level	N * (%)	Intervals	Standard **
1 (Resuscitation)	11 (100)	0	0
2 (Emergent)	138 (95)	12.64±5.0	10-15
3 (Urgent)	404 (98)	26.03± 9.6	30-60
4 (Non- Urgent)	22 (81)	62.93±17.3	60-120
5 (Referred)	10 (100)	110.70 ±26.8	120-240

Numbers of agreements between two groups,

Maximum time to point of care (based on Manchester triage system)

Discussion:

Our finding revealed that nurses and physicians triage had more than 90% overlap at all levels of triage indicated the high accuracy of nursing triage.

Several studies had been shown the accuracy of nurse triage decision. For example, Goransson et al demonstrated that registered nurses triage had only 58% agreement with the expected acuity rating. But in this study the wide range (22–89%) of accurate triage clearly was a big limitation (24). Abbasi et al., reported low reliability of nursing triage (21). In other hand, the result was in line with Gorason et al., declared the high accuracy of nursing triage (25). The total agreement level of this study was acceptable based on recommendation of Australian college of emergency medicine, determined the kappa of 0.60 as the minimum acceptable agreement level (2). Similar to our investigation Gorason et al. and Worster et al. achieved the moderate (0.46) and good (0.76) agreements, respectively (26, 27).

Based on previous works there were not any significant relation among accuracy of nursing triage and personal characteristics, attitude, and experience level (24, 28). However, nurses’ knowledge has direct relation with appropriate triage of patients (29). It mandates the improvement of staffs’ knowledge regarding the level of patients’ emergency severity; it could be led to minimize the waiting time, enhance the quality of services, and reduce mortalities (30). Training is a key factor in change and improvement of nurses’ knowledge. In this regards, several new training techniques can be used to teach the triage such as gaming technology, web-based training, and etc. (31) which improve the accuracy of the triage process. The prolonged time interval between initial nursing triage and physician examination lead to harmful delays in achieving timely emergency care (32). In contrast, shortening the duration of time between patient presentation and treatment, may increase levels of patient satisfaction and reduced ED overcrowding (33). It could be concluded that trained nurses are able to perform appropriate triage of ED patients and improve the patients’ safety and satisfaction, consequently.

Conclusion:

It seems that there is an acceptable inter-rater agreement between emergency nurses and physicians regarding patients’ triage in terms of ESI system.