Effectiveness of a Training Course on Accuracy of Triaging of Pediatric Patients.

BACKGROUND: In the context of a new but busy Pediatric Emergency Department, the risk of missing patients who need more emergent care can be reduced by timely and accurate triaging. In the emergency department of King Fahad Armed Forces Hospital, the Canadian Triage and Acuity Scale had already been implemented, including the pediatric version (PaedCTAS). However, a common observation remained that critical patients did not always receive priority with subsequent delays in management. To improve this accuracy, a training course was administered to health care professionals responsible for triaging of pediatric patients. AIM: To determine the effectiveness of a training course on accuracy of triaging of Pediatric Patients.
METHODS: A triage training course was conducted over two months, with patient encounter sheets reviewed before the course for 6 months and after the course for 12 months. Accuracy was calculated by comparing it to level as determined by two pediatric emergency physicians. Also, admission rates were used as a surrogate marker to also determine accuracy.
RESULTS: A total of 31 053 patient sheets were reviewed. There was a considerable improvement in the correct determination of all triage levels, with accuracy ranging from 56.5% to 78.3% before the course, and reaching from 79.1% to 90.8% after the course with a statistically significant difference. Triaging errors still present were mainly in the form of down-triage.
CONCLUSION: Our training course in triage has a significant impact on the accuracy of triaging of ill pediatric patients. Further improvement can be obtained by repeated courses and direct feedback with debriefing sessions on challenges to triage level determination. Copyright:

Introduction

In emergency settings, a key strategy to decrease waiting times, especially for those with critical illnesses, is an effective triage protocol. Initially introduced in the United States in 1950, triaging and its success is currently an essential component of patient care in the emergency department [1], [2]. Many systems have been developed to help prioritise patients. The one we used in this research study was the PaedCTAS (The Paediatric Canadian Triage and AcuityScale), but many others have been validated including the MTS (The Manchester Triage System) and the ATS (Australian Triage Scale) [3]. The challenge in triaging of all patients, but particularly those in the pediatric age group, is the ability to make a quick decision based on a brief encounter. Not only do heart rate and respiratory rate vary according to age, but also increase significantly in response to fever, anxiety and pain [4], [5]. To avoid this pitfall, the CTAS, initially implemented in 1999 with a recent revision in 2014, takes into consideration subjective evaluation in addition to objective assessment, and includes modifiers related to the history and the physiologic status. The acuity levels are assigned from 1 (most urgent) – 5 (least urgent) [6].

Ineffective triaging has been demonstrated to have grave consequences, including prolonged wait times, higher acuity patients deteriorating and avoidable tragic outcomes [7], [8].

In King Fahad Armed Forces Hospital, a tertiary teaching hospital in Saudi Arabia, the Pediatric emergency department serves a large population with a very high rate of visits, which highlighted the importance of being able to accurately implement a triaging protocol. Triage has always been the responsibility of nurses, but in most pediatric patients, the physician on duty would be consulted. Therefore, a PaedCTAS triage course was administered over 2 months to all nurses and physicians working in the ED, with the aim of evaluating the accuracy of triaging by reviewing the ED sheets of patients seen before and after the course.

Subjects and Methods

The study protocol was approved by the Ethics Committee of King Fahad Armed Forces Hospital in compliance with the Declaration of Helsinki and the Good Clinical Practice guidelines. This was a prospective study with the aim of assessing the accuracy of triage in the Pediatric Emergency Department before and after a PaedCTAS course. Recorded triage score was obtained by reviewing the documented notes on the ER sheet of each patient. The correct score was calculated based on findings on the sheet and was determined by two different Pediatric Emergency physicians. Only scores where there was full interobserver agreement were included in the study.

The triage course was a weekly 3-hour session that encompassed two months (total of nine sessions), and each nurse and physician was expected to attend three of these sessions. Full compliance was ensured by integrating this session into the monthly schedule. The three hours consisted of a one-hour orientation lecture on the importance and technique of triaging, followed by an hour of interactive discussion of cases, and finally time allocated for a workbook to be used for each participant with a required pass score of 90%.

The staff all felt confident with their skills following the course, but there were reports of confusion after a couple of months, so feedback was provided, and a refresher course was prepared and conducted but without complete attendance due to scheduling difficulties.

The primary outcome of this study was the comparison of accuracy of triage before and after the triage course. Secondary outcomes included identification of most difficult triage levels to identify and comparison of admission rates to expected rates for each triage level as a surrogate marker for triage accuracy.

Review of patient encounter sheets started in January 2010 for 6 months, followed by 2 months of the training course then post-course sheets review for 12 months. Only patient encounter sheets where two pediatric emergency physicians agreed upon the triage level were included in the study.

Statistical analysis was performed via SPSS for Windows Version 15.0. Rates of accurate triage were calculated as percentages, and patient characteristics were expressed as mean and standard deviation. Chi-square and McNemar tests were used to compare results before and after the course, with p-value of less than 0.05 being considered significant.

Results

The study encompassed a total of 20 months, during which patient encounter sheets were reviewed. As illustrated in Figure 1, the total number of sheets initially evaluated for inclusion were 37961 of all patients registered in the PED. Patient encounters were then excluded if the sheets were found to be incomplete (511 patients, 1.3%), if patients were over 16 years of age (231 patients, 0.6%), or if both pediatric emergency physicians did not agree on the CTAS level (6,166 patients, 16.2%). The total remaining patient encounters that were included were 31053.

Figure 1

Selection process of sheets eligible for the research study

The patient encounter sheets reviewed showed the baseline characteristics outlined in Table 1, with a very slight female predominance. Mean age was 5.5 ± 2.9 years, showing a normal distribution curve. An important point to note is that patients who frequently presented to the PED may have more than one patient encounter sheet. The exact number falling into this category were not counted but are not expected to bias results with such a large sample size.

Table 1

Baseline characteristics of the study population

Characteristic	Participants N (%) Total = 31053
Gender
Male	14874 (47.9%)
Female	16179 (52.1%)
Age group
0-30 days	820 (2.6%)
1-12 months	8923 (28.7%)
1-6 years	13859 (44.6%)
7-12 years	6978 (22.5%)
13-16 years	473 (1.5%)
Arrival by ambulance	1073 (3.6%)
Shift of arrival
Day (08:01 – 16:00)	9715 (31.3%)
Evening (16:01 – 00:00)	14598 (47.0%)
Night (00:01 – 08:00)	6740 (21.7%)
Final Disposition
Discharge	27515 (88.6%)
Admission to Pediatric department	1704 (5.5%)
Admission to Pediatric Intensive Care Unit	602 (1.9%)
Mortality	257 (0.8%)

Regarding disposition, mortalities also included patients who were dead on arrival, as well as those who were not aggressively resuscitated because of a Do Not Resuscitate (DNR) order.

Of the total sheets reviewed, 10747 were assessed before the triage course. Table 2 demonstrates rows of correct CTAS levels as calculated by two pediatric emergency physicians, whereas the columns demonstrate the CTAS level calculated by the triage team and show number and per cent of patients triaged in each level before the triage course.

Table 2

Comparison between calculated and correct CTAS before triage course

CTAS level calculated by the triage team						Total
Actual CTAS	Level 1	Level 2	Level 3	Level 4	Level 5
Level 1	90 (78.3%)	17 (14.8%)	7 (6.1%)	1 (0.9%)	0 (0.0%)	115
Level 2	55 (12.9%)	240 (56.5%)	87 (20.5%)	38 (8.9%)	5 (1.2%)	425
Level 3	172 (3.0%)	458 (7.9%)	3854 (66.8%)	846 (14.7%)	441 (7.6%)	5771
Level 4	56 (1.6%)	167 (4.7%)	125 (3.5%)	2573 (72.3%)	636 (17.9%)	3557
Level 5	0 (0.0%)	0 (0.0%)	44 (5.0%)	223 (25.4%)	612 (69.6%)	879

The number of patients in Triage levels 1-5 was 115, 425, 5771, 3557 and 879 respectively. Accurate triage level assignment occurred between 56.5% and 78.3% with the least accuracy occurring in Triage level 2 and the highest in level 1.

After completion of the triage course, a total of 20306 sheets were reviewed (Table 3), with the number of patients in each triage level being 273, 934, 11378, 6396 and 1325 respectively. Accurate triaging ranged from a minimum of 79.1%, which was in level 4 to a maximum of 90.8% in level 1.

Table 3

Comparison between calculated and correct CTAS level after triage course

	Calculated CTAS					Total
Correct CTAS	Level 1	Level 2	Level 3	Level 4	Level 5
Level 1	248(90.8%)	15(5.5%)	10(3.7%)	0(0.0%)	0(0.0%)	273
Level 2	50(5.4%)	768(82.2%)	83(8.9%)	33(3.5%)	0(0.0%)	934
Level 3	333(2.9%)	504(4.4%)	9449(83.0%)	788(6.9%)	304(2.7%)	11378
Level 4	11(0.2%)	152(2.4%)	240(3.8%)	5057(79.1%)	936(14.6%)	6396
Level 5	0(0.0%)	0 (0.0%)	31(2.3%)	128(9.7%)	1166(88.0%)	1325

The accuracy of triaging was calculated before and after the course, with statistically significant improvements in all PaedCTAS levels, as illustrated in Figure 2.

Figure 2

Percentage of accurate triaging before and after course

The greatest improvement occurred in Triage Level 2, The number and percentage of errors before and after the course decreased dramatically, the differences highly significant with a p-value of less than 0.001 for all levels (Table 4).

Table 4

Comparison of number and per cent of triage errors in each level before and after triage course

Triage level	Triage errors No (%)		P-value
	Before course	After course
Level 1	25 (9.2%)	25 (2.2%)	0.001*
Level 2	185 (43.5%)	166 (17.8%)	< 0.001*
Level 3	1917 (33.2%)	1929 (17.0%)	< 0.001*
Level 4	984 (27.7%)	1339 (20.9%)	< 0.001*
Level 5	267 (30.4%)	159 (12.0%)	< 0.001*
Total	3378 (31.4%)	3618 (17.8%)	< 0.001*

Statistically significant at < 0.05.

A more detailed analysis of the errors that had occurred prior to the triage course revealed that there was significantly more down-triaging than up-triaging for Triage levels 2, 3 and 4. As expected, Level 1 was only down-triaged whilst Level 5 was only up-triaged (Figure 3a and 3b).

Figure 3

A) and B) Up and down triage before and after course

An alternative method to assess the accuracy of triaging is how well it correlates with admission rates. For CTAS Level 1 patients, there was a marked decrease in admission rates into the Pediatric Inpatient department with an increase in PICU admission rates after the course. Triage level 2 patients, on the other hand, displayed higher rates of admission into the pediatric department with less admission into the PICU (Table 5).

Table 5

Patient disposition according to triage level

Triage level	Inpatient admission			ICU admission
	Before course	After course	p value	Before course	After course N = 20306	p value
1	5/115 (4.3%)	4/273 (1.5%)	< 0.001	108/115 (93.9%)	260/273 (95.2%)	< 0.001
2	283/425 (66.6%)	850/934 (91.0%)	< 0.001	39/425 (9.2%)	72/934 (7.7%)	< 0.001
3	117/5771 (2.0%)	357/11378 (3.1%)	< 0.001	31/5771 (0.5%)	81/11378 (0.7%)	< 0.001
4	48/3557 (1.3%)	17/6396 (0.3%)	< 0.001	11/3557 (0.3%)	0/6396 (0%)	< 0.001
5	14/879 (1.6%)	9/1325 (0.7%)	< 0.001	0/879 (0.0%)	0/1325 (0.0%)	N/A

Discussion

Inspired by the importance of accurate triaging in Pediatric Emergency Department settings, we aimed in this study to enhance this using a triage course. This included a large number of patient encounter sheets, reaching 31053 after exclusion of approximately 6000 sheets for various reasons.

Our patients’ characteristics were found to follow a normal distribution curve, which is expected. We found a mean age of 5.5 years, although El-Desoky et al., [9] who performed a study in a nearby hospital and found that the toddler age group had the highest number of patients. A possible explanation for this is the fact that our hospital had many sickle cell patients making up approximately 20% of our pediatric population (according to an internal census). Most of these patients were older children and adolescents, possibly pushing up the mean age in our study.

There was a significant improvement in accuracy with the training course. Of the 10747 patient sheets completed before the course, the accuracy rate was 56.5% and 78.3% jumping to 79.1% to 90.8% in the 20306 sheets after the course. The second set of sheets were included immediately after all staff were trained, and may, therefore, be falsely low as there was a learning curve with several discussions to provide feedback on incorrectly determined triage levels. Although we were hoping for higher rates, realistically, triage is a challenging task in the pediatric age group for many reasons. First of all, according to CTAS guidelines, heart and respiratory rates should be measured at rest [10]. However, the psychological stress of being in an unfamiliar environment makes this an almost impossible feat. Additionally, tachycardia and/or tachypnea may be normal physiological responses in febrile children rather than an indication of respiratory distress or hemodynamic instability [1].

In both pre-course and post-course groups, level 1 was the most easily identified, explained by the fact that patients requiring resuscitation are critically ill enough to be recognisable. Initially the least accurate was level 2 which was alarming as these may easily deteriorate if unattended to. After the course level 4 was the least recognised, which although incorrect, is expected to have less of an impact on morbidity and mortality.

The number and percentage of errors before and after the course decreased significantly, with those errors remaining being mostly down-triaging. This is contrary to findings in other studies where the main issue was over-triaging, such as with Chang et al., [12] who reported that abnormal vital signs let to over-triaging of pediatric patients, with a subsequent delay of more urgent patients. Many of the doctors and nurses at our institution realised that there was usually a tendency to up-triage and we have emphasised that excessive subjective modification of acuity level was discouraged by CTAS guidelines [6] unless it is applied with caution such as in a case of tachycardia in a crying child. We suspect that in our trials excessive down-triaging may have been due to underestimation of the patient’s abnormal vital signs because of exhaustion or lack of experience.

An alternative method to assess the accuracy of triaging is how well it correlates with admission rates, which has been considered a surrogate marker for severity. Gravel et al., [13] performed a CTAS multicenter validation study and found a strong correlation between the triage level and various markers of severity, including admission rate. The admission rates reported in this study after the triage course correlate well with some of the pooled admission rates calculated after a multicenter study that included twelve Canadian Pediatric Emergency Departments [13]. The pooled rates reported by them were 61%, 30%, 10%, 2% and 0.9% for levels 1 – 5, respectively which especially agrees with our results for the lower triage levels [1], [3], [9], whilst triage levels 1 and 2 were within anticipated CTAS level admission rates of 70 – 90%, 40 – 70%, 20 – 40%, 10 – 20% and 0 – 10% respectively [10]. This contrasts with the admission rates reported according to the patient encounter sheets before the triage course, which emphasised the beneficial effect of the training the PEM staff received.

In conclusion, our assessment of the accuracy of triaging in this study was performed both via comparing assigned triage levels to that of two experienced pediatric emergency physicians, as well as by the admission rates according to assigned triage levels. There was marked improvement using both parameters. Limitations of this study were mainly that there was a high turnover of nursing staff, as well as over-crowding of the PED during the winter months, both of which may have led to falsely low accuracy. We recommend follow-up studies to assess waiting times after the training course and compare them with international standards [14].

Limitations of the study: This study was performed in an Emergency department with unique circumstances, which reduces its generalizability. We tried to maximise internal validity by having two independent physicians review the charts. Possible confounding factors were high turnover rates of nurses and a high prevalence of sickle cell disease patients, where a patient with fever would need immediate attention as per hospital policy. Repeated courses and review sessions were performed to adjust for the high turnover.

What is known: Using PaedCTAS for triaging in Emergency departments has been proven to prioritise care to critical patients and ensure optimal distribution of resources depending on the severity of illness and presentation. This is usually organised by a well-established Pediatric Emergency team with nurses well-trained in Emergency care and with expertise in pediatric care.

What this study adds: Our study assessed triaging in a newly established Pediatric Emergency Department (PED) with nurses who had little to no experience in recognising pediatric emergencies. A triage course performed by the PED team was found to be effective in that more accurate triage categorisation was performed. Our study also suggested that constant reinforcement and debriefs helps boost and maintain the learning curve.