Diagnostic Accuracy of Extended Focused Assessment with Sonography for Trauma Performed by Paramedic Students: A Simulation-Based Pilot Study.

BACKGROUND: Training on how to perform a prehospital extended focused assessment with sonography in trauma (EFAST) has resulted in improved outcomes for trauma patients in several countries. The result of previous studies showed good accuracy despite minimal training. However, data on the diagnostic accuracy among untrained paramedic students and the course length in middle-income countries is scarce. In Thailand, the current paramedic education does not include training on prehospital ultrasounds. In the present study, we aimed to investigate the diagnostic accuracy of EFAST among ultrasound-naïve paramedic students and factors that are associated with successful posttest training.
METHODS: Final-year paramedic students attending a 4-year university program were included in this study. A 2-h didactic training session and 1-h hands-on workshop were led by experienced emergency physicians. The diagnostic indices for EFAST interpretation were obtained pretraining and posttraining. The participants' ultrasound image acquisition was also evaluated individually on a mannequin model using a standardized assessment tool.
RESULTS: In total, 47 paramedic students were voluntarily enrolled and underwent EFAST training. Of these participants, 31 (66%) reported having >1 year of experience in the prehospital field. Four were advanced emergency medical technicians before becoming paramedic students. The sensitivity, specificity, positive predictive value, and negative predictive value after training were 85.7% (95% CI, 81.5-89.3), 81.6% (95% CI, 74.2-87.6), 91.6% (95% CI, 87.9-94.4), and 71% (95% CI, 63.3-77.8), respectively. Previous prehospital experience was not associated with accuracy.
CONCLUSION: This study demonstrated that paramedic students in Thailand were able to achieve a competency comparable with that of other medical professionals in a simulated environment. The total 3 h training course was sufficient for them to acquire EFAST skills.

Introduction

Prehospital point-of-care ultrasound (POCUS) has been widely used for patients with trauma, acute respiratory distress, chest pain, stroke, and cardiac arrest.1 Improvements in patients’ outcomes have been demonstrated in both civilian and military populations treated using POCUS. The latter scenario involved identifying and triaging casualties for urgent surgical management.2 Interestingly, there was a practice-changing report that described how the utilization of prehospital focused assessment with sonography in trauma (FAST) could improve the administration of treatments on-the-scene and at trauma bays.3 FAST and pleural ultrasound enabled the diagnosis of pneumothorax and predicted the subsequent need for interventions.4 Extended FAST (EFAST) is an additional diagnostic tool that uses ultrasound technology to detect bleeding signs in the hemothorax, hemoperitoneum, and pneumothorax, particularly in the trauma patient. For emergency medical services (EMS) providers, the FAST exam was reported to achieve 80.1%–100% accuracy in identifying intraperitoneal free fluid.5–7 Moreover, the accuracy of prehospital pneumothorax detection using EFAST was 82.1%–99.4%.8,9 The EMS system in Thailand was established in 1980.10 Since then, emergency nurses and physicians have provided prehospital care of patients. However, there has been a shortage of human resources in this field. Consequently, paramedic training was introduced in Thailand to fulfill the gap since 2010. Within this time frame, approximately 600 graduates who were qualified by the National Institute for Emergency Medicine were added to share the task of emergency response.

According to the Thai Qualifications Framework for Higher Education for the Bachelor of Science in Paramedicine Program, ultrasound skills are not mandatory skills for the qualification of paramedics in Thailand. Although several articles have shown the benefit of EFAST for physicians in the prehospital setting, the benefit of performing prehospital EFAST by Thai paramedics has not been investigated.7,8,11,12 To date, numerous ultrasound curricula have been developed according to the literature, but there has been no standardized training to establish the clinical outcome, especially for the middle-income countries.13 The duration of the curricula varied from 1 h to 2 days, and most of curricula comprised both didactic lecture and practical session.11 A prospective, observational study with paramedic participants from Canada who were mostly ultrasound-naïve and underwent a 1-h training course focusing on the identification of important landmarks showed a high accuracy of FAST interpretation (85.6%).5 Another recent study from the United States found that paramedic and emergency medical technician (EMT) students could accurately detect pneumothorax, pericardial effusion, and cardiac standstill from ultrasound images after a 1-h lecture that covered only the basic concepts of ultrasound and pathologic identification.14

This research aimed to evaluate the diagnostic accuracy of EFAST in simulated scenarios as performed by paramedic students after completing an EFAST training course.

Materials and Methods

Study Design

This was a quasi-experimental study conducted on paramedic students. The participants completed a one-day, hands-on EFAST training program instructed by three experienced emergency physicians (EP) who were certified by the World Interactive Network Focused on Critical Ultrasound and took an expertise role in teaching bedside POCUS to medical students and emergency medicine (EM) residents. According to the American College of Emergency Physicians Policy Statement on Ultrasound Guideline,15 EPs have qualified as a leader of emergency ultrasound education to both physicians and nonphysician health-care providers such as paramedics and nurses. Moreover, the study’s methodology as well as the EFAST training was presented and reviewed by the scientific committee of the Department of Disaster and Emergency Medical Operation. This study was approved by the institutional review board of Faculty of Medicine of Vajira Hospital, Navamindradhiraj University (COA 094/2562), Thailand.

Study Setting

This study was conducted at the Department of Disaster and Emergency Medical Operation, Navamindradhiraj University in Bangkok, Thailand.

Population

Final-year paramedic students attending the Bachelor of Science (Paramedicine) program, which is a 4-year program of Navamindradhiraj University, were recruited in our study. The students were a part of the graduating class of 2019 or 2020 and had never received any formal training on ultrasonography before as it was not a prerequisite skill for graduation. All participants were informed that the course was not a part of the regular curriculum and would not affect their grades. Instead of using name tags, students wore a label with a numerical identifier to maintain anonymity. After the enrollment process, baseline characteristics of each student were collected through a questionnaire that included age, sex, grade point average (GPA), years of experience in the prehospital field, any observation of EFAST ultrasound during the paramedic program, and numbers of ambulance service calls attended. Each of the instructors was EM physicians recruited from the Department of EM, Faculty of Medicine, Navamindradhiraj University and Taksin Hospital, Bangkok, Thailand.

Intervention

All participants were required to complete the standard 3-h EFAST course (Table 1), which comprised a 2-h didactic training session and 1-h hands-on experience. We followed the 2-h style curriculum, which had been proven to be effective among EM residents and paramedics.8,16 The first hour of didactic training included an introduction, ultrasound physics, knobology, and image acquisition. The next hour covered EFAST and pleural ultrasound. After completing the theoretical part, the participants were divided into small groups of six to eight participants per instructor and attended a hands-on practical session for 60 min per group using a standardized patient. During the hands-on session, participants were asked to demonstrate all standard EFAST landmarks, which were identified as 1) hepatorenal or Morison’s pouch, 2) splenorenal space, 3) pelvis, 4) pericardium, 5) left pleural space, and 6) right pleural space.

Table 1

Curriculum Content of the EFAST Course for Paramedic Students

Topics	Contents
Lecture: fundamentals of ultrasound	Ultrasound physics, transducers, and knobologyImage orientation and optimization
Lecture: EFAST	Surface anatomyEFAST standard viewsCase review and summary
Hands-on session	Transducer placementDemonstration of each view

Abbreviation: EFAST, extended focused assessment with sonography in trauma.

Outcome

Before the training started, all paramedic students were required to complete the pretest evaluation by interpreting 10 EFAST prerecorded video clips in 20 min. After completing the training course, they were asked to complete the posttest, which comprised two parts. First, they must interpret another set of 10 EFAST video clips. Second, they were assigned to perform the objective structured clinical examination (OSCE) of EFAST using the ultrasound training model (CAE Blue Phantom™), which was evaluated by two instructors. The simulation model used in this study imitated the real human chest and abdomen, in which fluid could be injected into free spaces to reproduce ultrasonographic images. The ultrasound machines used in this study were Philips Ultrasound Model Affiniti 70 with curvilinear C5-1 and linear L12-3 MHz transducers. Their EFAST skill, performance, and quality of image acquisition were evaluated using a standardized assessment tool. The tool involved a task-specific checklist adopted from the Quality of Ultrasound Imaging and Competence score and confidence score of FAST skill.17–19 The pass mark for the OSCE exam was calculated using the standard settings of the modified Cohen method.20,21 The primary outcome was the diagnostic accuracy of the EFAST examination, which was interpreted by paramedic students and were reported as sensitivity, specificity, predictive values, and likelihood ratios. The secondary outcome was to find factors associated with the diagnostic accuracy and passing of the posttraining examination.

Data Analysis

All analyses were conducted using STATA version 15.1 (StataCorp LP, College Station, TX). Normally distributed data were reported as the mean with standard deviation. Differences in continuous and categorical variables between the two groups were assessed using two independent t-tests and Chi-square test or Fisher’s exact test, respectively. The diagnostic accuracy of EFAST as interpreted by paramedic students was evaluated via an analysis of the receiver operating characteristic. The interrater agreement between two EP was calculated using Kappa statistics. The univariable and multivariable analyses for factors associated with passing posttraining evaluations were estimated using random effects logistic regression with a population-averaged estimator. Multivariable models were developed by adjusting for covariates with p < 0.15 in univariable models.

Results

Our study enrolled 51 paramedic students. Of them, three students were not available during the course period and one participant was excluded because the training was not completed. Finally, we recruited 47 participants to our study (Table 2). There were 34 women and 13 men with a mean age of 23.81 ± 5.17 years. Almost half of the participants had accumulated a GPA of 3.01–3.50—20 participants (42.6%). Four (8.5%) participants were certified advanced emergency medical technicians (A-EMT), but the others (43, 91.5%) were only EMT level. Regarding prehospital field experience, 27 (57.4%) had 1–3 years of previous experience. Of the 47 participants, 24 (51%), 12 (25.6%), and 11 (23.4%) had ridden on ambulance service calls for 1–20, 20–40, and >40 times, respectively. Finally, of those, 23 (48.9%) had previously observed any EFAST ultrasound during their rotation to emergency departments.

Table 2

General Characteristics of Participants (n = 47)

Baseline Characteristics	Number (Percentage)
Sex
Female	34(72.3)
Age (years), mean ± SD	23.81 ± 5.17
GPA (accumulative)
2.00–2.50	1(2.1)
2.51–3.00	12(25.5)
3.01–3.50	20(42.6)
3.51–4.00	14(29.8)
Previous degree
EMT	43(91.5)
A-EMT	4(8.5)
Years of prehospital field experience (years)
<1	16(34.0)
1–3	27(57.4)
>3	4(8.5)
Numbers of ambulance service call attended (time)
1–20	24(51.0)
21–40	12(25.6)
>40	11(23.4)
Any observation of ultrasound during paramedic program	23(48.9)

Abbreviations: SD, standard deviation; GPA, grade point average; EMT, emergency medical technician; A-EMT, advanced emergency medical technician.

In this study, we calculated the diagnostic performances on the basis of the interpretation of EFAST prerecorded video cases, as shown in Table 3. Overall, diagnostic indices improved after didactic training. The pretest evaluation showed a sensitivity of 72.1% (67.2–76.6), a specificity of 61.7% (51.1–71.5), a positive predictive value (PPV) of 88.3% (84.1–91.7), a negative predictive value (NPV) of 35.6 (28.3–43.4), a positive likelihood ratio (LR+) of 1.88 (1.45–2.45), and a negative likelihood ratio (LR−) of 0.45 (0.36–0.57). The posttest yielded higher indices, namely, a sensitivity of 85.7% (81.5–89.3), a specificity of 81.6% (74.2–87.6), a PPV of 91.6% (87.9–94.4), an NPV of 71% (63.3–77.8), an LR+ of 4.65 (3.28–6.6), and an LR− of 0.18 (0.13–0.23). The pretest and posttest accuracies were 70% (95% CI, 65.6–74.1) and 84.5% (95% CI, 80.9–87.6), respectively. The Youden Index (0.67) was calculated from posttest sensitivity and specificity, as shown in Figure 1. Among the students, 41 (87.2%) passed the OSCE exam by two raters, as shown in Table 4. The overall interrater agreement of this exam was 79.2% (95% CI, 75.5–82.8).

Table 3

Diagnostic Performance of EFAST Ultrasonography Among Paramedic Students

Test	Sensitivity(95% CI)	Specificity(95% CI)	PPV(95% CI)	NPV(95% CI)	LR+(95% CI)	LR−(95% CI)
Pretest	72.1	61.7	88.3	35.6	1.88	0.45
	(67.2–76.6)	(51.1–71.5)	(84.1–91.7)	(28.3–43.4)	(1.45–2.45)	(0.36–0.57)
Posttest	85.7	81.6	91.6	71	4.65	0.18
	(81.5–89.3)	(74.2–87.6)	(87.9–94.4)	(63.3–77.8)	(3.28–6.6)	(0.13–0.23)

Abbreviations: EFAST, extended focused assessment with sonography in trauma; PPV, positive predictive value; NPV, negative predictive value; LR+, positive likelihood ratio; LR−, negative likelihood ratio; CI, confidence interval.

Figure 1

Receiver operating characteristic (ROC) curve demonstrating area under the curve (AUC) analysis of extended focused assessment with sonography in trauma (EFAST) among paramedic students. The Youden index was 0.67.

Table 4

Result of the Posttraining Objective Structured Clinical Examination (OSCE)

Evaluating Checklist	Number (Percentage)
Part 1: General skill
1.1 Transducer selection	47 (100)
1.2 Demonstration of the transducer orientation	45 (95.7)
1.3 Adjustment for depth and gain	43 (91.5)
Part 2: Image acquisitiona
2.1 Hepatorenal space	37 (78.7)
2.2 Pelvis	43 (91.5)
2.3 Splenorenal space	39 (83)
2.4 Pericardium	39 (83)
2.5 Pleural spaces	36 (76.6)
Part 3: Interpretation
3.1 Imaging diagnosis	28 (59.6)
4.1 Overall performance	37 (78.7)
Numbers of students passed the testb	41 (87.2)

Notes: aImage acquisition was evaluating according to from the Quality of Ultrasound Imaging and Competence (QUICk) assessment tool. bPass mark was calculated using modified Cohen method.

In the univariable analysis, being male had an unadjusted OR 1.91 (95% CI 1.00–3.67, p-value of 0.05). When adjusted for age, sex, GPA, and prehospital field experience, the adjusted OR (ORadj) of males was 1.6 (95% CI 0.75–3.42, p-value of 0.22), whereas other variables were not associated with the accuracy of using the EFAST exam for students who had passed both posttraining evaluations (Table 5).

Table 5

Risk Factors Associated with Passing Posttraining Evaluations

Factors		Univariable Analysis		Multivariable Analysis
		OR (95% CI)	p-value	ORadj (95% CI)	p-value
Sex
Male		1.91 (1–3.67)	0.05	1.6 (0.75–3.42)	0.22
Age
≥ 22		1.15 (0.61–2.16)	0.67
GPA
3.01–4.00		0.87 (0.46–1.64)	0.67
Previous degree
A-EMT		1.7 (0.56–5.13)	0.35
Years of prehospital field experience (years)
>3		1.44 (0.49–4.22)	0.50
Numbers of ambulance attended (time)
>50		1.85 (0.92–3.73)	0.08	1.42 (0.63–3.17)	0.4
Any observation of EFAST ultrasound during paramedic program		1.31 (0.75–2.29)	0.34

Abbreviations: GPA, grade point average; A-EMT, advanced emergency medical technician; EFAST, extended focused assessment with sonography in trauma; OR, odds ratio; ORadj, adjusted odds ratio; CI, confidence interval.

Discussion

This study was the first research to evaluate the diagnostic performance of EFAST ultrasonography among near-graduation paramedic students in Thailand. After a short training, we found that paramedic students showed a sensitivity of 85.7% and specificity of 81.6% in detecting a positive EFAST exam. In the past literature, the diagnostic measures were reported: sensitivity ranged from 67%–97.5%, and specificity ranged from 56%–97%.11 In a Korean study, Kim et al evaluated FAST for prehospital providers in 240 sampled volunteers blinded to computed tomography (CT) results and found a sensitivity of 61.3% (95% CI 50.5%–71.2%) and specificity of 96.3% (95% CI 92.1%–98.3%).6 However, the study was designed to investigate emergency medical technicians and was performed on random patients who underwent abdominal CT scans. Waterman et al evaluated ultrasound image acquisition from Canadian aeromedical critical care paramedics using a mannequin5 and found that the FAST interpretation was correct with an accuracy of 85.6%, even after 1 h of didactic training. Another study by Basnet et al which was similar to our setting, observed EFAST from emergency medical officers performed on severe-trauma patients and with direct injury to the trunk in an emergency department (ED) after 2 h training. The outcome revealed a sensitivity of 94.8% and a specificity of 99.5% when compared with either additional imaging or intraoperative findings.8 This study pointed out that nonclinician health-care professionals from a middle-income country who were unfamiliar with scanning ultrasound could perform the EFAST exam effectively. The difference in our study compared with the previous studies may have influenced a slightly lower value of diagnostic performance due to the time limitation for interpreting the ultrasonographic imaging files. Hence, paramedics opted to take more time to assess for EFAST, as shown in both studies by Waterman et al and Basnet et al.9,22 When focusing solely on the pretest result, it was revealed that the students could achieve moderate sensitivity and specificity. This might be because 48.9% of the students had observed EFAST on ED patients before receiving the formal training in our study. However, another comparable study by Bhat et al14 which was conducted among prehospital care providers reported mean pretest scores of 63.9%. The percentages of participants who had prior ultrasound formal training in the study was much lower (3.6%), but the mean immediate posttest score was increased to 93.5% after a focused 1-h lecture. The comparison implied that with a proper training could improve the performance of the participants who were unfamiliar with ultrasonography regardless of prior ultrasound knowledge.

Another question was how much time should be given to efficiently train undergraduate paramedics. Upon reviewing the literature, we found that the courses varied from 1 h to 2 days.11 Numerous studies including the aforementioned studies proved that in just several hours, didactic training achieved good results in training prehospital personnel and improved the pretraining and posttraining outcomes.22–26 A longer version of the EFAST curriculum was arranged in a 2-month period, which comprised multiple educational modalities and included a pretest, a 2 h lecture, pocket flashcards, hands-on training, internet-based modules, ultrasound imaging acquisition in different situations, a review session, and a final posttest after completion of 6 weeks of training. The study revealed that successful improvement and knowledge retention were achieved after intensive training over a longitudinal time frame.16 However, our study, which was designed to minimize the theoretical session and essentialize hands-on training following the methods of Kim et al and Waterman et al, resulted in acceptable diagnostic indices.6,22

In logistic regression analysis, we did not discover any factor that was statistically significant enough to explain an association between posttraining accuracy and performance. According to this finding, there were no associations between EFAST posttraining success and prior ultrasound experience or medical background, which were reported by Press et al.16

Limitations

This study investigated diagnostic performance among a small population of paramedic students. The finding from this study revealed a trend in wider paramedics but was not truly represented. We were unable to recruit graduated students because of the dispersed location of paramedics around Thailand. Furthermore, the trial was solely simulation-based and hands-on using a standardized patient. Hence, the scenario was based solely on video files rather than real positive cases, so clinical judgment and anatomic variation were not assessed. The formal training in the future curriculum should include simulations in more realistic situations and an evaluation of more theoretical parts such as ultrasound imaging physics, different available techniques, technologies, imaging artifacts, and noises. Posttraining evaluation was performed immediately after the course; hence, the long-term knowledge retention of EFAST skills should be considered in future research. Owing to small numbers of enrolled participants, the data in our study was analyzed entirely and did not compare the difference years of the students.

The measurement of the image acquisition time was not incorporated into the course design. Occasionally, this aspect mattered in the operational field. Prehospital personnel encounters with the injured patient determined the time limit and prioritization for ultrasound in assessing the patient.

Conclusion

This study emphasized that even ultrasound-naïve paramedic students may be able to interpret and perform well on EFAST in a simulation situation with minimal training. Future research should evaluate whether early training of EFAST can impact clinical outcomes of paramedic performance like in various countries.