Simulation experiences of paramedic students: a cross-cultural examination.

BACKGROUND: Simulation-based education is an important part of paramedic education and training. While accessing clinical placements that are adequate in quality and quantity continues to be challenging, simulation is being recognized by paramedic academics as a potential alternative. Examining students' satisfaction of simulation, particularly cross-culturally is therefore important in providing feedback to academic teaching staff and the international paramedic community.
OBJECTIVE: This study aimed to compare simulation satisfaction among paramedic students from universities in Australia and Jordan.
METHODS: A cross-sectional study using a paper-based English version of the Satisfaction with Simulation Experience Scale was administered to paramedic students from all year levels.
RESULTS: A total of 511 students participated in this study; 306 students (60%) from Australia (Monash University) and 205 students (40%) from Jordan (Jordan University of Science and Technology). There were statistically significant differences with large effect size noted in all three original factors between Australian and Jordanian students: debrief and feedback (mean =38.66 vs mean =34.15; P<0.001; d=0.86), clinical reasoning (mean =21.32 vs mean =18.28; P<0.001; d=0.90), and clinical learning (mean =17.59 vs mean =15.47; P<0.001; d=1.12).
CONCLUSION: This study has demonstrated that simulation education is generally well received by students in Australia and Jordan although Australian students reported having higher satisfaction levels then their Jordanian counterparts. These results provide important data for paramedic educators involved in simulation-based education and training in Australia and Jordan and pave the way for other cross-cultural examinations to be explored.

Introduction

The role of a paramedic has developed significantly in the last 30 years, moving away from a “stretcher-bearer” into the role of an autonomous clinician, practicing skillful medical procedures in the prehospital environment.1 Along with advances within the profession itself, the education model for paramedics has also evolved from a traditional “apprentice model with on-the-job training” into a professional-level tertiary educational program, incorporating 3 years of “preemployment” university education.1 University-level education provides students with comprehensive understanding of advanced concepts in anatomy, physiology, professional behavior, ethics and research, complemented by clinical skill training and clinical exposure through both simulation and clinical placements.1,2

Once qualified, paramedics are required to perform high-level clinical skills, often in suboptimal and uncontrolled environments.3 However, the ability to expose students to these experiences in order to facilitate the attainment of a requisite level of competence has become increasingly challenging due to limitations in the availability of clinical placements and actual patient exposure.3 These challenges highlight the real requirement for an adjunct to complement the limited clinical exposure students are able to access, with an increasingly popular and empirically driven solution being simulation.3–5 Simulation provides a safe, learner-centered environment for students to develop both technical and nontechnical skills but also provides a stepping stone to prepare students for clinical placements and ultimately for the complexity and demands of professional practice.1,4,6

In recent decades, the integration of simulation into health care education has grown exponentially internationally.4 Simulation can be broadly described as an educational technique in which “real-world” concepts are integrated into scenarios utilizing low to high fidelity tools to reenact “real-world” situations.3,7 Simulation provides students with an authentic, relevant, and realistic environment to practice their clinical skills and critical thinking, which is safe and nonthreatening.7 Simulation-based education allows educators to design targeted scenarios that are tailored to the curriculum, reinforcing learning objectives in an environment where patient safety is not compromised and can be followed by a structure debrief and feedback.4,5 Simulation is a crucial component of paramedic education and as such must be continuously reassessed to ensure students are satisfied with this teaching method and that they have “active and meaningful” learning experiences.3 Literature suggests, “Satisfaction exerts a greater influence on academic performance than performance exerts on satisfaction.”7 Therefore, a student’s ability to develop skills and knowledge can be linked to their satisfaction with simulation, thus placing emphasis on appropriate evaluation on the ability to “create satisfying and engaging learning experiences.”7

The paramedic programs

The Bachelor of Emergency Health (BEH) is an Australian-based undergraduate degree taught by Monash University that incorporates 3 years of multifaceted education (4 years if students study a double degree with a Bachelor of Nursing). It incorporates traditional and blended delivery as well as up to 600 hours of clinical placements and low- to medium-level fidelity simulation-based education. The BEH degree commenced in 2004 and provides students with a qualification to seek employment as a paramedic in Australian or other ambulance services around the world. In Jordan, a 4-year Bachelor of Paramedic and Emergency Care (BPEC) has been offered at the Jordan University of Science and Technology (JUST) since 2011. The BPEC program includes a variety of topics provided via lectures, tutorials, or practical formats, in addition to clinical placements either in hospital or on an ambulance achieving between 300 and 400 hours. Practical classes incorporate low to medium fidelity simulation to allow students to practice the assessment and management of a variety of medical or traumatic conditions. Both programs are taught in the English language, have similar curricula structure and content, and are accredited training programs providing an opportunity to undertake a cross-cultural comparison.

The Satisfaction with Simulation Experience Scale (SSES) was initially developed by Levett-Jones et al7 in 2011 to assess the differences in satisfaction between nursing students using medium and high fidelity manikins during simulated-based learning. It was found to be a valid and reliable scale, and study participants (n=344) were highly satisfied with simulation as a means of learning regardless of the level of fidelity.3,7 In 2012, this study was replicated to assess satisfaction among paramedic students at an Australian University and demonstrated similar findings to those found in the study by Levett-Jones et al.3 Both studies concluded that further research is required to examine satisfaction among other cultural contexts.3,7 Undertaking a cross-cultural examination is important for the paramedic profession for two reasons. First, cross-cultural examination provides paramedic programs with important quality assurance measures; such as, benchmarking processes, assessment standardization, or simulation techniques. Second, understanding if cultural differences exist in paramedic education is particularly timely given the expansion of the paramedic profession and mobility of paramedic graduates to different countries.8

This study aimed to compare simulation satisfaction among paramedic students from Australia and Jordan using the SSES.

Methods

Design

A cross-sectional study using a paper-based English version of SSES was administered to students of all year levels from the BEH at Monash University and BPEC at JUST during semester 2, 2013.

Participants and procedures

All undergraduate paramedic students enrolled in the BEH (total enrollment n=549) or the BPEC (total enrollment n=225) were eligible to participate in the study. At the conclusion of classes, students were invited to participate in this study. Ethics approval for this study was obtained from the Monash University and JUST human ethics committees. Participants were provided with an explanatory statement and informed that participation was both anonymous and voluntary. The process was facilitated by a nonteaching member of staff, and participants were asked to complete a questionnaire containing the SSES and a brief set of demographics.

The process took ~10 minutes, and consent was implied by completion of the questionnaire. Participants were informed about the study in an explanatory letter provided to them before completing the questionnaire. They were advised that this study was both anonymous and confidential in nature and that they could withdraw from the study before submitting their questionnaire. No identifying information was required on the questionnaire, and the data were analyzed on a group basis. Neither incentives nor follow-ups were undertaken. There were no exclusion criteria for this study.

Instrumentation

The SSES is an 18-item scale that involves students having to rate their agreement with each item on a 5-point Likert scale (where 1 =strongly disagree and 5 =strongly agree). While developed originally by nursing researchers, the SSES is generic in nature and can be replicated to other health-related disciplines. The authors of the SSES found the scale to be valid and multidimensional with three factors: factor 1, debriefing and reflection (items 1–9); factor 2, clinical reasoning (items 10–14); and factor 3, clinical learning (items 15–18). The SSES demonstrated high reliability with Cronbach’s alpha coefficients of 0.94, 0.86, 0.85 across each factor.7 No items are reversed scored. No item stems were altered in any way for cultural interpretation.

Data analysis

The Statistical Package for Social Sciences (Version 20.0; IBM Corporation, Armonk, NY, USA) was used for management and the generation of statistics. Descriptive statistics including mean and standard deviations were used to summarize the demographic and SSES data. Inferential statistics using independent t-tests and one-way analysis of variance were used to compare the differences between year levels, sex, and country. A Spearman’s rank-order correlation (rho=) analysis was also undertaken to test associations between satisfaction scores and previous health care background and the number of simulations. Initial analyses were conducted to ensure there was no violation of the assumptions of normality, linearity, multicollinearity, and homoscedasticity. The effect sizes (d=) were calculated to evaluate the findings; results are considered statistically significant if P-value is <0.05.

Results

Participant demographics

A total of 511 students participated in this study; 306 students (60%) from Australia (Monash University) and 205 students (40%) from Jordan (JUST). This distribution represents a 55% and 91% response rate from Monash University and JUST. The background of the participants is described in relation to year level, sex, and age. There was a good distribution between all year levels (first years 22.7%, second years 28.4%, third years 27.4%, and fourth years 21.3%). Of the 511 participants involved in the study, the majority of students (82.2%) were <26 years of age n=420, female (61.3%) n=313, and actively involved in a manikin-based simulation during their degree more than 20 times (37%) n=187. Of the total participants, the majority did not have a previous health care-related qualification (88.5%) n=452. The complete distribution of demographic data are reported in Table 1.

Table 1

Distribution of demographic data

Variable	MU (%)	JUST (%)	Sig
Year level
Year 1	97 (31.7)	19 (9.3)
Year 2	104 (34.0)	42 (20.5)
Year 3	73 (23.9)	67 (32.7)
Year 4	32 (10.5)	77 (37.6)
			P<0.001
Sex
Male	102 (33.3)	96 (46.8)
Female	204 (66.7)	109 (53.2)
			P=0.002
Age (years)
<21	144 (47.1)	45 (22)
21–25	129 (42.2)	102 (49.8)
26–30	23 (7.5)	53 (25.9)
31–35	6 (2.0)	5 (2.4)
36–40	4 (1.3)	0 (0.0)
			P<0.001
Previous health care degree
Yes	35 (11.4)	24 (11.7)
No	271 (88.6)	181 (88.3)
			P<0.001
How many simulations with manikins have you actually been involved in?
<5	14 (4.6)	16 (7.8)
Between 5 and 10	73 (23.9)	31 (15.1)
Between 10 and 15	47 (15.4)	43 (21.0)
Between 15 and 20	52 (17.0)	46 (22.4)
>20	120 (39.2)	69 (33.7)
			P=0.018

Abbreviations: JUST, Jordan University of Science and Technology; MU, Monash University; Sig, significance.

A number of items produced high mean scores. For example, Australian students reported that “This was a valuable learning experience (item 14)” mean =4.48 (0.61) and that “The simulation tested my clinical ability (item 16),” mean =4.46 (0.61). For Jordanian students, several items also produced high mean scores “The simulation helped me to apply what I learned from the case study (item 17)” mean =4.11 (0.83), and “The debriefing provided an opportunity to ask questions (item 4)” mean =4.09 (0.41). The lowest mean score reported by the Australian students was “The simulation helped me to recognize patient deterioration early (item 13)” mean =3.77 (0.89), while the lowest item for the Jordanian students was “The simulation developed my clinical reasoning skills (item 10)” mean =3.51 (1.08). Complete item-level results are reported in Table 2.

Table 2

Comparative item-level results of the SSES between Australian and Jordanian cohorts (n=511)

Item	MUmean (SD)	JUSTmean (SD)	Sig
The facilitator provided constructive criticism during the debriefing (item 1)	4.30 (0.58)	3.67 (0.93)	P<0.001
The facilitator summarized important issues during the debriefing (item 2)	4.30 (0.57)	3.74 (0.88)	P<0.001
I had the opportunity to reflect on and discuss my performance during the debriefing (item 3)	4.29 (0.65)	3.74 (0.88)	P<0.001
The debriefing provided an opportunity to ask questions (item 4)	4.43 (0.68)	4.09 (0.41)	P<0.001
The facilitator provided feedback that helped me to develop my clinical reasoning skills (item 5)	4.35 (0.61)	3.67 (0.89)	P<0.001
Reflecting on and discussing the simulation enhanced my learning (item 6)	4.42 (0.67)	3.76 (0.89)	P<0.001
The facilitator’s questions helped me to learn (item 7)	4.17 (0.65)	3.72 (0.86)	P<0.001
I received feedback during the debriefing that helped me to learn (item 8)	4.38 (0.60)	4.09 (0.42)	P<0.001
The facilitator made me feel comfortable and at ease during the debriefing (item 9)	4.02 (0.72)	3.67 (0.90)	P<0.001
The simulation developed my clinical reasoning skills (item 10)	4.36 (0.62)	3.51 (1.08)	P<0.001
The simulation developed my clinical decision-making ability (item 11)	4.37 (0.63)	3.59 (0.94)	P<0.001
The simulation enabled me to demonstrate my clinical reasoning skills (item 12)	4.34 (0.64)	3.94 (0.74)	P<0.001
The simulation helped me to recognize patient deterioration early (item 13)	3.77 (0.89)	3.62 (0.90)	P=0.067
This was a valuable learning experience (item 14)	4.48 (0.61)	3.62 (0.92)	P<0.001
The simulation caused me to reflect on my clinical ability (item 15)	4.40 (0.62)	3.64 (0.88)	P<0.001
The simulation tested my clinical ability (item 16)	4.46 (0.61)	4.04 (0.46)	P<0.001
The simulation helped me to apply what I learned from the case study (item 17)	4.30 (0.69)	4.11 (0.83)	P=0.004
The simulation helped me to recognize my clinical strengths and weaknesses (item 18)	4.43 (0.64)	3.68 (0.64)	P<0.001

Abbreviations: JUST, Jordan University of Science and Technology; MU, Monash University; SSES, Satisfaction with Simulation Experience Scale; SD, standard deviation.

There were statistically significant differences with large effect size noted in all three original factors between Australian and Jordanian students. Debrief and feedback (mean =38.66 [4.36] vs mean =34.15 [5.98]; P<0.001; d=0.86), clinical reasoning (mean =21.32 [2.63] vs mean =18.28 [3.93]; P<0.001; d=0.90), and clinical learning (mean =17.59 [2.17] vs mean =15.47 [1.58]; P<0.001; d=1.12). These results are suggestive that Jordanian students were less satisfied with simulation when compared to their Australian counterparts.

No statistical differences were detected between sexes and age groups. Between groups, the analysis of variance demonstrated statistical significance (P<0.001) for the three SSES factors for first to third years. Statistical differences were detected with factor 3 (clinical learning) for fourth-year students.

The relationship between a previous health care degree and the number of simulations actively undertaken and each SSES factor was examined using Spearman’s rank-order correlation. There was a medium negative correlation between health care backgrounds and the debrief and feedback factor for Jordanian students −0.481 (P<0.001) and the clinical reasoning factor −0.380 (P<0.001). A small negative correlation was also found in the clinical learning factor −0.274 (P<0.001). No relationships were detected in the Australian cohort.

There was a small to medium positive correlation for each factor for Jordanian students: debrief and feedback, 0.274 (P<0.001); clinical reasoning, 0.337 (P<0.001); and clinical learning, 0.288 (P<0.001). These results suggest that there are some relationships between the number of simulations undertaken and satisfaction with simulation. Again, no relationships were detected in the Australian cohort.

While the total scale produced an overall Cronbach alpha coefficient >0.90, significant differences were noted between both cohorts in the clinical learning factor: Australia 0.86 versus Jordan 0.22, suggesting serious internal consistency issues with this factor.

Discussion

This study attempted to examine and compare the level of satisfaction with simulation between Australian and Jordanian cohorts of paramedic students. Overall, both cohorts reported generally high levels of satisfaction with simulation, with Australian students reporting higher levels of satisfaction. These overall self-reported results suggest that simulation-based education is well received by students at these institutions. Importantly, these findings should be welcomed as they add to the sparse research undertaken on simulation in paramedic education.

Results from this study found that Australian students reported greater satisfaction across each factor in the SSES with simulation when compared with their Jordanian counterparts. No clear conclusions can be drawn for these differences. For example, both undergraduate programs use similar pedagogical approaches, use paramedic tutors in simulations, spend similar time in simulated environments, and have similar opportunities to practice skills in clinical placements both in and out of hospital. One possible reason could be the experience of the staff at Monash University. The BEH at Monash University was first offered in 2004, while the BPEC only began 4 years ago.2 Another point of difference may have been the subtle interpretation of items in the SSES by the Jordanian students. As no studies have examined the cross-cultural psychometric properties of SSES, it is quite possible this may be the cause of differences. For example, a study by Fogg et al9 suggests that language, culture, and heritage influences a student’s learning style, therefore, potentially influencing the student’s perceived satisfaction with simulated learning environments.

Other studies that have examined health care students and their satisfaction with simulation have similarly reported generally high levels of satisfaction among students. Results from these studies have demonstrated that simulation supported student’s development of knowledge, skills, and attitudes in their chosen field irrespective of the level of fidelity used.5,10 Although not addressed in our study, the other notable finding from previous studies is that while students have demonstrated high levels of satisfaction with simulated learning environments, students also report that clinical placements remain an important component to their education, and that simulation should complement and not replace placements.7

When examining the factor of “debrief and reflection”, differences were found among both cohorts of students. According to Mayville,11 the process of debriefing provides students with purposeful direction to help improve their thinking and thought processes. Interestingly, when comparing year levels, contrasting and statistically significant results were produced for the majority of year levels. For example, Australian students were less satisfied from first year to fourth year, whereas, Jordanian students reported being more satisfied toward the final stages of their degree. Again, no clear conclusions were evident in these differences. One possible reason in the Australian cohort could have been the mismatch between complexity of simulations and lack of time for debriefing and reflection. For example, in the final years of the degree, time constraints are a potentially limiting factor with relatively large number of students. This is potentially supported given that no association was found between the number of simulations undertaken by Australian students and their overall satisfaction levels.

Another factor could include variable paramedic educator experience and whether they received adequate training on how to effectively deliver feedback to students. Jordanian fourth-year students depict a large increase in satisfaction when compared to students of the first 3 years of their degree, opposite of their Australian counterparts. One possible reason for this result could have been developing stronger student–teacher relationships with their educators, therefore, students have perceived and received meaningful feedback and debrief from their educator.

Limitations

As for any survey-based study, bias is an unavoidable limitation. There is always the possibility that there are differing attitudes when comparing participants to nonparticipants. It could be suggested that students who have strong views or opinions on their simulation experiences are more likely to participate in such a survey. The study does not factor in students’ experiences outside of paramedic education and how they may impact on the students’ perceptions toward paramedic simulation education. The study may be limited by the fact that it did not consider the student’s differing amount of “actual” involvement with simulation; therefore, the results found may not be a true illustration of the cohort as a whole. For example, one would expect more involvement in simulation as students’ progress through their degree. This study is also limited by the fact that the universities examined may not be a true representation of other universities in the two countries that offer the same degree.

Conclusion

This study has demonstrated that self-reported simulation education is generally well received by students in Australia and Jordan, while Australian students reported having higher satisfaction levels than their Jordanian counterparts. Reasons for these differences are unclear. Also, the results suggest that the items within the SSES may have been interpreted differently by different cultures. These results provide important data for paramedic educators involved in simulation-based education and training in Australia and Jordan, particularly around clinical learning, debriefing, and feedback. Results also pave the way for other cross-cultural examinations to be explored.