Exploration of simulation-based medical education for undergraduate students.

ABSTRACT: Over the past decades, extensive studies have underscored the growing importance of simulation-based medical education (SBME) for medical students. However, the underlying influence of SBME on undergraduate students is yet to be investigated. This work is a single-center cohort study involving 1178 undergraduate students who were divided into a control group and an SBME group. All participants gave their written informed consent. We compared the theoretical and practical achievements of these 2 groups and distributed a feedback questionnaire. Results show that SBME significantly improves the practical or theoretical achievements of students (P < .001). The humanistic care (improvement rate: 69.2%) and doctor-patient communication (improvement rate: 56.3%) performances of these studies were vastly improved. The students in the SBME group tend to allocate more time to communicating with others. SBME is an effective teaching method that can improve the reflective capacity and communication skills of undergraduate medical students, thereby resulting in their relatively improved performance.

Introduction

Surgery education is an important branch in clinical medicine. Since its advent in the early 20th century, surgery education has undergone a tremendous transformation from an “apprenticeship” and “journeymanship” toward a training model based on knowledge of basic sciences, research, and patient responsibility. Surgery acts as the “bridge” course that connects the student role to the clinical role and plays an indispensable role in learning the necessary skills in surgery.[ In undergraduate medical education, the traditional education model, where students listen to their teachers, learn knowledge by themselves through their understanding or observations, and are not given the opportunity to learn by immersion, thereby affecting their transition from theory to practice.[

With increasing medical information and research, medical education is facing problems in keeping up with course development. As well, patients are paying more attention to the “practice” from medical students and residents on them. Therefore, clinical medicine has begun to focus on the safety of patients and the quality of their practice and not merely on medical education, which may introduce challenges for medical educators. To overcome this challenge, educators are continuously adjusting the course structure, organize group discussions, and introduce opportunities for students to engage in autonomous learning and independent research.[ However, the classroom model remains disconnected from actual clinical settings. Many students also lack training in theoretical study, physical examination, and diagnosis. Therefore, an educational reform aiming to improve teaching effectiveness and satisfy strict teaching dictates is currently underway in the medical community and has given rise to simulation-based medical education (SBME).[

As a complex intervention method in medical education, SBME was defined by Isenberg et al as “In broad, simple terms a simulation is a person, device, or set of conditions which attempts to present education and evaluation problems authentically.[ The student or trainee is required to respond to the problems as he or she would under natural circumstances.”[ Based on the assumption that learning will be improved in a learning context that closely reflects the clinical context, SBME is deemed to improve clinical medicine education by replacing the classroom model with highly simulated patients and clinical scenarios. Various local functional models, computer-student interaction models, and clinical virtual simulations can make the learning process closely reflect real-world environments and medical ethics by simulating real clinical scenarios involving patients.[ At the same time, the safe environment produced via this simulation permits the failure of students in practice.[ With its proper safety and high repeatability, SBME has been widely used and has become an obligatory route for the reform and development of medical education.[ Many well-known SBME centers have been built worldwide, such as in the University of Miami, the University of Pittsburgh, and UCLA,[ all of which provide professional training teams and venues, adopt independent and profit-making patterns, examine the design of case-based teaching, pay extreme attention to final summaries, and establish a perfect evaluation system.

However, SBME remains in its infancy in China. Despite many difficulties, SBMEs in China have evolved considerably in recent years. In this study, junior students learning surgery from 2018 to 2019 were chosen. The evaluations of these students were used to provide clinical educators with a deeper understanding of the value of SBME for core clinical work scenarios.

Methods and materials

Study design and participants

A controlled experiment was conducted to evaluate the effect of SBME on the knowledge, self-efficacy improvement, and composite scores of clinical students. This work involved junior students studying in the Nanjing Medical University (NMU) between 2018 and 2019. These students attended theoretical learning sessions of surgery, which took the form of 48 classroom-based instruction sessions. The content of these lectures included introduction to surgery, standard aseptic techniques, trauma, and infections. These students were randomly separated into control and SBME groups for the next practice by using a computer to randomly generate numbers. An informed consent was obtained from all students. The practice was divided into 2 parts. The first part involved identifying the surgical instruments and preparing the aseptic. The second part involved sessions on tension suture, suture tying, cardiopulmonary resuscitation (CPR), changing infected wound dressings, and removing drainage tubes.

Difference between the control and SBME groups

All students were asked to undergo a 2-hour practice session after completing the corresponding theoretical learning. The session was led by a clinical teacher and involved 10 students. In the control group, the teacher initially performed practical operation and demonstration, after which the students cooperated to complete their tasks. The education process may be complemented with video or PowerPoint presentations.

In the SBME group, the students received web-based learning by using the “Muke.” app in the durations between their study and practice sessions. This previewed process may help students learn the content of their practice autonomously by consulting technical data by themselves. The practice began with clinical scenarios, in which the teacher added the content of practice (Table 1). Clinical operations were conducted on the molds or a human phantom following the active participation of students in the discussion and after they decided what would be the next action. In this process, the teacher was responsible for guiding the students to find, think about, and solve the problems by themselves. The flowchart of the whole process is shown in Fig. 1.

Table 1

The clinical scene applied in SBME.

Cases	Training items	Preparation and teaching process
Case 1: The patient is a 36-year-old man who had knee joint trauma for 1 hour. The wound is clean, and the skin around which is damaged and strained. The wound has been debridement; please close the wound.	1. Aseptic concept2. Lash-up processing capability3. Doctor–patient communication skills4. Operation skills (tension suture and suture tying)5. Teamwork ability	PreparationsStudents: The video preview of relative operations in the Muke app.Teachers: debriefing skills and intervention and guidance to students in the process of operation
Case 2: The patient is a 46-year-old man. An appendectomy was performed and this patient was discharged three days postoperatively. On the 10th day after the operation, the patient asked to take out the suture according to the discharge summary and complained of incision pain in recent days. The abdominal wound can be seen dressing pollution, local skin swelling, obvious tenderness and purulent secretion; please change the dressing and communicate with the patient.	1. Aseptic concept2. Lash-up processing capability3. Doctor–patient communication skills4. Operation skills (dressing change of infected wounds)	Teaching process1. The teacher firstly put the case in a real scene.2. Students discussion.3. Students say how to deal with it and where need to pay attention to, and began to practice4. Teacher guidance.5. Discussion and feedback
Case 3: The patient is a 38-year-old man who was hospitalized with “recurrent right upper abdominal pain for more than 6 months” and was diagnosed with cholecystolithiasis and cholecystitis. The patient underwent laparoscopic cholecystectomy (LC) and recovered well. The drainage tube can now be removed; please operate it and clarify the precautions	1. Aseptic concept2. Lash-up processing capability3. Doctor–patient communication skills4. Operation skills (removing the drainage tube)
Case 4: The patient is a 57-year-old man. During hospitalization for cervical spine fracture, the patient had a sudden cardiac arrest; please perform CPR for the patient immediately.	1. Aseptic concept2. Lash-up processing capability3. Doctor–patient communication skills4. Operation skills (tension suture and suture tying)5. Teamwork ability

Figure 1

The general flowchart of the whole research. SBME = simulation-based medical education.

Total grade

The total scores were computed based on the usual performance and final scores of the students. Usual performance was scored based on these students’ usual theoretical (2 times; 10 points) and practical achievements (1 time; 5 points). At the end of the semester, all students were required to take a final examination divided into 2 sections, namely, final theoretical (50 points) and practical achievements (35 points). Standard protocols were followed in all exams.

Feedback on the SBME

The students were asked to participate in an online survey assessing their self-efficacy improvement after completing the final exam. The survey covered 5 aspects, namely, SBME knowledge, operation method, learning experience, satisfaction with the SBME, and improvements in communication skills and humanistic care. The degree of satisfaction or agreement was scored on a scale of 1 (least satisfied) to 5 (most satisfied).

Statistical analysis

All analyses were performed by using SPSS version 23 (IBM Inc., New York, USA) and R version 3.6.1 (Lucent Technologies, New Jersey, USA). All statistical tests were 2-sided, and P-value <.05 indicated statistical significance. An independent t test was performed to compare the continuous variables of the normal distribution, and a Mann–Whitney U test was performed to compare the continuous variables of skew distribution. The scores were expressed as means plus or minus standard deviations (SD).

Results

Baseline characteristics of the participants

A total of 1178 students were enrolled, of which 46.1% were boys and 53.9% were girls. Among these students, 844 were assigned to the control group, and 334 students were assigned to the SBME group (Table 2).

Table 2

The general information of control and SBME group.

Students	Gender	Control group	SBME group	P value
The junior students of 2018	Boys	187	67	.392
	Girls	205	88
The junior students of 2019	Boys	209	75	.330
	Girls	243	104

Grades comparison between the 2 groups

The scores of all students are shown in Fig. 2. Results show that SBME can significantly improve these students’ usual (for the 2018 junior students, control group: 4.22 ± 0.68 vs SBME group: 4.51 ± 0.46, P = .001, and for the 2019 junior students, control group: 4.18 ± 0.68 vs SBME group: 4.40 ± 0.54, P = .036) and final performance of practical operation (for the 2018 junior students, control group: 28.38 ± 0.83 vs SBME group: 31.98 ± 0.88, P = .001, and for the 2019 junior students, control group: 29.12 ± 0.87 vs SBME group: 31.12 ± 0.69, P = .001). No sex differences were observed. SBME also had a positive effect on the final theoretical achievements (for the 2018 junior students, control group: 40.87 ± 0.47 vs SBME group: 42.63 ± 0.51, P = .001; for the 2019 junior students, control group: 41.32 ± 0.43 vs SBME group: 42.02 ± 0.52, P = .001). An interesting conclusion that emerged from the data is that the students in the SBME group tend to communicate with others, which may help reduce their mistakes (Fig. 3).

Figure 2

The grades between the control group and the SBME group. SBME = simulation-based medical education.

Figure 3

Difference in time allocation of 2 hours of practice.

Analysis of the lost points

We then classified the lost scores into 5 categories, namely, basic knowledge, operation skills, humanistic care, aseptic concept, and doctor–patient communication. Results of the preliminary analysis of per capita loss are presented in Fig. 4 and Table 3. The SBME group outperformed the control group across all aspects. In this case, SBME may significantly improve the performance of students in humanistic care (improvement rate: 69.2%), doctor–patient communication (56.3%), and aseptic concept (50.0%) yet only slightly improve their basic knowledge (15.5%) and operation skills (18.4%).

Figure 4

The lost points in the control group and the SBME group. SBME = simulation-based medical education.

Table 3

Analysis of the lost score.

Aspects	SBME group	Control group	Improvement rate∗
The basic knowledge	4.9	5.8	15.5%
Operation skills	3.1	3.8	18.4%
Humanistic care	0.4	1.3	69.2%
Aseptic concept	0.3	0.6	50.0%
Doctor–patient communication	0.7	1.6	56.3%

Feedback of SBME students

A total of 331 students from the SBME group provided their complete feedback on the SBME (Table 4). Before attending our course, only 39.5% and 48.5% of the students had a good understanding of SBME or knew how to do SBME exercises correctly, which slightly reflects the backwardness of SBME popularization in China's medical education. Nevertheless, >80% of the students were satisfied with our SBME and its projects (SBME course, 81.2%; tension suture and suture tying, 82.4%; changing infected wound dressings, 84.1%; removing drainage tubes, 82.9%; and CPR, 83.6%). The majority of these students also thought that SBME may help them improve themselves effectively, especially in dealing with similar situations for real patients (82.4%). Moreover, 79.3% of the students were satisfied with the fidelity of the teaching aids.

Table 4

The feedback of students in the SBME group.

Feedback items	1 point	2 point	3 point	4 point	5 point
	Very disagree/dissatisfied→Very agree/satisfied
Item 01: Before attending this course, I have a good understanding of the principles and operation methods of SBME	1.7%	3.4%	46.3%	34.3%	14.2%
Item 02: Before attending this course, I will especially study the operation method of related projects	0.8%	5.6%	36.1%	41.6%	15.9%
Item 03: Before attending this course, I know how to do SBME exercises correctly	2.1%	12.9%	45.5%	28.8%	10.7%
Item 04: My overall satisfaction with this SBME course	0%	0%	18.9%	56.7%	24.5%
Item 05: My satisfaction with the tension suture and suture tying project	0.0%	0.9%	16.7%	57.9%	24.5%
Item 06: My satisfaction with the dressing change of infected wounds project	0.0%	00. %	15.9%	57.5%	26.6%
Item 07: My satisfaction with removing the drainage tube project	0.4%	0.9%	15.9%	56.7%	26.2%
Item 08: My satisfaction with the CPR project	0.4%	1.7%	6.0%	57.5%	36.1%
Item 09: My satisfaction with the fidelity of teaching aids	0.4%	1.7%	24.0%	53.7%	20.2%
Item 10: My satisfaction with the teachers of SBME course	0.0%	0.0%	6.0%	45.9%	48.1%
Item 11: This course helps me better grasp the clinical operation methods	0.0%	0.0%	9.4%	53.2%	37.3%
Item 12: SBME provides me with a semi-real experience	0.4%	0.0%	15.9%	49.4%	34.3%
Item 13: SBME strengthens my grasp of knowledge	0.4%	0.0%	9.4%	54.9%	35.2%
Item 14: SBME improves my ability to apply knowledge	0.4%	0.0%	8.6%	54.9%	36.1%
Item 15: SBME helps me improve doctor-patient communication skills	0.4%	2.2%	24.0%	45.9%	27.5%
Item 16: SBME helps me improve my awareness of humanistic care	0.4%	0.9%	17.6%	51.1%	30.0%
Item 17: SBME stimulates my interest in learning	0.4%	0.0%	12.9%	51.9%	34.8%
Item 18: SBME relieves my pressure when I use real patients as learning objects	0.4%	2.2%	16.3%	48.9%	32.2%
Item 19: SBME has strengthened my confidence in dealing with similar situations for real patients in the future	0.0%	1.3%	16.3%	51.9%	30.5%
Item 20: My level of participation in the process of this course	0.0%	0.9%	21.5%	49.4%	28.3%

Discussion

The prevailing view is that SBME is superior over traditional didactic methods.[ SBME is also an increasingly essential educational strategy that plays an important role in improving patient safety.[ The trainees in SBME may achieve professional improvement by learning from their mistakes, which will help them avoid the same mistakes in real-life contexts.[ The primary motivation of our research was to evaluate the effectiveness of SBME in training undergraduate students. We divided the 1178 students in our sample into the control and SBME groups according to a certain percentage. Afterward, we explored the underlying effect of SBME on these students by conducting a randomized controlled study.

In sum, the students in the SBME group outperformed those in the control group. Tanoubi et al[ obtained the same conclusion in their prospective and randomized study, in which they found that SBME comprehensively improved the performance of trainees after their practice. In a recent systematic literature review, McGaghie et al[ investigated whether SBME with a deliberate practice yields better results than traditional clinical education.[ They found that SBME was more effective in achieving specific clinical skill acquisition goals compared with traditional clinical medical education (P < .001). These findings consistently point toward the superiority of SBME. On the one hand, SBME provides a favorable environment for critical thinking, focused reflection, and collaboration in which the trainees are given more choices and thereby effectively achieve their goals.[ On the other hand, these trainees tend to deliberate and focus on details based on the immersion posed by SBME, and repetition drills tend to provide improved results.[

Growing evidence suggests that the clinical skills obtained from SBME centers can directly improve the performance of medical students in real scenarios.[ SBME is not just a study setting with a high-fidelity mold; it provides students with high-level feedback and pressure that help them make judgments and medical decisions, develop their situational awareness, engage in teamwork, and demonstrate professional behavior.[ To a certain content, these arguments corroborate the findings of our work. Specifically, after 2 hours of practice, the students in the SBME group allocated more time to communicating with their classmates or teachers. Correspondingly, they demonstrated the best improvement in their humanistic care and doctor–patient communication performance. To date, many studies have identified a link between SBME and communication skills. For example, from a feedback questionnaire completed by 97 physicians, 70% of them positively rated the effect of SBME on the communication skills and competence of students.[ In real doctor–patient scenarios, the enhancement in communication play a pivotal role in discouraging medical disputes and improving physician–doctor relationships.[ Generally, an improved communication is often associated with improved teamwork. In SBME settings, teamwork training tends to improve practice skills, team performance, and knowledge.[ In this study, the teachers in each group were all clinicians in university-affiliated hospitals with substantial clinical experience. Under the SBME environment, these teachers have more chances to teach their clinical experience and communication skills to students.

The application of SBME in medical education is in nascent stages and standard assessment tools are still missing. Meanwhile, considering the diversity of application scenarios, a uniform measurement way is full of limitations. For this, most relevant studies quantified the effect of SBME under their own criteria through comparing outcomes and gathering feedback.[ For example, Korndorffer et al[ developed 0° and 30° laparoscopic camera navigation simulators for medical training. A total of 20 medical students were recruited and randomly divided into training and control groups. Finally, the performance of these students was scored by a blinded rater. They found that the training group performed better in optimal surgical view and procedure time. Mesko et al[ developed a 1-week low-dose-rate prostate brachytherapy course for radiation oncology residents at an academic institution. Outcomes of the SBME course were administered by 20-question multiple-choice test. Van Sickle et al[ divided 22 residents into the SBME laparoscopic suturing curriculum and control group. The performance of these residents was assessed by 2 independent reviewers blinded to the subject and the evaluation items include surgery time, error rate, and extra action. In this study, we employed 3 ways, theoretical achievements, practical achievements, and individualized feedback to assess the effect of SBME on students. Meanwhile, the average score was also refined to 5 aspects, basic knowledge, operation skills, humanistic care, aseptic concept, and doctor–patients communication. These quantification metrics are closer to the real cases of undergraduate medical education in China. Moreover, a large sample size (1178) of our study could significantly diminish the bias and improve the validity and reliability of the conclusion.

Debriefing is essential to enhance SBME learning, whose difference will result in the varied quality of SBME.[ Useful cognitive aids that are often a pocket card, script, or poster in practice could dramatically improve the debriefing process and SBME effect. Cheng et al[ applied network simulation programs into the SBME and obtained ideal effects. In our study, the internet interactive software “Muke.” app was interspersed throughout the whole SBME process, contributing to the mastery of participants to cognitive and behavioral outcomes, especially for medical undergraduates. Furthermore, students were led by a clinical teacher and bond as a group. Debriefing in the form of a team could help them heighten their communication and team-work skills, which was also represented in our results.[ Meanwhile, the involvement of a clinical teacher makes the whole process more flexible and rigorous.[ Moreover, we developed a feedback questionnaire consisting of 20 questions (specific comments on the course, self-promotion, recommendations for course improvement). Feedback is the primary component of debriefing. This bidirectional and interactive discussion could help participants reflect on their actions and performance.[

Despite these promising results, SBME still shows some deficiencies. First, given the limitations of schools and students, a rigorous randomized controlled trial design is not achievable in this work. All students enrolled in our study knew their group (control or SBME group), which may introduce an element of bias to our study and analysis results. Second, this research was only based on a single center, which may introduce a risk of contamination among groups given that the teachers in the SBME and control groups all came from the same affiliated hospital of NMU and may share information with one another. However, our large sample size that allows for sufficient statistical power is an essential strength of this study. Meanwhile, the participants consisted of junior students in 2018 and 2019, which may help reduce the influence of potential confounding factors.

Conclusion

This study can provide a reference for developing innovative teaching models and promoting the application of SBME within the Chinese context. A comparison of findings from the control and SBME groups reveals that SBME may have a positive influence on the education of undergraduate students. The students in the SBME group tend to communicate actively with their classmates and teachers, which may explain the considerable improvement in their humanistic care and doctor-patient communication.

Author contributions

Shangqian Wang and Xiaohan Ren collected the data and performed the analysis. Jun Ye wrote the manuscript. Wei Wang and Huaxing Huang recruited the student participants. Chao Qin designed the whole study. All authors participated in the data analysis and approved the final version of the manuscript.

Conceptualization: Huaxing Huang, Chao Qin.

Data curation: Jun Ye.

Formal analysis: Jun Ye, Wei Wang.

Funding acquisition: Huaxing Huang, Chao Qin.

Investigation: Shangqian Wang, Wei Wang.

Methodology: Shangqian Wang, Jun Ye, Wei Wang, Chao Qin.

Visualization: Wei Wang.

Writing – original draft: Xiaohan Ren.

Writing – review & editing: Xiaohan Ren.