The comparison of teaching efficiency between virtual reality and traditional education in medical education: a systematic review and meta-analysis.

BACKGROUND: Virtual reality (VR) technology has developed rapidly in recent years and has been applied in many fields, including medical education. A meta-analysis was performed to compare the examination pass rate of medical students educated using VR and those receiving traditional education to evaluate the teaching effect of VR in medical education.
METHODS: The PubMed, Springer Link, Science Direct, and Wiley Online Library were searched from inception to May 2020. Articles meeting the inclusion criteria were then evaluated, relevant information extracted and a meta-analysis conducted. Students were allocated to a VR group, those trained using VR technology, and a traditional education group, those who received a traditional medical education.
RESULTS: Six studies were included in the meta-analysis. The results indicate a significant difference between the pass rate of students educated using VR and those receiving traditional medical education. The odds ratios and confidence intervals of individual studies and our meta-analysis are illustrated with a forest plot.
CONCLUSIONS: Students in the VR group performed better than those in the traditional education group. Teaching with VR may enhance student learning in medical education. Medical schools should consider making greater use of VR when educating students. 2021 Annals of Translational Medicine. All rights reserved.

Introduction

Traditional medical teaching is lecture-centric and didactic. This mode of teaching is largely based on attendance and memorization (1). In addition to theoretical learning, hands-on practical training also plays an important role in this teaching (2). These methods attract obvious limitations. Traditional lectures are boring and monotonous, and an absence of standardization and realistic models results in many students unable to master practical skills fully (2). Advances in digital technology have been proposed as offering a new way for modern medical education and training (3).

Virtual reality (VR) is a simulation in which computer-generated graphics are utilized to create a realistic world based on the user’s commands, such as gesture and verbal commands (4). An additional feature of VR is real-time interactivity, which means VR technology can receive time-changing input from the user and then modify the environment it created (5). This outstanding feature enables VR to have considerable potential in medical education at both the theoretical and practical levels.

In medical education, VR has been used successfully in the teaching of cranial anatomy allowing for the virtual manipulation of body parts, including the cranial bones by students wearing specialized goggles (6-8). VR has also been used to simulate surgical procedures. After receiving a specifically designed VR curriculum, both novice and experienced surgeons showed a significant improvement in performing laparoscopic colorectal surgery, illustrating that such training is equally suitable across all learning stages (9). Another study showed that participants performed better in knot tying scores when using a 360-degree VR video than a 2D video teaching group (10). Similarly, a study of a VR training curriculum for ophthalmology showed that the median pre-course score was improved by using a VR curriculum (11).

Despite this, some researchers still express concerns about the use of VR training in medical education as it may be viewed as less effective than traditional means of learning (12). Feedback from a laparoscopic suturing training program using box trainer only or box trainer and VR simulation suggested VR simulation had little effect on the operation (13). Some researchers are also worried about the physical and mental problems raised by VR, such as VR sickness (14). We conducted a meta-analysis to assess the overall strength of evidence comparing the pass rate of students trained using VR and those trained using traditional teaching methods to gauge the effectiveness of VR in medical education. We present the following article following the PRISMA reporting checklist (15) (available at http://dx.doi.org/10.21037/atm-20-2785).

Methods

Literature search strategy

According to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (15). A systematic search of relevant articles in PubMed, Springer Link, Science Direct, and Wiley Online Library databases was conducted from inception to May 2020. The search terms were (“Medical education” OR “Medical learning” OR “Medical training”) AND “Virtual reality.” There was no restriction on the language of publication.

Also, the references of all retrieved articles were reviewed to identify all potentially related articles.

Selection criteria

Studies were deemed eligible if they met the following criteria:

The studies were cohort studies or case-control studies;

The studies conducted research on VR and traditional education training in medical courses;

The pass rate of the students in VR group and traditional education group were available;

The studies were published as full-length articles.

No restriction was imposed concerning the type and size of the population studied. Two reviewers independently selected the articles that met the above criteria. A disagreement was resolved by consensus.

Data extraction

For each included study, we extracted the following characteristics:

The last name of the first author(s);

The year of the publication;

The country of the study;

The study design;

The number of students enrolled and the pass rate of students from VR and traditional education courses.

Statistical analysis

Students were allocated to a VR group and a traditional education group. We analyzed the pass rate of students in the VR group and the traditional education group. The odds ratio (OR) and 95% confidence interval (CI) were used to present the statistical values derived from case-control studies’ efficacy analysis. We calculated a pooled estimate of ORs and 95% CIs using the random effect model (16). Also, we calculated the quantity I2 to describe the degree of heterogeneity (17) and conducted a subgroup analysis. We assessed the quality of included studies using the Newcastle-Ottawa Scale. A score of 9 points represents the highest quality of the study. If the corresponding category’s content was not mentioned in the articles included, no points were awarded for that category. All statistical assessments were 2-sided, and the significance level was defined as P<0.05. The STAT 12.0 software was used for the statistical analysis of studies.

Results

Search results and study characteristics

shows the selection process to determine eligible studies. Several studies were excluded, either because they were qualitative or because they lacked a traditional education control group. In total, six studies (10,18-22) met our selection criteria and were included in the analysis (). These studies included a total of 633 experimental subjects involving first-year students, postgraduate, and hospital residents. Information on the total number of participants for each study and pass rate was extracted. Four studies (18,20-22) were conducted in North American countries (America and Canada), 1 (10) in England, and 1 (19) in Korea.

Figure 1

An illustration showing how the studies included in this meta-analysis.

Table 1

Characteristics of studies included in the meta-analysis

Study	Country	Study design	Respondent	Quality score	VR			Traditional education
					Event	Total		Event	Total
Jung et al. [2012] (19)	Korea	Case-control study	Freshmen	5	23	38		18	38
Real et al. [2017] (22)	America	Case-control study	Postgraduate	6	171	237		139	221
Hashimoto et al. [2018] (18)	America	Case-control study	Hospital resident	5	14	14		8	13
Yoganathan et al. [2018] (10)	England	Case-control study	Postgraduate	5	17	20		12	20
Maytin et al. [2015] (20)	America	Case-control study	Hospital resident	4	4	4		2	4
					3	4		0	4
Park et al. [2007] (21)	Canada	Case-control study	Postgraduate	4	1	12		0	12

VR, virtual reality.

Comparison of the pass rate of the VR group and traditional education group

The summary OR of pass rates for the VR group versus the traditional education group was 1.85 (95% CI: 1.32–2.58), with no statistically significant heterogeneity among the results of individual studies (I2=12.3%, P=0.34). The ORs and confidence intervals (CIs) of individual studies and our meta-analysis are illustrated with a forest plot (). This indicates that students’ pass rate in the VR group was higher than those of students in the traditional education group.

Figure 2

Forest plot of the pass rate of the VR group versus the traditional education group. OR, odds ratio; CI, confidence interval; VR, virtual reality.

Comparison of the pass rate of the VR group and traditional education group within different countries

A subgroup analysis of the results from different countries showed an OR of 1.79 (95% CI: 1.23–2.60) in North American countries and 2.12 (95% CI: 0.98–4.60) in Asian and European countries (, ). This indicates that in North American countries, the pass rate of VR trained students are possibly higher than those of students educated in traditional courses. However, in Asian and European countries, this relationship was not significant.

Table 2

Subgroup analysis of the passing rate of the VR group versus the traditional education group

Subgroup	No. of studies (ref)	OR (95%CI)	I2	Pheterogeneity
Country
North America countries	4 (18,20-22)	1.79 (1.23–2.60)	31.5%	0.21
England and Korea	2 (10,19)	2.12 (0.98–4.60)	0.0%	0.38
Profession
Freshmen	1 (19)	1.70 (0.69–4.23)
Postgraduate	3 (10,21,22)	1.64 (1.13–2.39)	0.0%	0.48
Hospital resident	2 (18,20)	15.73 (2.35–105.04)	0.0%	0.93
Size
Small	2 (20)	13.50 (1.19–153.38)	0.0%	0.73
Medium	4 (10,18,19,21)	2.68 (1.32–5.44)	0.0%	0.43
Large	1 (22)	1.53 (1.03–2.27)

OR, odds ratio; CI, confidence interval; No., number; VR, virtual reality.

Figure 3

Forest plot of the pass rate of the VR group versus the traditional education group according to different countries. OR, odds ratio; CI, confidence interval; VR, virtual reality.

Comparison of the pass rate of the VR group and traditional education group with different stages of medical training

A subgroup analysis of results in different stages of medical training showed an OR of 1.70 (95% CI: 0.69–4.23) in the freshmen group, 1.64 (95% CI: 1.13–2.39) in the postgraduate group, and 15.73 (95% CI: 2.35–105.04) in the hospital resident group (, ). This demonstrates significant differences in the pass rate of postgraduate and hospital resident groups. However, there was no significant difference in the pass rate in the freshmen group.

Figure 4

Forest plot of the pass rate of the VR group versus the traditional education group according to different professions. OR, odds ratio; CI, confidence interval; VR, virtual reality.

Comparison of pass rate of VR group and traditional education group with different sample sizes

We divided the sample size of groups used in the analysis into a small, medium, and large group. Groups with a sample size of 10 or less were designated as small; those with a sample size of greater than 10 and less than 100 designated as a medium; and those with a sample size of over 100 as large.

A subgroup analysis of these groups showed an OR of 13.50 (95% CI: 1.19–153.38) in the small group, 2.68 (95% CI: 1.32–5.44) in the medium group, and 1.53 (95% CI: 1.03–2.27) in the large group. This showed that regardless of the sample size, the pass rate of VR students was higher than that of those receiving traditional education (, ).

Figure 5

Forest plot of the pass rate of the VR group versus the traditional education group according to sample size. OR, odds ratio; CI, confidence interval; VR, virtual reality.

Discussion

The results of this meta-analysis suggest that students training with VR achieve better pass rates than those educated using traditional training methods. Our result showed a significant difference in the pass rate of the VR group and the traditional education group (OR =1.85, 95% CI: 1.32–2.58).

In the subgroup analysis stratified by profession, the pass rate of postgraduate and hospital residents in the VR group was higher than that of postgraduate and hospital residents in the traditional education group. This may suggest that VR can be used in training for the acquisition of complex skills and specialized knowledge. In the subgroup analysis stratified by country, VR training provided better pass rates indicating a global application. Results in the subgroup stratified by sample size produced similar results, indicating VR can train various student group sizes.

This analysis supports the results of other studies comparing the use of VR and traditional forms of medical training. VR training attracts strong student satisfaction concerning learning and knowledge acquisition (23-26). Students felt that VR improved their autonomic knowledge due to its real-time feedback and three-dimensional computer-generated scenery (4,6-8). VR has been used to simulate an operating room environment where students practiced laparoscopic colorectal surgery, resulting in a reduction in mistakes they made in a real environment (9). Students also express a greater level of confidence in completing the tasks they had been trained to do and developed a better understanding of the processes involved (27).

However, there are still some disadvantages to VR which remain to be solved. Firstly, the transfer of skills after VR training to the clinical setting needs further evaluation (28). Secondly, VR procedures, particularly those involving surgery, require careful design, including highly complex software design (29). Lastly, and most obviously, the question of whether VR can ever simulate a real-life environment completely accurately requires consistent review (2).

Our analysis’s strength was comparing the teaching effect of VR and traditional education based on the pass rate of examinations. The use of objective indicators, such as the pass rate, can effectively reduce bias. However, our study also had important limitations. The conclusions reached are less convincing, as only six studies were included in the analysis. Also, an important source of bias may have been introduced as the sample size of some of the analysis studies were small. Finally, the geographical coverage of the studies was limited.

Conclusions

This meta-analysis shows that there is a statistically significant difference in exam pass rates between medical students undertaking traditional and VR based education. The application of VR to medical education facilitates the acquisition of medical knowledge. Accordingly, consideration should be given to combining VR with traditional teaching. We contend that VR will play an important role in medical education in the future.

The article’s supplementary files as