An Experimental Study On Usefulness Of Virtual Reality 360° In Undergraduate Medical Education.

PURPOSE: Various smartphone-based virtual reality (VR) applications allow the users to view 360° videos of real or simulated places. A 360° VR is captured with a special camera that simultaneously records all 360° of a scene unlike the standard video recording. An experimental study was conducted where 4th-year medical students participated in a workshop. PATIENTS AND METHODS: The study was conducted at College of Medicine (COM-J), King Saud bin Abdul-Aziz University for Health Sciences (KSAU-HS) in Jeddah, Saudi Arabia. 360° VR videos including, pre-briefing and debriefing sessions were held for the experimental group, whereas group two (control group) was provided with the interactive lecture. A total of 169 undergraduate medical students attend the 4th year at the College of Medicine (KSAU-HS) Jeddah.
RESULTS: The response rate was 88% for 169 participants, 57 (VR) and 112 (conventional method). The majority of students (93%) thought that VR can be used in medical education. Post-MCQs score (out of 20) was significantly higher in the VR group, when compared to the conventional group (17.4+2.1 vs 15.9+2.9, p-value <0.001). The OSCE score was also better with the VR group (12.9+4.1 vs 9.8+4.2, p-value <0.001). Overall rating of VR satisfaction experience showed a mean of 7.26 of 10.
CONCLUSION: VR provides a rich, interactive, and engaging educational context that supports experiential learning-by-doing. In fact, it raises interest and motivation for student and effectively supports knowledge retention and skills acquisition.

Introduction

Undergraduate and postgraduate medical education is constantly evolving from simple knowledge acquisition to medical-based education.1 These changes are attributed to several elements including changes in community expectations, advancement in medical sciences, shift in a health care environment, and changes in the physician role.2 Learning interactions and experiences are not prioritized among patients, as a result, such changes and patient safety and quality of care are highly prioritized in patient care.3 Competency-based medical education (CBME)4 is defined as “an outcome-based approach to the design, implementation, assessment, and evaluation of medical education programs, using an organizing framework of competencies”. The knowledge and skills through such an approach should then be demonstrated through clinical practice.4 In Saudi Arabia, inclination toward improvement in healthcare services led to modifications in medical education strategies towards CMBE.5 This shift has adopted a national-based framework, namely SaudiMEDs framework (Figure 1).

Figure 1

Saudi MEDs framework.

According to the SaudiMEDs framework, communication and collaboration competency is a core domain of medical competencies and is defined as “the ability to communicate effectively with patients and their relatives and to practice collaborative care by working in partnership within a multi-professional team”.6 Communication skill is not just a natural gift, rather it is an observable skill, which can be taught, learned, and measured in a structural manner. Observing a role model, lectures, and role-playing are traditional methods used in teaching such as skill. While MCQs, direct observation, Objective Structured Clinical Examinations (OSCE), multi-source feedback, and portfolio are preferred methods used in the assessment of communication skills.7

Different technologies can be used successfully as an educational tool to address these goals. These include video games, simulations, eg, virtual reality (VR), mobile applications, and flipped classrooms. Some of the techniques are currently emerging as innovative and effective tools in medical education to optimize the core learning activities.2 Students are offered to establish personal intellectual ownership of new concepts through VR as they visualize and interact with the abstract ideas.8 VR technology is one of today’s most exciting areas of technology.5 It is basically a way of simulating an environment and giving an opportunity to the users for interacting with the environment.8 VR is costly and need technicians for its maintenance; however, VR-based experiential learning as compared to conventional learning provides experiential learning with peers and simulated patients.8

Nowadays, learners are referred to as “digital natives”, as they are born in a digital-centric world and speak the language of technology fluently.9 They have higher expectations in their education than the previous generation, to cope with the demand of this generation.2 Medical education has evolved rapidly, from using technology in patient care towards the establishment of a smart virtual university hospital imitating a real-life hospital. Similar educational goals are practiced, such as improvement in knowledge retention, psychomotor skills, skill coordination, critical thinking, and learning teamwork.10

Various smartphone-based VR applications allow the user to view 360° videos of real or simulated places. A 360° VR is captured with a special camera that simultaneously records all 360° of a scene unlike the standard video recording.11–13 This enables the users to watch the video from different angles using a smartphone and VR goggles. The user controls the direction of the view by physically, turning the head around. Several reviews have documented positive outcomes of using VR to mimic real-world settings,14 develop students’ confidence,15 learn at own student’s pace, and repeat practice without any harm to a patient.16

Majority of the existing VR simulations in medical education focus on training of cognitive and psychomotor skills; for example: in anatomy education,17 surgical skills,18 emergency medicine,19 and nursing education.20 Various institutional practices are observed when communication skill development is considered in medical education. The assessment of the learner’s skills is attempted on the standardized patients. It serves to be both time-consuming and resource expensive; although, the practice offers various advantages. The use of VR is observed for providing a learning environment that is more controllable, secure, and safe. Learners are provided with the opportunity to practice their skills in an extensive and repetitive manner followed by feedback deprived of any consequence to a real or standardized patient.21

Consideration of using VR technologies in medical education requires a theoretical framework to support these technologies and to guide their implementation for education. The literature of empirical work in the virtual world22 has reported that experiential learning theory is underpinning empirical work in virtual worlds for education; although, there is a pedagogical gap.23 Based on the work of Dewey, Lewin, and Piaget, experiential learning is referred to as learning through action, learning by doing, learning through experience, and learning through discovery and exploration.24 In experiential learning theory, learning is defined as “the process whereby knowledge is created through the transformation of experience. Knowledge results from the combination of grasping and transforming experience”.21 The concept of the experiential learning illustrated as a cycle starts with Concrete Experience (CE), moving to Reflective Observation (RO), then to Abstract Conceptualization (AC), and finally to Active Experimentation (AE), which was first introduced in 1984 by David A. Kolb. Indeed, debriefing is essential to experiential learning theory to let students reflect on the VR experience.25 Experiential learning by Kolb was appropriately presented with virtual learning technology as a student is placed directly to the environment with his or her complete attention and allow repetition and then reflection on the action.26,27 Contrasting of the VR with other conventional modes for teaching, the VR has an edge over others given its subject matter of experience, which is challenging to be demonstrated in conventional modes.24

The VR experiential aspect allows the inducement of interactivity which lacks in the conventional learning paradigm. Since VR allows the learner to derive information from more than one sensory source, where engagement accelerates and potential for ambiguity declines following an interactive multi-disciplinary nature.28 Various researches in education, rehabilitation, and cognitive neuroscience have supplemented the interactive nature of VR. Such as, the 3D world can be portrayed in various forms, which allow the development of various cognitive abilities. The controllable aspect in the virtual environment amplifies its potential for the development of necessary communication skill among clinical professionals.29 In this study, the effectiveness of using VR is evaluated in comparison to conventional educational tools in teaching communication and collaboration skills in undergraduate medical students at King Saud bin Abdulaziz University for Health Sciences, Jeddah 2017–2018.

Methods

Study Design

The study was conducted at College of Medicine (COM-J) – King Saud bin Abdul-Aziz University for Health Sciences (KSAU-HS) in Jeddah, Saudi Arabia. COM-J was established in 2010 and has an adapted curriculum from the University of Sydney, Australia with modification to meet local needs. The curriculum is divided into blocks according to organ systems with the integration of basic and clinical sciences in each block. Educational methods used are conventional and Problem-based Learning (PBL) sessions. The duration of the program is six academic years for stream 1: (students holding a high school graduate); 4½ academic years for stream 2: (university graduates).

Inclusion Criteria

The study included undergraduate Saudi medical students of the college of medicine (male and female), students enrolled for the academic year of 2017–2018, and currently studying in year 4 in both stream 1 and stream 2.

Sample Size

This is an experimental study design, where participants who fulfilled the inclusion criteria, were invited to participate in a half-day workshop about communication and collaboration competencies conducted every week for 6 months. A total of 492 students were enrolled in 2017–2018. The sample size was determined to be 51 students for each group. All students in 4th-year medical students (72 female and 119 male students) were invited to participate in the workshop. Students who signed the consent form and agreed to participate were included in the study, yielding a response rate of 35%. Participants were distributed into two main groups according to their PBL groups which were numbered randomly. PBL groups have been already developed according to students’ GPA levels by academic affairs. Study population who agreed to participate in the study were grouped into two groups, one experimental group, and other control. The two groups considered in this study are as follows:

Group 1 = VR group that comprised of 57 participants;

Group 2 = Conventional experiential learning group (communication training with simulated patients), which included 112 students.

Ethical Consideration

Ethical Approval Number: SP17-081-J from the International Review Board (IRB) was obtained from King Abdullah International Medical Research Centre (KAIMRC). Each participant provided written informed consent to participate voluntarily in this study and no marks were given. The objectives of the study were explained to the participants before proceeding to data collection.

Data Collection

The students received an email as an invitation to the workshop. The outcome factors include perception level, MCQs score, OSCE score, satisfaction level, and evaluation score. The OSCE scenarios and the learning contents length of 1-1¼ based on the lectures were similar as these were based on the aspects of communication. The data were collected in two stages during the workshop for both groups: stage one, pre-session and stage two, post-session.

The Pre-Session Data

A perception survey was conducted using a self-administered questionnaire before the inception of the workshop. The questionnaire comprised four parts where the first section collects the demographic data and determines the awareness level of using technology and VR in medical education. The third section studies the perception of using technology and VR, whereas the perception of benefits and drawbacks of using VR were assessed in the last section. A Pre-MCQs test (20 items) was held to evaluate the knowledge retention of communication and collaboration competencies given at the beginning for 30 mins supervised by a member of the research team.

The Post-Session Data

The same MCQ’s in the pre-test were studied in the post-session. Also, a psychometric analysis was conducted, which reveals a difficulty index between (50–100%) with a good discrimination index between the upper and lower groups. Subsequently, OSCE was held by double-blinded expert examiners and trained simulated patients to assess the skills acquisition. This helped in discussing the role of each team members in the healthcare team. Item examination was adopted from a Kalamazoo assessment tool,29 for evaluating communication and interpersonal skills.30 The tool was modified according to a checklist used in KSAU-HS. Ratings according to the items, using 2 if performed completely, 1 if performed but not fully completed, and 0 if not performed.

Following it, a satisfaction survey was conducted for the obtained data subsequent to VR session for the intervention group only using a questionnaire based on Likert scale in which the highest score is number 10 and the lowest score is 1. It comprised of five different sections evaluating the VR environment perceived quality, easiness of usage, usefulness, and effectiveness along with perceived response focused emotions and flow of usage. The last section studied the overall rating of VR experience.

After the workshop, an assessment form was provided for assessing the level of quality based on 5 point-Likert scales (5=Excellent, 4=Very Good, 3=Good, 2=Fair, and 1=Poor) along with the level of agreement (5=Strongly agree, 4=Agree, 3= Undecided, 2=Disagree, and 1= Strongly Disagree). Both perception survey and satisfaction survey were validated in the English language. The reliability of questionnaire was calculated using Cronbach alpha and its value came out to be Cronbach α = 0.75. Moreover, intraclass correlation coefficient was used to assess the test–retest reliability that yielded an adequate result, ie, intraclass correlation = 0.78. Assessment validity for OSCE and MCQs had been conducted by subject experts for content validity. Moreover, face validity and construct validity had been done by the Medical Education department at COM-J.

Procedure

360° VR videos including, pre-briefing and debriefing sessions were held for the experimental group, whereas group two (control group) was provided with the interactive lecture. During this workshop, the two groups were segregated to avoid contamination of the data. Group one received 3 VR videos which include (effective communication with patient, effective communication with relatives and non-health care professions and importance of the multidisciplinary team in healthcare) for 15 mins with a pre-briefing and debriefing for 30 mins done by experts to fill up the knowledge gap. Furthermore, VR video transcripts were written and modified by content experts and scenario writers to mimic real-life situations. So, students were able to feel as if they are inside that environment. Group two received the same objectives given for group one but in a conventional manner for 45 mins duration. The similar techniques depicted in the video were elucidated through lectures only.

Data Assessment

In the study, SPSS version 20.0 was used for statistically evaluating the collected data. The mean and standard deviation of the workshop evaluation score, pre and post-MCQs scores, OSCE score, and satisfaction score were calculated. The mean and standard deviation of the subjective scores on the Likert scale were also calculated. p-value ≤ 0.05 was considered statistically significant. For descriptive statistic, frequency and percentages were reported for qualitative variables. Mean and the standard deviation was reported for quantitative data. The bivariate analysis, independent sample t-test was used to compare between the groups with a two-tail significance level of 0.05.

Results

A total of 169 undergraduate medical students attend the 4th year at the College of Medicine (KSAU-HS) Jeddah. The age of the participants ranged from 19 to 29 (median age 21 years). Majority of the participants were males (70%), whereas 30% were females. The results reveal that majority of participants were from stream one (73%) and 27% were from stream two. Table 1 shows that VR group included 57 students (34%) and conventional group included 112 students (66%) (Table 1).

Table 1

Demographics Of Participants In The Virtual Reality And Conventional Based Groups (n=169)

		n	%	SD	p-value
Groups	Virtual reality	57	34	17.2	0.37
Conventional method	112	66	15.8
Gender	Male	119	70	–	–
Female	50	30
Streams	Stream 1	119	73	–	–
Stream 2	43	27

In the perception survey, students were asked about their perception towards VR in medical education. The results showed that 50% of participants are only slightly familiar with the concept of VR. In addition, 158 (94%) participants have not tried VR in medical education yet. Still, 157 (92.8%) student thought that VR can be used in medical education. Students believe that technology can aid in education with a mean score of 8.8 of 10. Table 1 shows a mean score of 87.4 +17.2 for VR group and 89.8 + 15.8 for conventional group (p-value =0.37). This survey measured four other sections as stated below the percentages of agreement levels in each section.

Table 2 shows age and gender distribution under both virtual and conventional lectures. Majority of the younger participants rate VR better as compared to conventional lectures, while most participants were male (42.60%); 27.8% male rate conventional lectures better than VR.

Table 2

Age And Gender Distribution Under Virtual And Conventional Lectures

		Virtual Lecture, n (%)	Conventional Lecture, n (%)
Age	21–25	65 (38.46%)	35 (20.7%)
26–30	33 (19.5%)	36 (21.30%)
Gender	Male	72 (42.60%)	47 (27.8%)
Female	15 (8.8%)	35 (20.71%)

Figure 2 has illustrated the bar chart showing levels of agreement on perceptions of technology in education. In this section, the item “Effective technology integration can be a positive change agent in learning” achieved highest agreement with students (93%). Followed by 154 students (91%), who agreed on item “Technology integration into teaching and learning will be very important”, 153 students (90%) agreed on item “Technology integration will benefit students”, and 141 students (84%) agreed on item “Through the use of technological tools, students will work in an environment, which appeals to a variety of learning styles”. The least agreement of 118 students (70%) was for item “Technology uses in the classroom will encourage more student-centred learning”.

Figure 2

Bar chart, levels of agreement on perceptions of technology in education.

Figure 3 shows levels of agreement on perceptions of virtual technology education. In this section, item “I believe that virtual reality is an interesting form of technology and communication” got the highest agreement among the students (84%). This was followed by 113 (67%) students, who agreed on the item “Virtual education will increase flexibility to take classes at any time” and 108 students (64%) agreed on the item “Virtual education will increase flexibility to take classes at anywhere”. The least agreement item was reported for item “I believe virtual reality is a difficult communication technology to use” (46%).

Figure 3

Bar chart, levels of agreement on perceptions of virtual technology education.

Figure 4 demonstrates the levels of agreement on perceptions of affordance. In this section, the item “Virtual education will engage students in learning” has the highest agreement (77%). This was followed by 116 students (69%) for these two items “Virtual education will increase the quality of education by allowing students to learn at their own pace” and “Virtual education will be more effective for motivated and self-disciplined students”. The least agreement (28%) was achieved for item “Virtual education will improve leadership abilities”.

Figure 4

Bar chart, levels of agreement on perceptions of affordance.

Figure 5 shows levels of agreement on perceptions of opportunities. In this section, item “I believe that virtual reality is an interesting form of technology and communication” got 83% agreement and item “I am interested in learning using virtual world technology” got 78% agreement for 132 students.

Figure 5

Bar chart, levels of agreement on perceptions of opportunities.

In addition, questions related to the perception of benefits and drawbacks of using VR were asked. Regarding benefits, 125 students (74%) believe that VR can motivate the student to learn, 103 students (61%) believe that VR can make the difficult concept easier, and 67 students (40%) believe that VR is an active learning experience; 118 students (70%) believe that VR is expensive to implement, 106 students (63%) believe that there is not enough concept available to use VR, and 72 students (43%) believe that VR can isolate students.

Regarding the end of workshop evaluation score, a mean score of 38.7 for both workshops conducted in the VR group and conventional group with a standard deviation of 4.5 (VR) and 4.4 (conventional). This indicated that there are no differences in the conventional and VR group (p=0.98) (Table 3). The results of a post-MCQs score of both groups with a mean score of 17.4 ± 2.1 for VR group and a mean score of 15.9 ± 2.9 for conventional group (p-value <0.001) (Table 3). The students who have participated in the OSCE examination were 132 students with a response rate of 78%. A mean score of 12.9 ± 4.1 for the VR group and a mean score of 9.8 ± 4.2 for conventional group (p-value <0.001) (Table 3).

Table 3

Scores Of Participants In The Virtual Reality And Conventional Groups

Score	Virtual Reality Group	Conventional Group	p-value
End of workshop evaluation	38.1±4.5 (n=41)	38.1±4.4 (n=72)	0.98
OSCE	12.9±4.1 (n=47)	9.8±4.2 (n=85)	<0.001
Pre-MCQs	14.8±3.4 (n=57)	14.0±3.1 (n=111)	0.16
Post-MCQs	17.4±2.1 (n=49)	15.9±2.9 (n=105)	<0.001

Table 4 provides different mean values for men and women showing that satisfaction level of participants in the VR group was significantly different among men and women. The satisfaction survey measured four other sections as stated below:

Table 4

Satisfaction Level Of Participants In The Virtual Reality Group (n=51)

Gender	n	Mean	+	SD	p-value
Male	29	184.7	+	46.6	<0.001
Female	22	234.5	+	34.8

Section one, perceived quality of VR environment (items 1–4);

Section two, perceived easiness of usage of VR environment (items 5–10);

Section three, perceived usefulness and effectiveness of usage of VR environment (items 11–22);

Section four, perceived response focused emotions and flow of usage of VR environment (items 23–29).

Discussion

This study was conducted to evaluate the impact of using 360° VR on students’ knowledge retention and skills acquisitions in contrast to the conventional method of learning. The study has also assessed students’ perception in using VR in medical education and their satisfaction after they have used the VR in learning. Students from group one (360° VR technology) had a higher mean score in both knowledge retention and skills acquisition comparing with group two (conventional). Students’ in both groups believe that technology can aid in teaching with a mean score of 8.8 of 10. In addition, VR group students’ overall satisfaction was 7.26 of 10. Overall, it was found that implementing 360° VR videos into year 4 medical students provided beneficial experiential learning.

According to medical education program accreditation standards in the United States and Canada, “if a medical student does not encounter patients with a clinical condition, the medical student should be able to remedy the gap by a simulated experience”.26 Teaching communication skills in medical education through simulation done by the use of simulated patients (SPs) which are costly in terms of time and money.31 VR could offer a controllable, secure, and safe learning environment with the opportunity for a repetitive practice with feedback without consequence to real patients or SPs.32 However, these results are contrary to the findings of William33 who studied the Kuwaiti nurses’ effectiveness of simulation technology in the healthcare system. The use of simulation and instructional lectures in nurse training improved their performance equally. Still, limited data are available regarding the usage of VR in teaching communication and collaboration skills.

The present study has promoted the role of VR in the context of medical education as an innovative, cost-reasonable, and effective solution for the better and fast practical training. This would help the students in developing direct and meaningful understanding of the learning goals. In addition, computer simulation (MPathic-VR) featuring virtual humans can be used as an online course to teach medical students how to break bad news.34 The results showed statistically significant difference between pre and post-session test (p<0.001) with a greater effect of using VR.34 Few of the previous studies have also shown that virtual technologies are likely to improve students’ academic performance and skills training.35–39 The present study showed increased level of satisfaction among the students that is supported by few of the previous studies.40–42

The present study showed that VR videos’ scripts cases were written by content experts with a modification done by scenarios expert to mimic real-life situations and to represent the culture appropriately. The study 360° VR sessions also allowed users to use their senses to see, hear, and feel. This explained Kolb’s learning theory that people have their own preferred learning style which is based on the four stages learning cycle described earlier.24 All good innovations must start with good pedagogy, although the present study has suggested promising findings for technology innovations like VR for teaching. The satisfaction and motivation aspects were evaluated by Davis,43 who endorsed that the integration of VR in nurse training advances their motivation level. Another study by Rizzo and colleagues44 supported that the virtual technology equips the healthcare workers to expand their knowledge horizon, enhancing their communication skill, which helps in providing better care to the patient.

The results of the present study showed that approximately half of the students were only slightly familiar with the concept of VR and they believed that technology can aid in education. However, there were no differences in the conventional and VR group. The use of VR simulation has also been preferred by learners’ group as stated by Kron et al21 Moreover, another study concluded that VR prepares medical care students to cater the emotion-charged patient, assisting in accomplishing Saudi Vision 2030.45 These results are consistent with the results reported in the present study.

The present study has showed that experiential learning was a suitable theory. In addition, a pre-briefing session and a debriefing session were conducted in between the VR sessions during the workshop to allow students’ to actively participate, reflect, and give their feedbacks to fill the gaps. The results from this study support the fact that VR session does enhance the learning process. Furthermore, the result of this study ensures that undergraduate medical students meet their learning objectives, increase their knowledge and improve their skills. Indeed, Health professional educators should try to ensure that VR sessions are designed in a way to suits every learner and to engage them in the learning process. The study has encountered some of the limitations. Firstly, the number of females were less as compared to males. The drop out of female participants from the workshop and the study was due to logistic issues. Secondly, medical students from same level were involved; however, a better perception could be achieved by involving students from different academic levels.

Conclusion

The VR 360° videos for communication and collaboration skills showed a statistically significant improvement for undergraduate medical students in their knowledge retention, skills acquisition, and satisfaction levels. Indeed, findings in the study support the use of VR as a teaching tool to provide a rich, interactive, engaging educational context, thus supporting experiential learning-by-doing. The study has concluded that VR shows many effects on education and is likely to serve as an essential tool in medical education for the present generation. VR could play a role in interprofessional education to allow undergraduate health professions to learn about each other roles to improve collaboration and communication skills for better health care. Thus, research regarding VR simulation is needed to validate the VR as an educational tool. Secondly, health professional educators might look to integrate such technology in medical curriculums to make the material more interesting, easier to learn, and to adapt to the new generations’ needs.