3D virtual reality simulation in radiography education: The students' experience.

INTRODUCTION: Simulation forms a key element of undergraduate Radiography education as it enables students to develop their clinical skills in a safe environment. In this study, an immersive three-dimensional (3D) virtual radiography simulation tool was piloted in an undergraduate Radiography curriculum and user feedback retrieved.
METHODS: The 3D virtual simulation tool by Virtual Medical Coaching Ltd was introduced to first year radiography students (n = 105). This technology guided students through a comprehensive process of learning anatomy, radiographic positioning and pathology. Students then X-rayed a virtual patient in the VR suite using HTC Vive Pro™ headsets and hand controllers. Instant feedback was provided. An online survey was later disseminated to students to gather user feedback. Thematic and descriptive statistical analyses were applied.
RESULTS: A response rate of 79% (n = 83) was achieved. Most respondents (58%) reported enjoying VR simulation, whilst some felt indifferent towards it (27%). Ninety-four percent would recommend this tool to other students. The mean length of time it took for students to feel comfortable using the technology was 60 min (10-240 min). Most respondents (58%) desired more VR access. Students attributed enhanced confidence in the areas of beam collimation (75%), anatomical marker placement (63%), centring of the X-ray tube (64%) and exposure parameter selection (56%) to their VR practice. Many students (55%) advocated the use of VR in formative or low stakes assessments. Issues flagged included technical glitches, inability to palpate patient and lack of constructive feedback.
CONCLUSION: Student feedback indicates that 3D virtual radiography simulation is a valuable pedagogical tool in radiography education IMPLICATIONS FOR PRACTICE: 3D immersive VR simulation is perceived by radiography students to be a valuable learning resource. VR needs to be strategically implemented into curricula to maximise its benefits.

Introduction

Radiographers require considerable technological and scientific expertise in the production of diagnostic images and the responsible delivery of ionising radiation. Such expertise is developed and contextualised through clinical skills labs and clinical placement, both of which are integral to radiography education. Clinical skills labs provide students with an opportunity to learn in a safe environment through interactive clinical simulations. Whilst clinical skills labs are essential to the skill development of student radiographers, the costs associated with installation and maintenance of X-ray equipment are high. Lab access is limited by class size, timetabling confinements and staffing requirements. Acquisition of radiographic images in the X-ray lab must be supervised by qualified staff and can only be undertaken on anthropomorphic phantoms. These anthropomorphic phantoms are not suitable for projections that require a significant degree of manipulation e.g. extremity projections. Instead students role play these projections but are prohibited from acquiring radiographic images on each other for radiation safety reasons. Image acquisition is an important aspect of learning and reflective practice as it enables students to recognise and correct errors e.g. patient positioning, exposure, beam centring and collimation errors. There has been an extension of physical skills labs into computer-based virtual environments , and more recently into immersive 3D virtual reality (VR) simulation environments,4, 5, 6 which overcome some of the aforementioned limitations.

Virtual reality is an innovative technology that enables students to practice radiography in a virtual environment simulating real-life clinical scenarios. Learning is promoted in line with instructional design principles as students must apply their theoretical knowledge to perform radiographic examinations in the virtual environment. Hazell et al. recently explored the role of simulation-based learning in facilitating clinical readiness in Radiography. Their meta-synthesis revealed that students are best prepared for clinical placement by employing simulations that are authentic, realistic, and relevant to the development of a professional in the context of that profession. There is considerable evidence to suggest that active student learning, rather than passive learning, improves the quality of student learning and their learning experience. , Rather than focusing on rote memorisation of fact, active learning encourages students to accomplish higher-order objectives on Bloom's taxonomy, such as analysis, synthesis, and evaluation. Sapkaroski et al. found that novice students perceived VR to be just as effective as clinical role-play training in the skills labs and had the added advantage of being easily accessible and enabling users to correct mistakes at their own pace. Another study by Sapkaroski found that students who practiced extremity radiography using VR software performed better in a practical extremity examination after a three-week period than those trained using conventional simulated role-play during that time. The authors attributed this improvement to the inherent task deconstruction and the variety of visualisation mechanisms available in immersive VR environments. There has been widespread adoption of virtual simulation tools in radiation therapy education over the past decade.12, 13, 14, 15 However the same trends have not been observed in radiography education despite several simulators developed; Projection VR™ by Shaderware, Medspace VR™, Medical Imaging Training Immersive Environment (MITIE), Clinical Education Training Solution (CETSOL) VR Clinic™ and VR software by Virtual Medical Coaching Ltd. Literature on the efficacy of immersive 3D VR radiography simulation is limited as it has yet to be implemented in most undergraduate Radiography programmes. A literature search did not identify any peer-reviewed published empirical studies evaluating VR software by Virtual Medical Coaching Ltd. In this study, Virtual Medical Coaching Ltd.'s VR software was piloted as an educational tool for first year undergraduate radiography students. User feedback was retrieved to;

Determine whether students perceived this tool to be beneficial towards their confidence and skill development

Identify potential areas for improvement related to the VR software and its integration into the Stage 1 radiography curriculum

Establish student perspectives on the use of VR as an assessment tool

Method

Implementation of virtual reality into a Stage 1 radiography module

VR software (Virtual Medical Coaching Ltd) was validated by academic staff in advance of implementation into Stage 1 of the Radiography curriculum to ensure its suitability and alignment with Stage 1 learning outcomes. Students were provided with training videos on how to use the VR software. Face-to-face induction training in the VR suite was also provided to educate students on the use of both the software and hardware, which consisted of HTC Vive Pro™ headsets and hand controllers. There were two VR suites available to students. An online booking system was created by Dr John Stowe (Lecturer, University College Dublin), in which students were permitted to book four half-hour sessions each trimester. They were instructed bring a fellow student with them to observe their VR practice in order to flag trip or collision hazards during the session. The VR simulation tool was introduced to complement existing pedagogical approaches in the Stage 1 radiographic practice modules. Content covered in these modules consisted predominantly of projection radiography of the extremities, chest, abdomen, pelvis, spine and facial bones. These modules were delivered using mixed methods; four hours of lectures per week and a weekly one-hour small group practical session in the X-ray lab supervised by a graduate demonstrator. The VR radiography simulations guided students through a comprehensive process of learning anatomy, radiographic positioning, and pathology about each area of the body. Students took a brief quiz related to the aforementioned areas (Fig. 1 ). Upon completion of this quiz, they were referred a virtual patient to X-ray in the VR suite. Students were not limited in the number of times they could perform the procedure. See demonstration at https://www.youtube.com/watch?v=zL4UP3a47uc. Feedback was then sent to individual students in the form of the radiographic image they acquired and a short report on their performance. The report contained information on their performance in the following categories; gross pathology, regional patient position, regional anatomy, tube positioning, radiation dose, radiation safety and patient wellbeing. However, it did not segregate whether feedback in the report e.g. ‘poor’ patient positioning related to poorly answered quiz questions on the topic of positioning or actual positioning of the virtual patient in the VR simulation. A low-stakes (20% weighting) assessment using VR was incorporated into one module, in which students were required to perform an extremity examination using the VR system under direct supervision by academic staff.

Figure 1

VR user in action in the VR suite.

Evaluation of VR as a technological tool

An online survey (SurveyMonkey) was disseminated via email to all Stage 1 undergraduate students (n = 105) when they were midway through their second academic trimester. By this point, students had completed a two-week clinical placement block. The survey was used to evaluate students' perceptions of the VR radiography simulation software as an educational tool. Closed-ended questions based on a 5-point Likert scale (strongly agree, agree, neutral, disagree and strongly disagree) were included to encourage responses. A limited number of open-ended questions were included to allow further expression of opinion. The survey explored the students' opinions of Virtual Medical Coaching VR as a learning tool for developing their confidence and radiographic skills and as an assessment tool. Such feedback is also valuable in optimising integration of VR for future iterations of the curriculum. Feedback from students can be used to enhance the quality of education delivered and to ensure such education meets the learners needs. , Full ethical approval was obtained from the School of Medicine Research Ethics Committee in University College Dublin.

Results

The survey was distributed to 105 Stage 1 radiography students and had a response rate of 79% (n = 83). More than half of the respondents (58%, n = 48) enjoyed using VR (Fig. 2 ), whereas just 15% (n = 13) did not enjoy it, three of whom stated they did not yet feel comfortable using the technology.

Figure 2

Students' reported enjoyment levels using VR as part of their learning.

The mean length of time it took for students to feel comfortable using VR was 60 min ranging from 10 to 240 min. Only one quarter of the class of the class felt they had sufficient VR practice to support their learning, indicating that students desire more than four half-hour sessions per trimester. During this pilot study, there were only two VR suites available to this large cohort of students, thus bookings were limited. VR facilities have since been expanded to include a total of six 3D immersive VR bays and a computer-based version of the VR software that students can access remotely. This investment provides a real opportunity to maximise potential benefits of VR simulation-based learning and optimise its integration into our Radiography curriculum.

Fifty-seven percent found the ability to complete a procedure from start to finish without a clinical tutor present useful in developing their radiographic skills. Ninety-four percent would recommend the use of this VR simulation tool to other students. Many noted that VR could be expanded for use in other subjects, namely anatomy, CT, MRI and technology/radiation physics.

The majority of students agreed or strongly agreed that using VR developed their confidence in the following areas; centring and collimation of the X-ray primary beam, use of anatomical markers and selection of technical parameters (kVp, mAs, SID), image evaluation, knowledge of anatomy and patient positioning, as demonstrated in Fig. 3 .

Figure 3

Students' opinions on the benefit of VR in developing their confidence across several aspects of radiographic practice.

Most students felt VR did not contribute to their knowledge of patient dose tracking and radiation safety (Table 1 ).

Table 1

Student responses on whether the use of VR enhanced their confidence in their radiographic technique.

Areas of radiographic technique	Strongly disagree	Disagree	Neutral	Agree	Strongly agree	Total no. of responses	Weighted average
	Percentage response, student numbers in parenthesis
Collimation	0.00	12.16	12.16	56.76	18.92
	(0)	(9)	(9)	(42)	(14)	74	3.82
Use of Accessory equipment/markers	1.33	18.67	17.33	45.33	17.33
	(1)	(14)	(13)	(34)	(13)	75	3.59
Centring of X-ray tube to patient	5.19	12.99	16.88	50.65	14.29
	(4)	(10)	(13)	(39)	(11)	77	3.56
Exposure parameter selection	1.27	18.99	22.78	46.84	10.13
	(1)	(15)	(18)	(37)	(8)	79	3.46
Alignment of X-ray tube and Image Receptor	2.70	18.92	22.97	45.95	9.46
	(2)	(14)	(17)	(34)	(7)	74	3.41
Correct selection of SID	2.67	17.33	29.33	40.00	10.67
	(2)	(13)	(22)	(30)	(8)	75	3.39
Knowledge of Anatomy	5.26	21.05	32.89	31.58	9.21
	(4)	(16)	(25)	(24)	(7)	76	3.18
Determining Diagnostic Acceptability of Images	8.97	25.64	20.51	38.46	6.41
	(7)	(20)	(16)	(30)	(5)	78	3.08
Patient Positioning	13.58	30.86	16.05	27.16	12.35
	(11)	(25)	(13)	(22)	(10)	81	2.94
Practical skills in image processing	6.67	33.33	32.00	22.67	5.33
	(5)	(25)	(24)	(17)	(4)	75	2.87
Grid Use	12.33	36.99	26.03	16.44	8.22
	(9)	(27)	(19)	(12)	(6)	73	2.71
Knowledge of patient dose tracking	20.27	31.08	25.68	21.62	1.35
	(15)	(23)	(19)	(16)	(1)	74	2.53
Radiation Safety	17.57	43.24	24.32	9.46	5.41
	(13)	(32)	(18)	(7)	(4)	74	2.42

Other key issues flagged were technical glitches with the software, inability to palpate patient when positioning them and lack of feedback from the system. Students felt that the system should go one step further than providing them with the image as feedback but also some specific comments on the image e.g. indicating rotation, tilt, exposure and diagnostic acceptability of images (see Table 2 ).

Table 2

Thematic analysis of advantages and disadvantages of VR as an educational tool reported by Stage 1 Radiography students.

Students' Opinions on VR as an Educational Tool in Radiography
Advantages	Number of students
Enhanced knowledge of exposure factors	33
Opportunity to practice patient positioning	25
Critical analysis of resultant images is feasible	24
Flexible and accessible	18
Good tool for learning anatomy	16
Enjoyable, interactive form of learning	15
Ability to practice collimation	13
Ability to practice centring the X-ray beam to the patient	12
Can practice technique without associated radiation risks	9
Realistic simulation of a clinical environment	9
Ease of use	6
Reminds me to use anatomical markers	5
Reminds me to use grids when necessary	3
Disadvantages	Number of students
Patient already practically positioned and sometimes difficult to further position patient	33
Lack of feedback thus inability to know whether the image acquired was diagnostically acceptable	23
Technical glitches	22
Inability to palpate patient	17
Unrealistic e.g. due to lack of patient interaction	15
Lack of training	10
Limitations of the software
No lead shielding	6
Difficulty placing anatomical markers	4
Grid use difficult	3
Lack of patient variety	2

Discussion

The Virtual Medical Coaching VR simulation tool garnered a high level of satisfaction amongst first year undergraduate radiography students (58% enjoyed use, 27% neutral, only 15% disliked using it). Bridge et al. found that 34% of students enjoyed using MITIE computer-based simulation software to practice radiographic technique, while 46% felt neutral towards it and just 20% disliked it. The higher enjoyment levels reported in our study cannot be attributed specifically to the Virtual Medical Coaching VR software as the study demographics and VR delivery method varied. Bridge et al. evaluated student satisfaction amongst second year radiography students (n = 48). VR simulation-based learning is particularly useful in assisting students to master basic radiographic technique on ambulant, cooperative virtual patients, thus aligns strongly to Stage one learning outcomes. Additionally, the immersive VR experience in the VR suite may have been preferred over the computer-based MITIE software used by Bridge et al. Shanahan reported a slightly higher level of user satisfaction (65% enjoyed, 11% neutral) in a mixed cohort of radiography students (n = 84), most of which were mature students. The increased satisfaction levels may have been influenced by greater access to VR, as their computer-based software, Projection VR™ by Shaderware, was accessible remotely. Although the MITIE software was also computer-based, it could only be accessed by students on campus in supervised computer labs. Remote access provides flexible, self-directed learning opportunities for students. Ninety-four percent of respondents in this study reported that they would recommend Virtual Medical Coaching 3D immersive simulation to other radiography students. Based on the positive user feedback in this pilot study, we have invested in additional VR suites to provide further opportunities for student engagement in VR simulation-based learning. The computer-based version of this VR simulation tool will also be made available to our students in the next academic year. We anticipate this tool will be invaluable to our students during the COVID-19 global health pandemic, with a greater proportion of online education delivery required.

Virtual radiography simulation was deemed most useful in developing student confidence in the areas of X-ray beam centring, collimation, use of anatomical markers and selection of technical parameters. These findings correlate with improvements noted in practical assessments following VR training. , Gunn et al. stratified a first-year cohort into two groups; one group trained in the skills lab (n = 23) and the other group trained using computer-based VR simulation alone (n = 22). Students were then assessed performing foot and scaphoid projection radiography in the skills lab. Students who trained using VR performed better in X-ray beam centring, exposure factor selection, anatomical marker placement, patient positioning and equipment locks. Sapkaroski et al. carried out a similar study on a cohort of 76 students and found that the VR trained student cohort performed on average 11% better in patient positioning (p < 0.01) and 23% better in beam centring (p < 0.05) for posterior-anterior and oblique hand projections. Fifty-seven percent of respondents in our study found the process of completing a simulated radiographic procedure in its entirety (reviewing request card, positioning patient, setting up equipment, exposing and reviewing resultant image) beneficial to the development of their radiographic skills. Students selected which radiographic examinations to practice, thus tailored VR practice to their learning needs and could track their progress throughout the module, inspiring autonomous learning at an early Stage in the curriculum. Students were encouraged to critically evaluate the radiographic image produced and repeat the procedure amending their technique as many times as required until satisfied. Such reflective practice is integral to radiography education. Reflective practice promotes the development of clinical competencies in radiography students. Many students in our study felt VR also had a positive impact on their confidence in determining the diagnostic acceptability of images and their knowledge of anatomy. Ahlqvist et al. found that students’ skills in assessing image quality improved significantly after intervention using computer-based VR, which is a key factor in determining the diagnostic acceptability of images. Our students felt VR did not contribute to their knowledge of patient dose tracking and radiation safety. It is likely that first year students strongly associate lead shielding with radiation safety, as suggested by their feedback that shields were not available in VR. There are more effective ways of optimising dose such as beam collimation and appropriate exposure parameter selection, which may not be fully appreciated by novice radiography students. Furthermore, recent guidance advises against the use of lead shielding for most conventional radiographic examinations. The VR simulation software by Virtual Medical Coaching does not include an estimated dose-area-product for each examination, which would help students understand key concepts in dose optimisation. This feedback has been relayed to the software provider.

Most students recommended the use of VR in formative assessments but were strongly opposed to the use of VR for high stakes summative assessments. The National Forum for the Enhancement of Teaching and Learning in Higher Education in Ireland advocate assessment as learning in the form of an ungraded assessments focussed on self-monitoring, self-regulation, meta-cognition, learning and feedback, which is easily facilitated in VR. Students felt VR was lacking in realism in relation to key aspects of patient positioning such as inability to palpate bony landmarks and lack of patient interaction. Thus, students felt the clinical skills lab was more suitable than VR in recreating a realistic clinical scenario simulation for assessment purposes. Almost one quarter of the cohort experienced minor technical glitches when using the VR suite, which may have biased their perception of the reliability of VR for assessment. Most of the glitches noticed by students during this pilot study have since been resolved by the software provider. Students felt that VR was less realistic than clinical skills lab as there was no communication involved with the virtual patient, which correlates to student feedback in other studies. , Sapkaroski et al. implemented a haptic feedback virtual reality simulation with dynamic patient interaction and communication in the form of a Clinical Education Training Solution (CETSOL) VR Clinic™. Students perception scores regarding improvement of their clinical and technical skills were higher with the CETSOL VR Clinic™ than with the comparative benchmark simulation that did not provide dynamic patient interaction and communication. Even without the patient interaction element, VR simulation helps students to master their radiographic technique. Students that are more confident in their radiography technique may be in a better position to interact with patients during clinical practice, as the same degree of concentration on technique may not be required.

Although students enjoyed using this interactive simulation tool, many felt that the lack of feedback hindered their learning. The software provides a radiograph for each exposure that reflects technique carried out by the student and a brief performance report. The report does not distinguish between performance in the pre-VR quiz and the simulation itself. The first-year students in this study struggled to determine whether their technique was accurate based on the resultant image alone, perhaps due to lack of clinical experience. This process could be enhanced through provision of a benchmark ‘ideal’ image side-by-side with the radiograph they've produced for comparison. Based on this pilot study, the strategy for integrating VR into Stage one practice of radiography modules will be modified to incorporate academic staff feedback. Students will receive feedback via a grading rubric from staff evaluating the radiographic image they've produced. This will include feedback on specific aspects of the examination such as image quality, positioning (rotation, tilt etc), beam centring, collimation, anatomical marker placement and the overall diagnostic acceptability of the image. The virtual learning environment, Brightspace, facilitates this mechanism of feedback in an efficient, timely manner. Timely, focussed formative feedback is valued by students and helps guide their learning. This pilot study reiterated the importance of reviewing VR implementation strategies and the need for academic involvement to maximise its potential.

Study limitations

Students were limited in the amount of time they could spend using the VR simulation tool (four hours) due to limited VR equipment and a large class size, which may have negatively impacted the evaluation of this tool. Information on students’ technical abilities and gaming experience was not collected in this study, which may have impacted their experience as VR users. However this is unlikely as gaming experience had no influence on student perceptions of VR radiography simulation in previous studies. , While it is valuable to demonstrate student perceptions, these are subjective measures, and increased confidence is not synonymous with increased competence. In future studies, we will assess whether the Virtual Medical Coaching Ltd simulation tool resulted in actual improvements in technical skills.

Conclusion

It is widely accepted that simulation is a valuable pedagogical approach for diagnostic radiography education. The immersive 3D Virtual Medical Coaching Ltd simulation tool implemented in this pilot study was greatly valued by first year radiography students. Students felt more confident in their radiographic technique after using the tool and thoroughly enjoyed this interactive learning experience. However, it should be noted that VR was deemed a useful learning resource to supplement, not replace, existing clinical skills labs and clinical placement. The findings of this study will be used to improve and expand integration of VR in the next iteration of the undergraduate radiography curriculum.

Declaration of competing interest

None.