The effects of neonatal resuscitation gamification program using immersive virtual reality: A quasi-experimental study.

BACKGROUND: Clinical practice in neonatal intensive care units for nursing college students has been restricted due to the COVID-19 pandemic outbreak; thus, the gamification program has emerged as an alternative learning method. Consequently, there is a need to examine the effectiveness of such alternative learning methods to enhance the response to high-risk newborn emergencies.
OBJECTIVES: To examine the effects (neonatal resuscitation nursing knowledge, problem-solving and clinical reasoning ability, self-confidence in practical performance, degree of anxiety, and learning motivation) of a neonatal resuscitation gamification program using immersive virtual reality based on Keller's ARCS model.
DESIGN: A non-randomized controlled simulation study with a pretest-posttest design.
SETTING: Lab and lecture rooms of two universities in South Korea, from June to November 2021. PARTICIPANTS: Prelicensure nursing students.
METHODS: The virtual reality group (n = 29) underwent a neonatal resuscitation gamification program using virtual reality based on Keller's ARCS model. The simulation group (n = 28) received high-fidelity neonatal resuscitation simulations and online neonatal resuscitation program lectures. The control group (n = 26) only received online neonatal resuscitation program lectures. Changes in scores among these groups were compared using analysis of variance and analysis of covariance with SPSS for Windows version 27.0.
RESULTS: Post intervention, neonatal resuscitation knowledge [F(2) = 3.83, p = .004] and learning motivation [F(2) = 1.79, p = .025] were significantly higher in the virtual reality and simulation groups than in the control group, whereas problem-solving ability [F(2) = 2.07, p = .038] and self-confidence [F(2) = 6.53, p < .001] were significantly higher in the virtual reality group than in the simulation and control groups. Anxiety [F(2) = 16.14, p < .001] was significantly lower in the simulation group than in the virtual reality and control groups.
CONCLUSIONS: The neonatal resuscitation gamification program using immersive virtual reality was found to be effective in increasing neonatal resuscitation knowledge, problem-solving ability, self-confidence, and learning motivation of the nursing students who participated in the trial application process.

Introduction

The Bachelor of Science in Nursing curriculum is shifting from a content-based to a competency-based approach, emphasizing empirical learning for prelicensure nursing students to strengthen nursing competency (Brown et al., 2021). With the recent emphasis on patient safety and protection of rights, opportunities for nursing students to perform their duties in clinical practice have been limited, with a primarily observation-oriented practice (Shin et al., 2015). Practice in the neonatal intensive care unit (NICU) does not allow trainees' contact with patients because of the vulnerability of high-risk newborns and availability of specialized equipment (Kang and Yu, 2020). Moreover, clinical practice has been limited due to the COVID-19 pandemic outbreak, and alternative learning methods are being sought (Brown et al., 2021; Fogg et al., 2020; Ha and Lee, 2021).

Responding effectively to neonatal emergencies is critical for reducing newborn mortality and morbidity rates (Li et al., 2019). However, neonatal emergencies are stressful for medical staff, diminishing their decision-making abilities, which can negatively affect neonatal survival (Li et al., 2019). Various studies (Ghoman et al., 2020; Li et al., 2019; Shin et al., 2015) have verified that simulation-based education (SBE) helps strengthen emergency-response abilities of prelicensure nursing students, as direct training on high-risk newborns cannot be performed. However, in-person simulations are limited, as they entail resource-intensive characteristics and spatial-temporal constraints (Ghoman et al., 2020; Shin et al., 2019).

Gamification programs are a cost-effective and easily accessible simulation alternative, with the potential to improve medical professionals' knowledge, skills, and clinical performance abilities (Li, 2015; Pottle, 2019; Weiß et al., 2018). Furthermore, gamification programs are tools that can be used to acquire and apply knowledge by utilizing educational principles such as motivational effects and user-oriented education, iterative learning, and continuous feedback (Drummond et al., 2017). Virtual reality (VR) gamification programs for neonatal resuscitation education enable experiencing real-life problems without spatial-temporal constraints (Ghoman et al., 2020). However, VR programs require learners to initiate and sustain their learning and, hence, are limited in that they hinder learning immersion if they cannot induce learners' curiosity and learning motivation (Choi and Won, 2017; Radianti et al., 2020). Moreover, the limitations of gamification programs in education regarding health professions have been emphasized. Indeed, the inherent competitive characteristics of a game can intimidate some learners, and if the game instructions are not clear, it may cause a negative reaction (Bigdeli and Kaufman, 2017). Westera (2019) mentioned that lacking explicit framing can hinder in-depth insight and understanding, allowing only fragmentary learning. Therefore, when developing a VR gamification program, a theoretical framework that strengthens learning motivation and addresses the limitations of gamification learning should be used as an intervention strategy. According to Radianti et al. (2020), theoretical application is not considered when developing educational VR programs for improving learning effects. Keller's ARCS model (1987) was developed to induce and maintain learning motivation, and is used in various educational programs (Keller, 2016; Li and Keller, 2018; Malik, 2014). The use of this model was proposed in VR instructional design; it consists of four components: attention (A; induces and retains learners' attention), relevance (R; enables the recognition that the content learned is beneficial for achieving learners' goals); confidence (C; that learning can be well achieved), and satisfaction (S; provides rewards that meet learners' expectations; Keller, 1987). In previous studies (Goksu and Islam Bolat, 2021; Keller, 2016), the ARCS model was used for web-based instructional design, and learners' learning motivation and educational effects were improved. Qian (2014) noted that educational games using the ARCS model encourage voluntary participation, strengthen learning motivation, help experience joy in learning, and promote skill development.

The present study developed an immersive VR neonatal resuscitation gamification program using Keller's ARCS model (1987), which included neonatal resuscitation nursing knowledge, problem-solving and clinical reasoning ability, self-confidence in practical performance, degree of anxiety, and learning motivation, which have previously demonstrated significant effectiveness (Jeong and Choi, 2017; Nielsen and Harder, 2013; Shin et al., 2015; Yang, 2021). There are conflicting opinions about the possibility of replacing face-to-face simulation training in VR programs using immersive technologies (Ntaba and Jantjies, 2019). Some previous studies (Brady et al., 2018; Liaw et al., 2020; Succar et al., 2022) have demonstrated that VR programs have serious limitations in replacing face-to-face simulation training. In contrast, some other studies (Judd et al., 2021; Nas et al., 2020) have emphasized that VR programs can replace face-to-face simulation training. To verify the possibility of replacing face-to-face simulation training in VR programs in the healthcare education field, its effectiveness must be compared by including a face-to-face simulation group in the comparison. Therefore, to compare the abovementioned effectiveness, this study included three groups: a VR group, simulation group, and control group.

The hypotheses were as follows:

The neonatal resuscitation nursing knowledge of the VR group (i.e., participants who underwent the immersive VR neonatal resuscitation gamification program based on Keller's ARCS model), simulation group (i.e., participants who underwent the high-fidelity neonatal resuscitation simulation), and control group (i.e., participants who received an online neonatal resuscitation program (NRP) lecture) should increase after the intervention with the VR group showing greater improvement than both the simulation and control groups.

The problem-solving ability of the VR, simulation, and control groups should increase after the intervention with the VR group showing greater improvement than both the simulation and control groups.

The clinical reasoning ability of the VR, simulation, and control groups should increase after the intervention with the VR group showing greater improvement than both the simulation and control groups.

The self-confidence in performing neonatal resuscitation nursing of the VR, simulation, and control groups should increase after the intervention with the VR group showing greater improvement than both the simulation and control groups.

The anxiety of the VR, simulation, and control groups should decrease after the intervention with the VR group showing less anxiety than both the simulation and control groups.

The learning motivation of the VR, simulation, and control groups should increase after the intervention with the VR group showing greater improvement than both the simulation and control groups.

Methods

Study design

This quasi-experimental study applied a non-randomized controlled simulation with a pretest-posttest design.

Participants

The target population comprised prelicensure nursing students. Students with a cardiopulmonary resuscitation (CPR) certificate were included, while those who studied in other majors prior to admission in nursing, had professional experience in the clinical field for ≥1 year, or had prior experience in participating in simulation-related and VR game programs that would likely influence the effects of this study, were excluded.

The sample size was calculated using the G⁎power 3.1.9.2 (Faul et al., 2007). The significance level (α) = 0.05, power of the test (1-β) = 0.80, and effect size (f) = 0.80 in an analysis of covariance (ANCOVA) comparing the means of the three groups for hypothesis testing, indicated that the minimum sample size per group should be 22, which implied a total of 66 participants.

In this study, a total of 88 students (VR group = 31, simulation group = 28, control group = 29) were recruited. However, two VR group volunteers did not participate in the VR program, and three control volunteers responded to the questionnaire. Therefore, they were excluded from the study.

Therefore, our total sample size was 83, with 29 participants in the VR group, 28 in the simulation group, and 26 in the control group.

Study protocol

To establish the educational strategies for this program, the four components (attention, relevance, confidence, satisfaction) and 12 sub-factors (perceptual arousal, inquiry arousal, variability, goal orientation, motive matching, familiarity, learning requirements, successful opportunities, personal responsibility, intrinsic reinforcement, extrinsic rewards, and equity) of Keller's ARCS model (1983) were utilized (Fig. 1 ). For the immersive VR neonatal resuscitation gamification program, the VR program scenario and contents of intervention were established according to the Neonatal Resuscitation: 2020 American Heart Association Guidelines (Aziz et al., 2020) (Table 1 ).

Fig. 1

Components of Keller’s ARCS model as an intervention strategy for a virtual reality neonatal resuscitation gamification program. VR, virtual reality; NICU, neonatal intensive care unit.

Table 1

Contents and strategies of virtual reality neonatal resuscitation program based on Keller’s ARCS model.

NRP stage	NRP algorithm	VR scenario	Intervention contents
Assessment & NRP prep	Term gestationBreathing or cryingGood muscle tone	A situation in which a preterm baby born with premature rupture of membranes (PROM)a at 35 + 1 weeks has been relocated to a portable incubator in neonatal intensive care unit (NICU) and moved to the intensive care system (ICS).	Challenge presented: choose indicators where neonatal resuscitation program (NRP) implementation is required.As “take action,” a screen that assesses the necessity of NRP as is displayed by image; machines, instruments, and supplies required for NRP are selected.
A-block	Provide warmth	A situation in which the body of a newborn born at 35 + 1 weeks is cleaned with a warm towel to maintain its body temperature.	Challenge presented: choose how to maintain the body temperature of a newborn born at 35 + 1 weeks.As “take action,” virtual skill practice is performed utilizing a head mounted display (HMD) controller: radiant warmers (intensive care system) are applied.
	Position	A situation in which a thin towel is fixed under the shoulder of the baby into a sniffing position in order to position its posterior pharynx, larynx, and trachea in a straight line.	Challenge presented: choose a posture to maintain the airway of the newborn.As “take action,” image is displayed: animation character arranges the sniffing position.
	Clear airway	A situation in which a bulb syringe is used to remove secretions in the mouth and nose.	Challenge presented: choose the sequence of clearing the airway using a bulb syringe.As “take action,” virtual skill practice is performed utilizing an HMD controller: a bulb syringe is used to remove secretions in the mouth and nose.
	Dry, stimulate, reposition	A situation in which the newborn is cleaned again, the sole of the foot is tapped, the back is rubbed to provide stimulation, and the disarranged posture is repositioned (sniffing position).	Challenge presented: choose how to provide stimulation to the newborn.As “take action,” image is displayed: After turning the newborn into a lateral position, rub the back to provide stimulation.
A-blockEvaluation	Evaluate respirations, heart rate, and color	A situation in which the effectiveness of A block is reviewed and the necessity of implementing B block is evaluated.	Challenge presented: choose three evaluation items for Step A; choose which area to check for respiration in Step A; choose how many seconds to measure the heart rate.As “take action,” virtual skill practice is performed utilizing an HMD controller: measure the umbilical pulse to measure the heart rate.
B-block	Apply patient monitors	A situation in which the pulse oximeter and electrocardiography (ECG) are applied.	Challenge presented: choose which areas to place the pulse oximeter and ECG sensors.As “take action,” image is displayed: place ECG sensors.
	Provide positive-pressure ventilation	A situation in which the positive pressure ventilation (PPV) is applied.	Challenge presented: choose the appropriate mask size for the application of PPV.As “take action,” virtual skill practice is performed utilizing an HMD controller: PPV is provided.
B-blockEvaluation	Evaluate heart rate	A situation in which the effectiveness of B block is reviewed and the necessity of implementing C block is evaluated.	Challenge presented: if the heart rate is found to be 52 bpm according to the provision of B block, choose which stage to move to.As “take action,” virtual skill practice is performed utilizing an HMD controller: measure the apical pulse using a stethoscope and present results.
C-block	Provide positive-pressure ventilationAdminister chest compressions	Situation in which the PPV and chest compression are administered.	Challenge presented: choose the area, depth, and method of administering chest compression.As “take action,” virtual skill practice is performed utilizing an HMD controller: chest compression is provided.
C-blockEvaluation	Evaluate heart rate	A situation in which the effectiveness of C block is reviewed and the necessity of implementing C block is evaluated.	Challenge presented: if the heart rate is found to be 54 bpm according to the provision of C block, choose which stage to move to.As “take action,” virtual skill practice is performed utilizing an HMD controller: measure the heart rate by taking the apical pulse.
D-block	Administer epinephrine	A situation in which 1:10,000 epinephrine is inserted by ET-tube.	Challenge presented: choose the hand by which to hold the laryngoscope and the depth of intubation when performing endotracheal intubation.As “take action,” image is displayed: 1:10,000 epinephrine is inserted by an ET-tube.
	Insert Umbilical Vein Catheter (UVC)	A situation in which the UVC is inserted and 1:10,000 epinephrine is administered.	Challenge presented: choose the location to insert the UVC and the dose of epinephrine administration by IV.As “take action,” image is displayed: a UVC catheter is inserted in the umbilical vein and 1:10,000 epinephrine is administered.
D-blockEvaluation	Evaluate heart rate	A situation in which the effectiveness of D block is reviewed and the necessity of implementing D block is evaluated.	Challenge presented: if the heart rate is found to be 112 bpm according to the provision of D block, choose which stage to move to.As “take action,” virtual skill practice is performed utilizing an HMD controller: measure the heart rate by taking the apical pulse.

The immersive VR neonatal resuscitation gamification program, VR program scenario, and contents of intervention were established according to the Neonatal Resuscitation: 2020 American Heart Association Guidelines, as shown in Table 2.

NRP: neonatal resuscitation program; PROM: premature rupture of membranes; NICU: neonatal intensive care unit; ICS: intensive care system; HMD: head mounted display; ECG: electrocardiography; PPV: positive pressure ventilation.

Table 2

Homogeneity test of characteristics of subjects and dependent variables (N = 83).

Characteristics	Categories	Experimental group (n = 29)	Simulation group(n = 28)	Control group(n = 26)	χ2 or F	p
		N (%)	N (%)	N (%)
Sex	Male	2 (14.3)	4 (28.6)	8 (57.1)	5.77a	0.056
	Female	27 (39.1)	24 (34.8)	18 (26.1)
Satisfaction with nursing major	Dissatisfaction	2 (40.0)	2 (40.0)	1 (20.0)	4.14	0.387
	Neutral	9 (24.3)	13 (35.2)	15 (40.5)
	Satisfaction	18 (43.9)	13 (31.7)	10 (24.4)
Satisfaction with clinical practice	Dissatisfaction	4 (26.7)	4 (26.7)	7 (46.7)	4.08	0.057
	Neutral	10 (22.7)	17 (38.6)	17 (38.6)
	Satisfaction	15 (62.5)	7 (29.2)	2 (8.3)
Demand education mixed with technologies (VR, AR, MR)	Yes	27 (34.6)	26 (33.3)	25 (32.1)	0.32a	0.853
	No	2 (40.0)	2 (40.0)	1 (20.0)

Fisher’s exact test; VR, virtual reality; AR, augmented reality; MR, mixed reality.

The content validity index of each item (I-CVI) of the draft program scenario was verified by one head nurse in the NICU, one professor of pediatrics and head of the NICU, one professor of child nursing, one professor in the School of Emergency Medicine with experience in developing a VR gamification program, and one professor in the Department of Education. Only contents higher than an I-CVI of 0.8 were included in the program. The final program—an immersive VR neonatal resuscitation gamification program that enables hands-on experience in a virtual environment using a head-mounted display (HMD; Oculus Rift SⓇ)—was developed by a company specialized in VR program development. The content validity was verified by experts (one nurse in the NICU and one professor in pediatric nursing).

The study recruitment notice and uniform resource locator (URL) for the research agreement form were posted on online bulletins and social networking services of nursing university presidents across South Korea. The study only included students who completed the online written consent form and provided personal contact numbers. The URL of the pre-test was sent to the participants' phones one week before the intervention.

Intervention

For the VR group, the program ran for two weeks, between July and August 2021. In the first week, a URL to an NRP lecture (30 min) was provided to participants in all three groups. In the second week, the NRP immersive VR gamification program (50 min) was provided to the VR group in the classroom of a university in South Korea (Table 1; Supplementary Video 1).

For the simulation group, the program ran for two weeks, between September and November 2021. In the first week, a URL to an NRP lecture (30 min) was provided to participants. In the second week, the NRP high-fidelity simulation using a premature simulator (Laerdal® Premature Anne; pre-briefing: 10 min, simulation: 20 min, debriefing: 20 min, total: 50 min) was carried out in the simulation center of another university in South Korea.

For the control group, a URL to an NRP lecture (30 min) was provided to participants between June and July 2021 (Appendix 1).

Data collection

This study was approved by the University Institutional Review Board after reviewing its purpose, method, and guarantee of the participants' rights. As the participants included the researcher's students belonging to vulnerable subject groups, research assistants participated in obtaining consent and writing the research agreement in the absence of a principal researcher. Participants were informed they could withdraw from the study at any time, data would be used only for research purposes, and anonymity and confidentiality would be guaranteed.

The participants' general characteristics and primary outcomes (neonatal resuscitation nursing knowledge, problem-solving and clinical reasoning ability, self-confidence in neonatal resuscitation nursing performance, degree of anxiety, and learning motivation) were measured using online questionnaires a week before the first class. The participants' secondary outcomes (neonatal resuscitation nursing knowledge, problem-solving and clinical reasoning ability, self-confidence in neonatal resuscitation nursing performance, degree of anxiety, and learning motivation) were measured using online questionnaires within a week after the intervention. Individual text messages containing the URL to the follow-up questionnaire were sent to all participants, so that the post-test could be conducted in individual spaces from June–November 2021. Each survey took approximately 15–20 min to complete.

Instruments

Neonatal resuscitation nursing knowledge

The neonatal resuscitation nursing knowledge measurement tool has been standardized by Yoo (2013) through tests of reliability and validity, based on the requirements of the NRP by the American Heart Association (Hazinski et al., 2015). It comprises 30 questions on the following: appearance, pulse, grimace, activity, respiration (APGAR) score measurement (2), initial assessment and intervention (1), aspiration (3), oxygen saturation measurement (2), nasogastric tube insertion (1), oxygen supply (2), chest compression (7), endotracheal intubation (7), medication (4), and anatomical knowledge (1). Each question is processed as “incorrect or unknown (zero points)” or “correct (one point).” Higher scores indicate greater knowledge of neonatal resuscitation nursing. The Kuder–Richardson estimator formula 20(KR-20) was 0.87 at the time of tool development and 0.78 in this study.

Problem-solving ability

The tool measuring the problem-solving ability of adults by Lee et al. (2008) was used. This tool comprises 30 questions across five sub-fields (six items each): clarification of problems, search for solutions, decision-making, the performance of solutions, and evaluation and reflection. Each question is based on a five-point Likert scale (1 = not at all; 5 = very much so). Scores range from 30 to 150, wherein higher scores indicate higher problem-solving ability. The reliability of the tool (Cronbach's α) was 0.93 at the time of development and 0.94 in this study.

Clinical reasoning ability

This was measured using the Korean version of Nurses Clinical Reasoning Scale developed by Joung and Han (2017), originally developed by Liou et al. (2016). It consists of 15 questions measured on a five-point Likert scale (1 = not at all; 5 = very much so). Scores range from 15 to 75, wherein higher scores indicate higher clinical reasoning ability. Cronbach's α was 0.93 at the time of development and 0.96 in this study.

Self-confidence in performing neonatal resuscitation

This was measured using the tool standardized by Yoo (2013), through tests of reliability and validity based on contents that should be performed in the NRP by the American Heart Association (Hazinski et al., 2015). It consists of 15 questions: item inspection, initial intervention, APGAR score measurement, situational judgment, aspiration, pulse oximeter application, oxygen supply, nasogastric tube insertion, chest compression, medication, reporting, condition observation, documentation (one question each), and endotracheal intubation (two questions). Each question is measured on a five-point Likert scale (1 = not at all; 5 = very much so). Scores range from 15 to 75, wherein higher scores indicate higher self-confidence in performing neonatal resuscitation. Cronbach's α was 0.96 at the time of development and 0.95 in this study.

Degree of anxiety

This was measured using a tool translated into Korean by Kim and Shin (1978) from the State–Trait Anxiety Inventory (STAI) developed by Spielberger et al. (1983). It consists of 20 questions related to state anxiety, measured on a four-point Likert scale (1 = not at all; 4 = very much so). Scores range from 15 to 75, wherein a score of 30 or lower indicates low or no degree of anxiety, and a score of 31 or higher indicates a high degree of anxiety. Cronbach's α was 0.93 at the time of development and 0.90 in this study.

Learning motivation

The “Learning Motivation Test Paper” by Song and Keller (2001), translated from Keller's (1987) “Course Interest Survey,” was partially modified to suit this study's purpose with the authors' permission. It consists of five sub-fields with 30 questions (six questions each): clarification of problems, search for solutions, decision-making, performance of solutions, and evaluation and reflection. Participants respond using a five-point Likert scale (1 = not at all; 5 = very much so). Scores range from 30 to 150, where higher scores indicate higher motivation for learning. Cronbach's α was 0.82 in the previous study (Yu and Chae, 2005), and 0.88 in this study.

Data analysis

Data were analyzed using the statistical package PASW 27.0 for Windows (SPSS Inc., Chicago, IL, USA), through parametric analysis, as the participants were normally distributed. The pretest homogeneity of the general characteristics and dependent variables (newborn resuscitation nursing knowledge, problem-solving ability, clinical reasoning ability, self-confidence in performing neonatal resuscitation nursing, degree of anxiety, and learning motivation) of the VR, simulation, and control groups were analyzed using the chi-square test, Fisher's exact test, and one-way analysis of variance (ANOVA). The a priori and post hoc differences between the dependent variables as well as the a priori and post hoc mean differences of the three groups were analyzed using ANCOVA and Bonferroni-adjusted post hoc tests.

Results

General participant characteristics and homogeneity testing

There were 29, 28, and 26 participants in the VR, simulation, and control groups, respectively. The homogeneity of the general characteristics of the groups was verified, and there were no significant differences in sex, satisfaction with the nursing major, satisfaction with clinical practice training, and demand education mixed with technologies of VR, augmented reality, and mixed reality (MR; Table 2 ). Furthermore, the test of homogeneity of a priori dependent variables among the three groups demonstrated homogeneity, as there were no significant differences in neonatal resuscitation nursing knowledge, problem-solving ability, clinical reasoning ability, self-confidence in performing neonatal resuscitation nursing, and learning motivation. However, a significant difference among the groups was found in the degree of anxiety. The post-analysis found that the simulation group's degree of anxiety was significantly higher compared with the VR and control groups, and the homogeneity in the degree of anxiety of the three groups was not secured (Table 2).

Effectiveness in the VR, simulation, and control groups

The neonatal resuscitation nursing knowledge score of the three groups increased from pre-intervention to post-intervention (VR group: 12.52 ± 4.38 to 18.00 ± 2.55; simulation group: 12.79 ± 6.10 to 15.79 ± 5.43; and control group: 10.81 ± 4.35 to 11.85 ± 4.08). While there was no significant difference between the VR group and the simulation group, there was a significant improvement in the VR group compared to the control group. Hence, Hypothesis 1 was partially accepted.

The problem-solving ability score of the VR and simulation groups increased from pre-intervention to post-intervention (114.52 ± 14.16 to 122.72 ± 15.68 and 115.82 ± 17.88 to 118.25 ± 16.02, respectively), while that of the control group decreased from pre-intervention (114.54 ± 12.75) to post-intervention (106.24 ± 24.52). The VR group significantly improved in comparison to both the simulation and control groups. Therefore, Hypothesis 2 was partially accepted.

The clinical reasoning ability score of the three groups increased from pre-intervention to post-intervention (VR group: 45.83 ± 10.44 to 59.66 ± 9.44; simulation group: 49.75 ± 10.76 to 56.35 ± 9.97; and control group: 44.96 ± 10.86 to 53.69 ± 12.02). However, the VR group did not significantly improve in comparison to the simulation and control groups. Hence, Hypothesis 3 was partially accepted.

The self-confidence score of the three groups increased from pre-intervention to post-intervention (VR group: 46.45 ± 11.51 to 62.48 ± 8.48; simulation group: 50.00 ± 11.89 to 56.57 ± 9.21; and control group: 42.62 ± 12.90 to 47.00 ± 12.13). The VR group improved significantly in comparison to both the simulation and control groups. Thus, Hypothesis 4 was accepted.

The anxiety score of the three groups decreased from pre-intervention to post-intervention (VR group: 59.14 ± 9.62 to 56.72 ± 7.50; simulation group: 66.46 ± 8.60 to 57.65 ± 10.35; and control group: 59.50 ± 8.35 to 57.65 ± 6.86). However, the VR group did not significantly improve in comparison to the simulation and control groups. Hence, Hypothesis 5 was partially accepted.

The learning motivation score of the VR and simulation groups increased from pre-intervention to post-intervention (115.38 ± 12.84 to 117.69 ± 10.74 and 114.07 ± 13.26 to 114.12 ± 13.09, respectively), while that of the control group decreased from pre-intervention (111.31 ± 12.59) to post-intervention (102.00 ± 13.47). However, there was no significant difference between the VR group and the simulation group, and there was a significant improvement in the VR group compared to the control group. Hence, Hypothesis 6 was rejected.

The a priori and post hoc mean scores for neonatal resuscitation nursing knowledge and learning motivation were significantly different between the VR and simulation groups compared with the control group. The scores for problem-solving ability and self-confidence in performing neonatal resuscitation nursing were significantly different in the VR group than in the simulation and control groups, whereas those for the degree of anxiety were significantly different in the simulation group than in the VR and control groups. Analysis of the a priori and post hoc mean differences in the clinical reasoning ability of the groups showed no significant differences (Table 3 ).

Table 3

Comparison of dependent variables among experimental, simulation, and control groups (N = 83).

Variables	Pretest				Posttest				Mean Change
	VR groupM (SD)	Simulation groupM (SD)	Control groupM (SD)	ANOVAF (p)	VR groupM (SD)	Simulation groupM (SD)	Control groupM (SD)	Analysis of covarianceF (p)	VR groupM (SD)	Simulation groupM (SD)	Control groupM (SD)	Analysis of covarianceF (p)
Knowledge	12.52 (4.38)	12.79 (6.10)	10.81 (4.35)	1.22 (0.300)	18.00 (2.55)a	15.79 (5.43)b	11.85 (4.08)c	6.49 (<0.001) a.b.>c.‡	5.48 (4.13)a	3.00 (6.96)b	1.04 (5.53)c	3.83 (0.004) a.b.>c.‡
Problem solving-ability	114.52 (14.16)	115.82 (17.88)	114.54 (12.75)	0.07 (0.934)	122.72 (15.68)a	118.25 (16.02)b	106.24 (24.52)c	2.55 (0.035) a..>c.‡	8.2 (2.02)a	1.99 (4.85)b	1.31 (2.92)c	2.07 (0.038) a.>b.c. ‡
Clinical reasoning ability	45.83 (10.44)	49.75 (10.76)	44.96 (10.86)	1.57 (0.214)	59.66 (9.44)	56.35 (9.97)	53.69 (12.02)	1.38 (0.242)	8.21 (10.85)	6.60 (24.93)	8.73 (9.88)	2.79 (0.123)
Self-confidence	46.45 (11.51)	50.00 (11.89)	42.62 (12.90)	2.52 (0.087)	62.48 (8.48)a	56.57 (9.21)b	47.00 (12.13)c	3.21 (0.011) a.>c.‡	16.03 (9.77)a	6.57 (15.76)b	4.38 (16.52)c	6.53 (<0.001) a.>b.c.‡
Anxiety	59.14 (9.62)a	66.46 (8.60)b	59.50 (8.35)c	5.50 (0.006) b.>a.c.†	56.72 (7.50)a	57.65 (10.35)b	57.65 (6.86)c	5.53 (<0.001) b.>a.c.‡	−2.42 (10.29)a	−8.96 (11.59)b	−1.12 (9.63)c	16.14 (<0.001) b.>a.c.‡
Learning motivation	115.38 (12.84)	114.07 (13.26)	111.31 (12.59)	0.70 (0.498)	117.69 (10.74)a	114.12 (13.09)b	102.00 (13.47)c	5.59 (<0.001) a.b.>c.‡	2.31 (11.03)a	0.05 (15.47)b	−9.12 (17.12)c	1.79 (0.025) a.b.>c‡

Experimental group (n = 29); Simulation group (n = 28); Control group (n = 26); †In the preliminary homogeneity test of the experimental and control groups, the three groups showed no significant differences in their general characteristics and dependent variables except for anxiety; ‡Post-hoc: Bonferroni-adjusted test; a.b,c Different lowercase letters in lines indicate the statistical analysis results of the VR, simulation, and control groups in this study, and post-hoc mean differences and scores are summarized.

Discussion

This study investigated the effects of an immersive VR neonatal resuscitation gamification program to verify whether the high-risk neonatal nursing competency of nursing college students improved. First, the VR and simulation groups demonstrated no significant difference, but both groups showed an improvement in neonatal resuscitation nursing knowledge and learning motivation compared with the control group. A systematic literature review (Foronda et al., 2020) reported that participants in 13 of 17 VR simulation nursing education programs showed significant improvements in nursing knowledge. Wang et al. (2018) noted that immersive VR strengthened learning motivation by increasing learners' immersion in study and situational control. In the present study, a 3D immersive VR NICU was created. The program was designed for the self-directed learning of learners, with external stimuli blocked by an HMD, which appeared to strengthen learners' learning motivation. Qian (2014) noted that the video information and rules of educational games promoted educational effectiveness by stimulating players' learning motivation, enabling enthusiastic and active participation in learning. This study considered the individual characteristics of learners and the applicability of educational content and learning goals, when designing games. Specifically, to strengthen attention among the components of Keller's ARCS model (1987), this study sought to encourage learning motivation by using strategies to secure perceptual and inquiry arousal and variability in program development. Brown et al. (2021) and Gravina (2017) noted that competency-based programs in the nursing curriculum significantly affect higher academic performance compared with the traditional content-focused curriculum. Therefore, instructors must identify quiz scores and skill errors for each learner as well as areas that have not been understood, and provide additional opportunities to strengthen nursing students' competency and engagement in clinical experience.

In this study, problem-solving ability and self-confidence in performing neonatal resuscitation in the VR group significantly improved compared with the simulation and control groups. Wang et al. (2018) asserted that problem-solving abilities could be strengthened by repeatedly applying a game that presents questions and answers in a VR environment, supporting the findings of this study. Moreover, learners in the VR group were given opportunities to repeatedly perform neonatal resuscitation using HMD controllers in a VR environment. When incorrectly performed, the mistake was immediately highlighted and another opportunity was given. Such a learning design improved their self-confidence in performing neonatal resuscitation due to the immediately correcting behavior and experiences of success. The simulation group was also provided an opportunity to perform CPR using high-fidelity simulators (Premature AnneⓇ), but only post hoc debriefing was provided, without immediate feedback or coaching. Successful neonatal resuscitation experience did not occur, which is believed to have led to such results. Chancey et al. (2019) noted that self-confidence in performing pediatric emergency medicine improved when errors were resolved immediately in skills, and when opportunities to acquire new skills and practice them in a timely manner were given. However, in this study, feedback and debriefing methods, alongside environmental differences between virtual and face-to-face simulations, were applied differently to the VR and simulation groups. Thus, to identify differences in effectiveness between VR and face-to-face simulation in the future, differences in performance self-confidence must be analyzed after providing the same feedback and coaching.

The VR group did not significantly improve in comparison to the simulation and control groups. The degree of anxiety of the simulation group significantly decreased compared to the VR and control groups. Although the VR NRP provided to the VR group was in the form of a game, it is believed that the burden of using unfamiliar VR games and HMDs may have constrained the reduction of anxiety. Anxiety may have remained even after the intervention due to a cognitive gap felt because the VR group applied skills by pressing the HMD controller instead of performing them manually. Pre-briefing is important for reducing learners' anxiety (Sharoff, 2015). Therefore, when designing future VR programs, it is deemed necessary to provide pre-briefing that is structured and systematized according to the Healthcare Simulation Standards of Best Practice™ (Watts et al., 2021) by INACSL, allowing learners to use HMDs in VR to become familiar with the use of HMDs and VR SBE. Furthermore, it is necessary to establish an MR environment and utilize technology for hand-tracking systems in an integrated manner in order to provide opportunities to physically practice skills rather than by HMD controllers in an integrated environment; this may reduce learners' burden owing to unfamiliar environments or technical tools, enabling experiences similar to that in reality.

In this study, the clinical reasoning ability of the VR group did not significantly improve compared with that of the simulation and control groups. Bae et al. (2019) stated that a systematic debriefing protocol is necessary to enable learners to self-evaluate and reflect on simulated situations; such a protocol plays a significant role in improving clinical reasoning ability. Watts et al. (2021) emphasized the establishment of structured and designed debriefing and reflection processes, including structured purpose-oriented debriefing. Additionally, clinical reasoning ability is strengthened by greater exposure to various clinical cases (Plackett et al., 2020). As this study included only simple neonatal resuscitation cases, learners may not have gained sufficient experience to improve clinical reasoning. Rather than simply focusing on introducing innovative technologies and tools into the curriculum, it is necessary to allow learners to experience simulations of various cases and receive debriefing based on the INACSL Standards of Best Practices (Watts et al., 2021), when developing VR programs.

The limitations of this study and directions for future research are as follows: First, the long-term continuation effect of intervention could not be verified because the program's effect was verified only before and immediately after the intervention, with no further investigation. Hence, survey research measuring the long-term continuation effect is necessary to evaluate objective effects of the intervention program. Second, the differences in actual intervention competency reinforcement could not be identified, as only the measurement tool of a self-reported questionnaire was used to verify the program's effects. Research verifying effectiveness by observing competency in neonatal resuscitation skills, where hands-on skills are important, is necessary. As the ultimate purpose of program development is to assist newborn resuscitation, clinical effectiveness, such as neonatal morbidity and mortality, and educational effects must be identified.

Conclusions

With the COVID-19 pandemic outbreak, on-site training of nursing college students in the NICU has been restricted; there has thus been a need to develop creative self-learning content. To improve the competency of nursing college students in nursing high-risk newborns, we developed an immersive VR neonatal resuscitation gamification program based on Keller's ARCS model. Some positive effects were observed in the trial application process. Therefore, this may be a nursing education method that can replace SBE in areas with limited resources.

The following is the supplementary data related to this article.

Supplementary material

Supplementary Video 1

Actual training video. This video shows a student running VR NRP and practicing. This video was filmed with the student's prior consent.

CRediT authorship contribution statement

Sun-Yi Yang: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing. Yun-Hee Oh: Investigation, Project administration, Supervision.

Declaration of competing interest

None.