E-learning use in the review of neonatal resuscitation program in physicians: a scoping review.

OBJECTIVE: To determine if e-learning interventions are efficient to review Neonatal Resuscitation Program (NRP) and to prevent performance deterioration in neonatal resuscitation of already-certified healthcare professionals. STUDY
DESIGN: In this scoping review, we searched for manuscripts published until June 2020 in five databases. We included all studies on e-learning use for NRP review in already-certified healthcare providers.
RESULTS: Among 593 abstracts retrieved, 38 full-text articles were assessed for eligibility. Five studies were included. Four studies evaluated the effectiveness of e-learning interventions immediately or months after their completion by providers. These interventions did not consistently enhance their NRP knowledge and their performance. One study showed that a growth mindset can influence positively neonatal resuscitation performance after an e-learning simulation.
CONCLUSION: There is not enough evidence to conclude that e-learning interventions can prevent neonatal resuscitation knowledge and performance decay in already-certified providers. More research is needed on the use of e-learning simulation-based scenarios to improve NRP retention.

Introduction

The Neonatal Resuscitation Program (NRP) is a widely recognized program proven to reduce mortality associated with perinatal asphyxia [1]. Unfortunately, medical residents and physicians are at risk of losing their learned NRP skills if they are not exposed frequently to neonatal resuscitation. In fact, the retention of knowledge and skills learned in standardized courses has been shown to significantly decrease 4–6 months after NRP certification [2, 3]. To prevent this knowledge and skill decay, the new 2020 NRP guidelines recommend booster training to improve neonatal resuscitation outcomes [4].

With the rapid evolution of information technology, e-learning has become a popular medium in medical education [5]. E-learning is the delivery of knowledge or training by electronic devices, such as a computer or mobile phone. A network can be used to facilitate the delivery of information and interaction between learners [6]. Many studies evaluated the outcomes of this new teaching method. Fontaine et al. conducted a systematic review and meta-analysis to evaluate the effect of adaptive e-learning environments (AEE) on knowledge, skills, and clinical behavior in health professionals and students. They concluded that AEE were more beneficial in improving learning skills rather than factual learning [7].

E-learning interventions are becoming more accessible, especially in the context of the COVID-19 pandemic. They have been widely used in limited-resource communities to improve the dissemination of more up-to-date medical knowledge. Bolan et al. reviewed literature on technology-based newborn health learning initiatives in low- and middle-income countries and found promising simulation and e-learning methods [8]. E-learning may be particularly useful to expose rural providers to rare high-acuity medical scenarios, such as neonatal resuscitation. In fact, a study of 26 rural hospitals showed that many healthcare professionals had not performed NRP skills in the last year. A correlation between frequency of skill performance and performance confidence level was also found [9]. The emergence of telemedicine may be an avenue to facilitate the education of remote physicians by experienced off-site instructors [9].

Multiple studies explored methods to slow down neonatal resuscitation knowledge and skills decay amongst healthcare providers. Bender et al. found that a simulation-enhanced booster session in neonatal resuscitation, completed 9 months after NRP, improved procedural skills and teamwork behaviors of residents rotating through neonatal units [10]. Scarce evidence exists around the benefits of e-learning for neonatal resuscitation review in healthcare providers already trained in NRP. The goal of this study is to review the literature on e-learning interventions to boost the knowledge and skills in neonatal resuscitation of already-certified healthcare professionals.

Methodology

Data collection and analysis were performed in accordance with the best practice PRISMA guidelines for scoping review [11]. PRISMA is the acronym for “Preferred Reporting Items for Systematic Reviews and Meta-Analyses”. PRISMA is the standard for reporting essential items of systematic reviews and meta-analyses. The review question, inclusion criteria, and methods for this scoping review were established in advance and documented in a protocol.

Search strategy

With a librarian trained in literature search, we conducted a search in five databases (Ovid Medline, Central, Embase, CINAHL, and ERIC) for all studies published until June 2020. Articles written in another language than French or English were excluded. No beginning date cut-off was set, and the last search was in June 2020. The references of included articles were also reviewed to look for additional studies. The details of the research strategy are available in the Appendix.

Study selection

Inclusion criteria

All studies on e-learning use for NRP review in already-certified healthcare professionals were included. Editorials, opinions, and reviews were excluded. Studies were excluded if their full texts were not available.

Outcome measures

The primary outcomes were knowledge and skill level in neonatal resuscitation.

The secondary outcomes were factors influencing the relation between e-learning intervention and knowledge and performance evolution.

Method of study selection

Study selection was performed by the principal investigator (SS). The articles were first screened based on title and abstract examination. Then the selected articles were read entirely by the principal investigator. The final eligible articles met inclusion criteria and reported at least one of the outcomes of the review.

Data extraction

Data from eligible articles were collected and organized in an extraction table designed for this review. This extraction table was based on the “JBI template source of evidence details, characteristics and results extraction instruments” [12]. It was filled by the principal investigator. The author recorded information about the authors, year of publication, study design, objectives, participants (characteristics and number), context (location and time of the study), intervention type, methodology, and outcomes of interest.

Analysis and presentation of results

E-learning methods of delivery being broad, sources of evidence were divided based on their mode of delivery (computer, smartphone or other communication means) and organized in a table. A descriptive summary is provided with the results and describes how these results are pertinent to the study question.

Results

Review decision process

We identified 937 articles using our search strategy and 2 from our review of reference lists of included articles. After removal of 346 duplicates, the title and abstract of 593 studies were screened based on our project’s inclusion criteria. 544 studies were excluded, and 11 studies were not included because only conference abstracts were available. The full texts of the remaining 38 studies were assessed for eligibility. 33 studies were excluded; 11 articles did not include an e-learning intervention, 12 had a study population that was not NRP-certified, 7 were trial protocols without published results, and 3 were literature reviews. As a result, 5 studies were included in our scoping review. Our review decision process is shown in a flow diagram (Fig. 1) adapted from the PRISMA flowchart, developed by Moher et al. [13].

Fig. 1

Flow diagram adapted from PRISMA 2009 flowchart.

Study characteristics

Study characteristics and key findings of the five included articles are summarized in Table 1. Four studies had an interventional design and the other had an observational design. All studies’ participants were already certified in NRP. Four studies included neonatal and pediatric physicians and medical trainees and two included other healthcare providers (nurses, respiratory therapists, and neonatal nurse practitioners).

Table 1

Characteristics and key findings of included studies.

Reference	Study design	Objectives	Participants	Intervention	Methodology adopted	Key findings
Computerized simulator system
Curran et al. [14]	Randomized pretest-posttest control group study	To evaluate the effectiveness of ANAKIN, a computerized simulator system, to boost neonatal resuscitation knowledge, skills, and self-reported confidence beliefs	Third-year, undergraduate medical students N = 60	Mannequin simulator integrated with a computer-based assessment program and videoconferencing technology (ANAKIN)	Students initially participated in an NRP workshop, filled out a knowledge test, and received a performance score They were randomized to receive ANAKIN system booster or NRP training video booster 4 months after NRP certification and filled a pre- and post-confidence survey and pre- and post-knowledge tests Both groups received ANAKIN system booster 8 months after certification. They filled out a pre-confidence survey, pre-knowledge tests, a user satisfaction survey, and received a face-to-face performance score	Knowledge: knowledge level significantly decreased at 4- and 8-month assessments. There was no significant difference between the two groups Performance: skill retention significantly decreased and did not differ significantly between the two groups over the 8-month period Self-confidence: confidence level increased significantly after booster exposure at 4- and 8-month assessments. There was no significant relation between self-reported confidence and performance Satisfaction: students indicated being highly satisfied with the educational technology system
Cutumisu et al. [15]	Cross-sectional study	To determine if growth mindset influences neonatal resuscitation performance	RN, RT, NNP, and neonatal consultants and fellows N = 50	Computer game training simulator, including a tutorial and three resuscitation scenarios (RETAIN)	Participants’ demographic data (experience and education) were collected in a pre-game form Number of tries at each scenario were compiled Growth mindset was assessed after the game	Growth mindset: growth mindset is a moderator of the relation between education (level of education and last NRP course) and performance
Smartphone application
Hawkes et al. [16]	Pre-post study	To determine if the NeoTube application improves knowledge and skills in neonatal intubation during simulation	Pediatric and neonatal residents and fellows N = 20	NeoTube application, including “Procedural Instruction” and “Quick Reference” components	Participants used NeoTube for 15 min and completed a pre- and post-knowledge-based questionnaire and video-recorded intubation simulation	Knowledge: intubation knowledge significantly increased after using the application for both residents and fellows Performance: intubation skills only significantly improved amongst the residents. Time for intubation was significantly decreased post-viewing the application in all participants
Video
Kaczorowski et al. [17]	Randomized controlled trial	To compare the effectiveness of two booster modalities to increase the knowledge and skills retention in neonatal resuscitation	Family medicine residents N = 44	Booster 1: supervised hands-on practice session with mannequins Booster 2: video review of NRP and unsupervised mannequin practice	Participants were divided into three groups: hands-on booster group, video booster group, and control group The hands-on and video groups received a “booster” 3–5 months after their NRP course Knowledge and performance were assessed in each group at 6–8 months through a written examination and five scenarios	Knowledge: neonatal resuscitation knowledge decreased without statistical significance between three groups Performance: neonatal resuscitation lifesaving and life-supporting skills decreased without statistical significance between three groups. All groups made errors in lifesaving skills on follow-up. Hands-on group made significantly less errors in the life-supporting skills than the two other groups on follow-up
Website
Stephenson et al. [18]	Pilot study	To assess competency retention of NNP in high-risk, low-frequency procedures, using a web-based content refresher	NNP N = 10	Web-based content refresher, including objectives, procedure indications, equipment list, and procedure videos	Participants consulted the website and completed a pre- and post-knowledge test. Their LMA placement skills and clinical judgment were assessed during simulation 6 months later, participants were randomized into two groups and completed a complex airway management recorded simulation. Only group 1 reviewed the website prior to simulation on follow-up	Knowledge: knowledge scores were not statistically different before and after viewing the website on day 0 of the study Performance: 4/5 participants placed LMA correctly in group 1 (website viewing followed by simulation), compared to 1/5 participants in group 2 (simulation only). This difference was statistically not significant

NRP Neonatal Resuscitation Program, RN registered nurses, RT respiratory therapists, NNP neonatal nurse practitioners, LMA laryngeal mask airway.

Each study used an e-learning intervention reviewing NRP or components of neonatal resuscitation. Computerized simulator system intervention was evaluated in two studies. Curran et al. used the ANAKIN system, a mannequin simulator instrumented to provide clinical signs remotely controlled by an instructor and to monitor the effectiveness of resuscitation maneuvers [14]. A two-way audio and visual communication between trainee and instructor (videoconferencing) and a computer-mediated assessment program were integrated into this system. Cutumisu et al. developed a neonatal resuscitation training computer game simulator, RETAIN [15]. This simulation game included a short in-game tutorial and three simulation scenarios of increasing difficulty (blocked airway requiring suctioning; mask ventilation and chest compression, with recovery after performing chest compression; mask ventilation, chest compression, and epinephrine use, with recovery afterwards). Hawkes et al. tested a smartphone application intervention, named NeoTube [16]. This application consisted of two main components and combined 29 pages, 7 videos, and 19 images. The first component, the “Procedural Instruction” section, included information on airway anatomy, and on neonatal intubation indications, premedication, equipment, procedure, and complications. The second component, the “Quick Reference” section, provided information on calculations and a faster access to videos and images from the first section. Kawzorowski et al. used two “booster” interventions [17]. The video booster, an e-learning strategy, included a video review of NRP paired with unsupervised mannequin practice. Finally, Stephenson et al. created a web-based content refresher [18]. This website described LMA (laryngeal mask airway) advantages, indications, and contraindications. It showed an equipment list with a pictorial procedural guide. Videos of LMA placements were also included.

The goal of the four studies was to evaluate the effectiveness of their e-learning intervention to boost neonatal resuscitation knowledge and performance of healthcare professionals. Participants in Hawkes et al. and Stephenson et al. studies were exposed to the intervention on the same day as their knowledge and performance assessment. On the other hand, participants in Curran et al. and Kaczorowski et al. studies were evaluated 4 months and 1–5 months after receiving the booster intervention, respectively. Curran et al. also studied the influence of their intervention on participants’ self-reported confidence and the participants’ satisfaction with the ANAKIN system. Cutumisu et al. were the only authors who looked at the relation between an incremental theory of intelligence, growth mindset, and neonatal resuscitation performance after simulation. Growth mindset is defined by the belief that intelligence can be enhanced with effort and learning.

Study results

Neonatal resuscitation knowledge

Knowledge deterioration after NRP certification was illustrated in two studies. Curran et al. showed that knowledge levels deteriorated significantly and similarly in the two groups 4 months after an NRP workshop. The level of knowledge of participants remained unchanged at the 8-month interval, which suggests exposure to video or computerized simulator system did not enhance NRP knowledge. In Kaczorowski et al. study, average initial NRP knowledge written score was 223 (223/239 = 93%; SD = 10%) compared to an average of 191 at the follow-up test (191/239 = 80%; SD = 16%). The follow-up test scores were significantly lower (P < 0.0001) in three groups, despite exposure to a video or hands-on booster in two groups.

Knowledge enhancement immediately following an e-learning intervention was not consistently observed. Hawkes et al. found neonatal intubation knowledge significantly increased after using the NeoTube smartphone application. Knowledge questionnaire median score of all participants was 18.5 (range 8–28) previewing of NeoTube and 31 (range 24–35) post-viewing (P < 0.001). On the other hand, Stephenson et al. showed no statistical difference in LMA placement knowledge score after viewing a web-based content refresher (P = 0.067).

Neonatal resuscitation performance

Neonatal resuscitation skills deteriorated significantly in the months following NRP certification. Curran et al. found a significant deterioration in the level of neonatal skill retention for both groups 8 months after NRP workshop (P < 0.05). In Kaczorowski et al. study, all participants did an error-free performance in lifesaving skills at baseline and then committed lifesaving skills mistakes 6–8 months later, regardless of their group. However, the participants in the hands-on group made significantly fewer errors at follow-up compared to the combined control and video groups averaged in five scenarios (72% and 63%, respectively, P = 0.021).

Specific neonatal resuscitation skills were enhanced immediately after exposure to an e-learning booster in two studies. In Hawkes et al. study, the median skill score improved significantly from 10 (9–14) to 12 (9–15) (P = 0.048) in the medical residents’ group. Skills improvement was not significant within the fellows’ group, since the skill score increased from 12.5 (9–15) to 13.5 (10–16) (P = 0.154) after NeoTube viewing. Time taken to intubate was significantly reduced from a median of 39–31 s in all participants (P = 0.044). In Stephenson et al. study, LMA placement was more successful in the group exposed to a web-based refresher before the procedure (4/5 participants) compared to the group not exposed to the intervention (1/5 participants). This difference was not statistically significant (P = 0.058).

Self-reported confidence

Curran et al. study indicates self-reported confidence of participants improved significantly after their exposure to the ANAKIN system and the video (P < 0.001). At the 8-month interval, a nonsignificant correlation was found between the knowledge and performance score and the self-reported confidence.

User satisfaction

Curran et al. showed students were positive about their experience with ANAKIN system. Approximately 81.3% of participants agreed or strongly agreed that it improved their understanding of neonatal resuscitation and 87.5% agreed or strongly agreed that the ANAKIN system was a useful training tool. Seventy-five percent agreed or strongly agreed the ANAKIN system had better prepared them to manage a future neonatal emergency.

Growth mindset-moderated performance

A multiple linear regression was completed by Cutumisu et al. to evaluate if participants’ growth mindset moderated the relationship between the time since their last NRP course and their performance at the RETAIN simulation game. A robust correlation showed that the more recently the participants underwent their NRP course, the more tries they needed to finish the game. However, this association was only significant in participants that endorsed lower levels of growth mindset. Thus, there was a significant interaction between the last NRP course timing and growth mindset in predicting the number of attempts needed by healthcare providers during the game. Fewer mistakes were also noted in participants with a higher growth mindset. Growth mindset was also a moderator in the association between the level of education and performance.

Discussion

This scoping review aimed to explore e-learning use to review neonatal resuscitation in healthcare professionals already certified in NRP. Two studies reported neonatal resuscitation knowledge and skills significant deterioration in the months following NRP certification. Two e-learning strategies, a computerized simulator system in Curran et al. and a video refresher in Kaczorowski et al., did not prevent this progressive decline. Knowledge and skill deterioration in healthcare professionals are observed and studied in other acute care settings. Interestingly, resuscitation skills decay has been found to be more pronounced than knowledge loss. Binkhorst et al. found retention of pediatric basic life support skills was poorer amongst pediatrics staffs and residents, compared to knowledge retention [19]. Physicians are more likely to be exposed to theoretical aspects of pediatric resuscitation through textbooks, case discussions, and simulation observation than hands-on practice.

Hawkes et al. and Stephenson et al. studied whether a smartphone application and a web-based content refresher, respectively, could improve knowledge and skills of neonatal resuscitation procedures just before they are performed. Knowledge enhancement was only significant in Hawkes et al. study. The exact format and delivery mode of these two interventions were not provided by the authors, which limits the interpretation of this finding. The NeoTube application was possibly simpler to navigate and contained briefer and well-organized information, which are essential in a stressful resuscitation environment. Also, smartphone applications can be used at the bedside and can be more accessible than websites formatted for computers. Inability to demonstrate significant findings can also be explained by the small sample size of Stephenson et al. study. On the other hand, performance improvement after e-learning intervention was reported in Hawkes et al. and Stephenson et al. studies. This conclusion is consistent with existing literature on “just-in-time” (JIT) practice-learning approach. The JIT method is increasingly used to strengthen teamwork and technical skills just before the time in which they may be required [20]. For instance, Niles et al. showed that frequent “just-in-time” cardiopulmonary resuscitation (CPR) simulations were effective to optimize the time needed to achieve CPR skill success in pediatric intensive care unit (PICU) staff [21]. The JIT method is also being studied to support procedural training in various acute care settings, such as intubation in PICU [22]. It is important to mention that neonatal intubation performance only significantly improved amongst the residents’ group. This is likely because neonatal fellows have more experience in this procedure compared to residents and require more advanced training than a smartphone application to refine their intubation technique.

Knowledge and skill enhancement differ widely between the results of Curran et al. and Hawkes et al. This difference may be explained by the complexity of the booster intervention. Participants in Curran et al. study received an NRP tutorial. In comparison, participants in Hawkes et al. study reviewed neonatal intubation. It is easier to review a single procedure compared to a multistep algorithm that includes this procedure. Therefore, participants of Curran et al. study may have had better results if they would have been exposed to a single component of NRP resuscitation.

Curran et al. showed that self-reported confidence beliefs increased significantly after exposure to a booster. However, no relation was observed between confidence in neonatal resuscitation and students’ knowledge and performance. This finding limits the significance of self-reported confidence to estimate the competency of learners.

Instead of assessing the performance of participants after a booster intervention, Cutumisu et al. study evaluated the influence of mindset on their performance. This study was included in this review to show the importance of growth mindset as a moderator between learning and performance. In other words, if trainees believe their abilities can improve with effort and study, they are more likely to maintain their skills.

This scoping review has some limitations. First, only one author participated in the review decision process. No discussion was possible when the author was unsure of the pertinence of a study that could have led to a selection bias. Second, 11 studies were excluded because they were conference abstracts and their full text was not available. This limitation brings a possible publication bias. In the future, these studies’ authors could be contacted, and their most recent results could be asked to complete this scoping review. Third, the sample size of included studies was small, and participants were recruited within a limited number of centers. The number of included studies in our scoping review was also small. This could limit generalizability of the results of our scoping review.

In conclusion, there is not enough evidence to support that e-learning interventions can prevent the deterioration of neonatal resuscitation knowledge and performance in already-certified healthcare providers. The results of this review suggest that booster intervention just before procedures may increase the knowledge, and especially the performance, of practitioners. Also, promotion of growth mindset within medical training programs may facilitate the acquisition of knowledge and skills. More research is needed to evaluate if e-learning simulation-based scenarios can improve the retention of NRP in healthcare practitioners.

Appendix