Self-directed video versus instructor-based neonatal resuscitation training: a randomized controlled blinded non-inferiority multicenter international study.

OBJECTIVE: To compare the efficacy of video-assisted self-directed neonatal resuscitation skills course with video-assisted facilitator-led course.
METHODS: This multicenter, randomized, blinded, non-inferiority-controlled trial compared two methods of teaching basic neonatal resuscitation skills using mask ventilation. Groups of novice providers watched an instructional video. One group received instructor facilitation (Ins-Video). The other group did not (Self-Video). An Objective Structured Clinical Exam (OSCE) measured skills performance, and a written test gauged knowledge.
RESULTS: One hundred and thirty-four students completed the study. Sixty-three of 68 in the Self-Video Group (92.6%) and 59 of 66 in the Ins-Video Group (89.4%) achieved post-training competency in positive pressure ventilation (primary outcome). OSCE passing rates were low in both groups. Knowledge survey scores were comparable between groups and non-inferior.
CONCLUSIONS: Video self-instruction taught novice providers positive pressure ventilation skills and theoretical knowledge, but it was insufficient for mastery of basic neonatal resuscitation in simulation environment.

Introduction

Neonatal mortality remains high in both developing and middle-income countries [1-4]. Perinatal asphyxia is one of the most common and preventable causes of infant mortality [5, 6]. Approximately 10% of neonates, or about 13 million infants per year worldwide, require some type of assistance to initiate breathing. Approximately 1% require advanced resuscitation techniques, including chest compressions, intubation, and medication administration, to survive [2, 7, 8].

The most important step in delivery room resuscitation is providing effective assisted ventilation [9-11]. Therefore, healthcare personnel who are positioned to resuscitate neonates must also be proficient in the knowledge, skills, and attitudes to deliver effective ventilation [12-15]. Training birth attendants in cardiopulmonary resuscitation (CPR) has been demonstrated to be a useful strategy in reducing perinatal mortality [1, 16, 17].

The American Academy of Pediatrics’ Neonatal Resuscitation Program (AAP NRP) has been taught for many years in over 100 countries [1, 18]. One of the difficulties associated with the implementation of this program, particularly in underserved areas or understaffed circumstances, is that it requires a trained instructor. While training programs may be more available in certain situations, such as in urban areas, 50% of the global population live in medically-underserved areas where instructors may not be readily available, making it challenging to provide clinicians with effective neonatal resuscitation training. Self-directed training that does not require the presence of instructors may be an alternative learning model, since most newborns requiring resuscitation at birth respond well to effectively executed basic initial steps of resuscitation [2].

Self-directed, video-based courses have been a useful tool to train providers in resuscitation skills [19-22]. Such self-directed instruction strategies have the potential to further scale and implement neonatal resuscitation more widely yet it is unknown if video-based self-instruction will work as effectively as the traditional instructor-led strategy. New educational methods should be rigorously evaluated before deeming them equivalent to established modes of training, such as face-to-face courses.

Instructor led, video-assisted and simulation-based practice is the standard method of instruction for the AAP NRP. The objective of this study was to compare the efficacy of two methods of training novice healthcare learners for neonatal resuscitation, a self-directed neonatal resuscitation skills course (Self-Video) and an instructor-led neonatal resuscitation skills course (Ins-Video).

Methods

This study was an international, multicenter, randomized, blinded, non-inferiority-controlled trial, registered at ClinicalTrials.Gov (NCT01847911) and approved by the IRB of each participating center.

Five academic centers participated in this trial: one in Chile, two in Argentina, and two in the United States. Medical and nursing students without previous training or experience in neonatal resuscitation were invited to participate. Potential trial participants were shown a brief introductory video explaining the overall study goals. Those willing to participate provided written, informed consent and were then scheduled in blocks of six.

To evaluate baseline resuscitations skills, each participant was asked to complete the relevant components of a commonly used and externally validated Objective Structured Clinical Examination (OSCE) shown in Appendix 2, following the guidelines established in the NRP’s Textbook of Neonatal Resuscitation, 7th Edition [23, 24].

The OSCE was scored by certified AAP NRP instructors for a maximum score of 26. The OSCE performances were video-recorded (without capturing the face to maintain privacy) and subsequently reviewed to ensure inter-reviewer reliability. Two NRP instructors independently evaluated all OSCE videos; they were blinded to both group assignment (Self-Video or Ins-Video) and to OSCE performance time (baseline, post-intervention/training). To address potential inter-reviewer discrepancies, a third blinded certified NRP instructor evaluated the videos.

Immediately following the baseline assessment, participants were randomly assigned to one of two treatment groups, in a 1:1 ratio. Participants were randomized, based on computer-generated randomization protocol, to either the Self-Video Group or the Ins-Video Group. Group assignment was concealed from the investigators by using opaque and pre-coded envelopes prepared by the statistician. Each group was allowed 2 h to complete the same instructional video and scheduled skills-practice stations. The total running time of the instructional video was 20 min. The video introduced each performance element then instructed participants to stop the video and practice this skill. Skills practice in the Self-Video group was independent and unmonitored whereas in the Ins-Video group practice was facilitated by the instructor.

Video creation

The training video, used in both trial groups, was designed and produced by the SAVER Study Group, comprised of NRP-certified instructors, video producers, design advisors, and medical education specialists. This 20-min video provided instruction on how, why, and when to begin the initial resuscitation steps and positive pressure ventilation (PPV) using a T-piece resuscitator. The skills taught in this video were based on and consistent with lessons one through four of the NRP’s Textbook of Neonatal Resuscitation, 7th Edition [24]. Pause points at critical teaching moments were integrated into the video in order to allow trainees to review and practice resuscitation skills. These scheduled pauses added 1 h and 40 min to the total training time. Students practiced these skills on a neonatal mannequin with the same device used for pre- and post-intervention OSCE evaluation. This video was available in both English and Spanish at all trial sites, and is currently available at: https://www.youtube.com/watch?v=Zs4eCWr1wgw&t=2s.

Self-Video group

Following Bandura’s Social Learning Theory, students were organized into groups of three during the training session [25]. Student groups were then shown the training video and instructed to pause the video at specified pause points in order to practice the resuscitation steps presented in that segment, using a neonatal-sized mannequin, T-piece resuscitator, gas source, warmer, and other essential medical equipment. Students assigned to the Self-Video Group did not have access to an instructor during the 2-h training session.

Ins-video group

Students in the Ins-Video group were also divided into groups of three and were trained using the same instructional video and skills practice sessions as the Self-Video group. However, this group had an NRP instructor who was physically present, to act as a facilitator during the training and practice exercises. The instructor was available to assist students, as would typically occur in instructor-led NRP skills training.

Assessment of skills and knowledge

To evaluate the immediate impact of the training on individual performance of basic neonatal resuscitation skills, each participant was individually assessed using the same instrument as the baseline OSCE. The same pre- and post-intervention test was used in both groups. Additionally, a written test was administered to evaluate immediate retention of basic newborn resuscitation knowledge (Appendix 3). This standardized knowledge test has frequently been used by the NRP to assess participants and has a maximum score of 18. Instructors read standardized instructions and administered all tests in the local language (English or Spanish).

Initially, the trial planned an assessment at 3-month for skills retention. However, many participants moved to new positions and were unavailable for follow-up.

Endpoint measures

The primary outcome measure was PPV Competency: successful demonstration of three critical PPV skills (initiating PPV, performing PPV, and correcting PPV) while using the T-piece resuscitator. These critical skills were part of the post-intervention OSCE [24]. Secondary outcomes were the scores on the OSCE and knowledge test immediately after training.

Sample size and statistical analysis

We hypothesized that a self-instruction, video-based training program would not be inferior to a standard instructor-led program in conveying knowledge and performance skills. For the primary outcome, success was defined as demonstrating all three critical PPV skills. A score of 70% or higher on the 26-point OSCE was considered success in attaining basic resuscitation performance, and earning a 70% on the knowledge test defined successful knowledge acquisition.

It was calculated that 214 participants would be needed to show non-inferiority, assuming no differences between groups, a success rate of 75% or higher for each group on the primary outcome, a tolerance limit of 15%, alpha of 5%, and 80% power. This estimation was based on published training studies [26]. Per-protocol population was defined as participants who completed pre- and post-training assessments. As the study progressed, we noticed much higher PPV Competency rates than expected (~90% per group, versus 75%), as well as difficulty in reaching the targeted sample size. We checked whether a lower sample size would achieve 80% power given the higher PPV Competency rates per group. The analysis showed that N = 100 would be sufficient for that goal. Comparisons between groups were performed by Chi-square test or t-test for categorical or numerical values, respectively (or a non-parametric alternative to the t-test when assumptions were not met). Concordance between observers was measured by Intraclass Correlation Coefficient (ICC). Values > 0.70 were considered “adequate concordance”.

Use of non-inferiority test

The primary goal of many clinical studies is to determine whether a new intervention is not worse than the currently accepted standard of care [27]. For this reason, the SAVER Trial employed a non-inferiority approach for assessing endpoint measures. For the non-inferiority analysis, one-sided 95% confidence intervals were calculated for primary and secondary outcomes. Hypothesis testing was performed by Dunnett–Gent Chi-square test or modified t-test. Non-inferiority assessment of post-intervention OSCE means was performed on least-squares means resulting from an Analysis of Variance (ANOVA) model that included center and pre-training values as covariates. ANOVA models for knowledge test results only included center as a covariate.

Results

From January 2016 to December 2018, 134 students participated in the study. Sixty-eight students were assigned to the Self-Video group and 66 students were assigned to the Ins-Video group. Table 1 provides the demographic characteristics, baseline skills and knowledge (PPV competency rates, mean baseline OSCE scores, and OSCE pass rates). Both groups achieved low scores and pass rates on the baseline OSCE. Groups were not statistically different on these baseline measures

Table 1

Baseline participant characteristics.

	Sample (n = 134)	Video-based training (n = 68)	Instructor-based training (n = 66)	p value
Age (years, mean ± SD)	25.15 ± 3.27	25.05 ± 3.33	25.25 ± 3.22	0.723
Males (n, %)	58 (43%)	31 (46%)	27 (41%)	0.585
Medical students (n, %)	106 (79%)	54 (79%)	52 (79%)	0.929
Nursing students (n, %)	28 (21%)	14 (21%)	14 (21%)
Students demonstrating PPV competency (n, %)	3 (2%)	3 (4%)	0	0.379
OSCE score (mean ± SD) (max. 26 points)	3.87 ± 4.97	3.86 ± 4.73	3.87 ± 5.25	0.990
OSCE > 70% of total score (n, %)	7 (5%)	3 (4%)	4 (6%)	0.900

OSCE Objective Structured Clinical Examination.

Data were compared by χ2 test or t-test.

Table 2 presents the primary endpoint measures of the trial. Approximately 90% of participants demonstrated post-intervention competency in providing PPV in both groups.

Table 2

Osce and knowledge test pass rates after training.

	Video-based training	Instructor-based training	Difference (one-sided 95% CI)
Students demonstrating PPV competencya (n, %)	63/68 (92.6%)	59/66 (89.4%)	3.3% (−4.9; 11.4)
Passing OSCE (n, %)	34/68 (50.0%)	34/66 (51.5%)	−1.5% (−15.7; 12.7%)
Passing Knowledge Test (n, %)	44/45 (97.8%)	44/45 (97.8%)	0% (−5.1; 5.1%)

Tolerance limit = 15%.

Non-inferiority for these outcomes were confirmed by the non-inferiority Dunnett–Gent Chi-square test.

OSCE Objective Structured Clinical Examination.

aPrimary outcome.

The proportion of students passing the knowledge test after the training was high in both groups, with Self-Video being non-inferior to Ins-Video (Table 2 and Fig. 1).

Fig. 1

Proportion of students meeting the study outcomes.

PPV Competency was the primary outcome of the trial. Non-inferiority was concluded for students demonstrating PPV competency and for students passing knowledge test, as the lower boundary of the confidence interval did not cross the tolerance limit (dashed line), but not for students passing OSCE. Non-inferiority was confirmed by the Dunnett–Gent Chi-square test.

Regarding the overall OSCE after the training, the proportion of students passing the assessment was low and comparable between groups. Non-inferiority was not demonstrated when comparing the proportion of passing OSCEs.

The non-inferiority adjusted comparison of OSCE and knowledge test scores immediately after the training are presented in Table 3. Self-Video average OSCE and test scores were non-inferior to Ins-Video group.

Table 3

OSCE and knowledge test averages. Non-inferiority assessment adjusting for center and pre-training scores.

	OSCE	Knowledge test
REF: Instructor-based training	17.61 ± 0.58 (n = 66)	16.28 ± 0.24 (n = 45)
INT: Video-based training	17.11 ± 0.57 (n = 68)	16.08 ± 0.25 (n = 45)
Difference ± Standard Error of the Meana	−0.49 ± 0.87	−0.19 ± 0.35
One-sided 95% confidence limits	−1.86 to 0.087	−0.78 to 0.39

Non-inferiority for these outcomes were confirmed a modified t-test for testing non-inferiority.

Tolerance limit = 15% (OSCE = 2.64, Knowledge test = 2.42).

OSCE Objective Structured Clinical Examination.

aAdjusted for center and, in the case of OSCE, also for pre-intervention values (ANOVA).

Assessment of inter-rater reliability of OSCE scorers showed an ICC of 0.776 (95% CI = 0.720–0.822). ICC between independent raters and center investigators was 0.779 (95% CI = 0.669–0.847).

Discussion

Basic neonatal resuscitation requires that providers recognize when to initiate and stop PPV and additionally have the skills needed to provide these procedures effectively. Although many educational videos have been developed and used to teach these skills, no research has compared the efficacy of using these videos alone with their use in combination with an in-person instructor [22, 28].

In the early part of the learning curve of PPV with a T-piece resuscitator, video self-instruction combined with independent practice in the present study was enough to assist most students in achieving PPV skills competency. The only differences between the two groups were the presence and interventions of the instructors. The video materials, learning environment, simulation equipment, time on task, and learner populations were the same in both groups. The similar learning outcomes may be explained by the relative ease of the T-piece resuscitator compared with the more commonly used self-inflating bag [29].

Having an instructor present might be more important when learning a more complex skill, such as neonatal resuscitation using the self-inflating bag. In addition, the video materials were designed for the specific intent of teaching resuscitation using the T-piece resuscitator and were produced after multiple reviews and revisions. Therefore, the video instruction may, on its own, have been effective enough to obscure the benefit of practicing with an instructor.

The OSCE and Knowledge Test scores (secondary outcomes) are a more granular assessment of competency and served to quantify additional learning in this study. Only half of the students in both groups achieved competence on overall basic neonatal resuscitation, as measured by the OSCE. This finding suggests that, as naive learners gain psychomotor skills and knowledge, the video serves as a teaching adjunct. Video alone does not seem to replace instructor-based training and other education strategies, beyond the acquisition of basic knowledge and PPV skills. Considering that these learners were nursing and medical students, we would expect they were accustomed to having their healthcare-related skills assessed. The local language was used in the assessments as well, so the assessment format and language would not explain the lower-than-expected performances.

Regardless of the type of training participants received, a 2-h training session is perhaps insufficient for the incorporation of the full interventional sequence required to successfully perform basic neonatal resuscitation and therefore successfully pass the OSCE. This finding opens areas for further research.

Continued assessment and investigation of instructional strategies are imperative. Cavicchiolo et al. recently showed that a full modified NRP training with in-person instructors was insufficient to prepare students to demonstrate resuscitation competency on a standardized assessment of these skills [30]. In addition, short refresher courses, even when shown to improve performance skills, were not sufficient to allow students to successfully demonstrate their competence in these skills.

Other studies evaluating the use of self-administered video training programs have shown inconsistent outcomes. For example, Weiner et al. found that the NRP training program could be shortened by the use of a training video, while maintaining performance skills [22]. On the other hand, Mpotos et al. found that the presentation of a single training video was not sufficient to allow pharmacy students to acquire the skills needed to successfully perform basic CPR on adults [31]. Another study found that high school students could be successfully trained on adult CPR using an educational video facilitated by teachers [32]. In a more recent publication, Heard et al. compared three different adult CPR training strategies. This study compared video only, video supplemented by skills practice, and short duration standard instructor-led training programs. While those who received the video-only training did not achieve competence at performing CPR, the other two training methods were equally successful at skills training [33]. In another recently published study, Stephan et al. found that video-only training was insufficient to successfully train medical students on pediatric CPR [34].

Both the present study and the existing literature suggest that, in addition to training provided either by video or an in-person instructor, hands-on skills performance is essential to reaching student competence on basic NRP. Combined with a team-oriented, hands-on skills practice using simulation equipment, a video, as designed for this project by an interdisciplinary team of experts, may be a useful complement for NRP training and therefore reduce the total duration of in- person training. However, depending on the expected outcomes, a combined strategy or extended time should be considered.

This study has several limitations. First, it did not achieve the number of participants needed to satisfy its initial sample size estimates. However, it took fewer subjects to demonstrate non-inferiority because the success rate on the primary outcome was higher than originally estimated. A second limitation was the low number of 3-month tests due to lack of follow-up or scheduling conflicts. Moreover, while the OSCE evaluation was standardized, potential differences may have existed due to stylistic teaching variations by instructors. These differences may have come into play in the video plus instructor training groups conducted across different centers in three countries. This difference was anticipated and corrected for using an adjusted analysis by center and by using two independent, blinded evaluators to assess OSCE performance, but should still be taken into consideration.

As alluded to previously, this study involved the T-piece resuscitator rather than the more common self-inflating air bag. Resuscitation competence using the T-piece resuscitator may not accurately predict training success of students using the self-inflating air bag [29]. Therefore, our results may not be universally applied to sites using another resuscitation device or those without access to compressed gas.

Finally, the PPV skills demonstrated on a mannequin may not correlate with effective PPV skills provided to patients in real life [26]. Conversely, the inclusion of a large number of students and the international, multi-center characteristics of the present study tends to increase its external validity and thus its generalizability.

In the future, video-based solutions may become more common and necessary. Video-based self-instruction strategies are in line with recently published American Heart Association scientific statement on resuscitation education science [35]. Personalized education, similar to trends to personalize healthcare, tailoring learner education strategies may yield greater learner competency, while allocating instructors and other resources after validation and further high-quality evidence [36, 37]. The need for instructional methods that are compatible with social distancing is clear during and after the COVID-19 Pandemic.

In conclusion, the present study found that instruction for novice providers via a video-based system without a facilitator present is not inferior to similar instruction with a facilitator for learning three critical PPV skills and theoretical content, though neither approach was sufficient for mastery of neonatal resuscitation in a simulated environment. Based on these findings, self-administered video training may be considered as a complement to the standard instructor-directed training programs for teaching basic newborn resuscitation.

Appendix 1

Appendix 2

Appendix 3