Evaluating the use of high-fidelity simulators during mock neonatal resuscitation scenarios in trying to improve confidence in residents.

OBJECTIVE: The purpose of this study was to investigate the effect of the use of high-fidelity simulators with multidisciplinary teaching on self-reported confidence in residents.
METHODS: A total of 26 residents participated in a session led by a pediatric faculty member and a Neonatal Intensive Care Unit transport nurse using a high-fidelity pediatric simulator. Multiple scenarios were presented and each resident took turns in various roles. Pre-intervention surveys based on a 5-point Likert-type scale were given before the scenarios and were compared to the results of identical post-intervention surveys.
RESULTS: Statistically significant (p < 0.05) pre to post gains for self-confidence were observed. Improvements in confidence were analyzed using the mean difference. The largest improvement in confidence was seen in the ability to treat neonatal respiratory arrest. This was followed by the ability to supervise/run a code, and the ability to place an umbilical venous catheter.
CONCLUSION: These results revealed that high-fidelity simulation-based training has significant positive gains in residents' self-reported confidence.

Introduction

Each year, approximately 4 million babies are born in the United States. About 10% of newborns require resuscitation by health-care providers at the time of birth, and less than 1% will require advanced resuscitation interventions.[1] The quality of resuscitation implemented by health-care providers in the first few minutes after birth correlates with the incidence of morbidity and mortality in the first 24 hours of life.[2] Therefore, it is essential that these health-care providers have the skills necessary to successfully carry out a neonatal resuscitation. In order to develop these skills, the Neonatal Resuscitation Program (NRP) was designed and is a standard accepted method for teaching neonatal resuscitation.[3] However, despite NRP training, studies have shown that the skills retained from these courses typically only last for 6–12 months.[1]

In order to become proficient in neonatal resuscitation, it is essential for health-care providers to routinely participate in real-life events. This poses a challenge for pediatric residents as the Accreditation Council for Graduate Medical Education (ACGME) and the Residency Review Committee (RRC) guidelines allow a maximum of 6 months of intensive care.[1] The restriction of duty hours placed upon residents also means there is less time for trainees to spend in the clinical environment, leading to a reduction in exposure to procedures and emergency situations.[4] In addition, the presence of other clinical personnel including nurse practitioners, physician’s assistants, and attending hospitalists decreases procedural opportunities for residents.[5]

In addition to the lack of opportunities faced by residents, pediatric procedural skills and decision-making pose unique challenges to residents due to the variability in a child’s anatomy, physiology, and behavior.[4] In comparison to adults, the pediatric airway is smaller in diameter as well as shorter in length. Second, the tongue of a child is larger in relation to the size of the pediatric oropharynx. The large size of the tongue can become an issue in intubation when trying to visualize the vocal cords. Finally, in both infants and toddlers, the larynx is cephalad in position.[6] All of these factors contribute to making resuscitation issues in pediatrics very difficult.

In pediatrics, the low number of resuscitations means that trainees receive inadequate exposure, which creates a feeling of unpreparedness.[4,7] Although there are many barriers to learning in the clinical environment such as time restrictions and competing for learning opportunities, simulation has been offered as a solution by providing residents and health-care providers with repetition and consistency.[8]

Simulations substitute real patient encounters through the use of artificial models, live actors, or virtual reality. The goal of simulation is to replicate patient care scenarios in a realistic environment in order to receive both feedback and assessment.[9] Past studies have shown that medical skills training which requires physical actions can be enhanced through actual practice in performing those actions.[10] According to Dull and Bachur,[7] the purpose of simulation is to educate through “active, repeated clinical experiences, giving and receiving immediate feedback, teaching leadership skills, and leveraging the controlled setting for predictable learning objectives, all while maintaining a safe learning environment.”

Past studies have been conducted in order to assess how health-care providers respond to mock codes through the use of simulation. In a study by Hunt et al,[11] simulation was used to evaluate the types and frequency of errors committed during pediatric mock codes. They found that resuscitation errors were more frequent than anticipated and occurred in every mock code in their study. Other studies have shown that residents who received simulator training scored better on both the Pediatric Advanced Life Support (PALS) written exam as well as during a mock resuscitation compared to residents who only received standard PALS training.[7] While simulation has been used throughout pediatrics to observe how health-care providers respond to mock codes, the purpose of this study is to determine the effect of simulation of neonatal resuscitation scenarios with a high-fidelity simulator on self-reported confidence in residents. As noted by Hegland et al.,[12] due to the fact that high-fidelity simulation training is quite expensive and resource demanding, it is important to study its effects.

Methods

We performed a sample size calculation using a two-tailed alpha of 0.05, power of 90% to detect a one-point difference in our Likert-type scale with an estimated standard deviation of 1, based on the responses obtained during our pilot field test of the questionnaire. This yielded a needed sample size of at least 23 for a study group. A total of 26 residents participated in a mock neonatal resuscitation session with Gaumard Scientific’s Newborn Hal, a high-fidelity pediatric simulator. Each resident took a five-question confidence survey immediately prior to the mock session.

We developed our survey as a modification of one published by Tofil et al.[13,14] which they based on the work of Cappelle and Paul.[15] We altered the questionnaire to be specific to neonatology. We then had two experts review the questionnaire to evaluate for internal validity and specifically for content validity. One of these reviewers is an expert in clinical and translational research, and the other specializes in quality improvement science. After this expert review, an internal field test was done in order to check for face validity. This test consisted of administering our questionnaire to 10 office staff with our objective being to determine whether or not the participants could understand and respond effectively to our questionnaire. We met this goal successfully which was expected in light of the fact that we had modified a previously used questionnaire. The criteria for pilot success were that from the perspective of the respondent:

The questions were unambiguous;

The questions were easily understandable;

The questions had appropriate answer options;

Instructions for completing the questionnaire were clear including how to indicate responses;

The questionnaire was easily completed in 5 min.

The pilot study met all these criteria and no changes were made to the questionnaire following the pilot.

The five-question survey was scored on 5-point Likert-type scale. Each participating resident was given a unique identifying number in order to compare individual resident’s responses on their pre- and post-survey. The survey consisted of questions regarding the resident’s perceived ability to perform various skills including the ability to perform intubations in neonates, ability to supervise or run a neonatal resuscitation, ability to treat neonatal respiratory arrest, ability to perform chest compressions, and ability to place an umbilical venous catheter.

Each mock session consisted of a small group of residents and was led by an attending physician and a Neonatal Intensive Care Unit (NICU) transport nurse. In each session, the residents were trained on how to set up equipment and how to execute procedures during the code using a high-fidelity mannequin. Multiple NRP scenarios were presented to the residents, and each resident took turns in various roles including: leader, airway, chest compressions, and umbilical venous line placement. The scenarios gave the residents the opportunity to use the Neo-Tee Infant T-Piece Resuscitator, suction, intubate, perform chest compressions, place umbilical venous lines, and give epinephrine and boluses.

A post-survey questionnaire was administered to the participants following the session. This survey was identical to the pre-survey questionnaire, containing the same questions and Likert-type scale. The post-survey questionnaire was used to evaluate for any changes in confidence in each component.

The data collected from the pre- and post-survey was analyzed using Graphpad Prism Version 7. Means were calculated for each individual question on the pre- and post-survey. An unpaired T-test was used to compare the means for each two groups. Given that data were nonparametric, medians were calculated, and the results compared utilizing Mann–Whitney analysis. For simplification, the final data were reported as means.

Results

The results from the resident’s pre- and post-survey questionnaires were collected and used to assess improvement in residents’ confidence in the scenarios using an unpaired T-test and Mann–Whitney test. A total of 26 residents participated in the pre-test survey, and those same 26 also participated in a post-test survey. A statistically significant improvement in confidence (p < 0.05) was seen in all evaluated aspects of the survey (Table 1).

Table 1.

Pre- and post-results.

			Pre-test answer (n = 26)	Post-test answer (n = 26)	Mean difference	P value
Q1	Ability to perform intubation in neonates	Mean ± SEM	2.423 ± 0.22	3.923 ± 0.19	1.500	<0.0001
		Median (25%, 75%)	1.75, 3.25	3.75, 5
		Strongly confident (%)	0	26.92
Q2	Ability to supervise/run a neonatal resuscitation	Mean ± SEM	1.885 ± 0.18	3.577 ± 0.20	1.692	<0.0001
		Median (25%, 75%)	1.2.25	3, 4
		Strongly confident (%)	0	15.38
Q3	Ability to treat neonatal respiratory arrest	Mean ± SEM	2.192 ± 0.20	4.038 ± 0.12	1.846	<0.0001
		Median (25%, 75%)	1, 3	4, 4
		Strongly confident (%)	0	19.23
Q4	Ability to perform chest compressions	Mean ± SEM	3.423 ± 0.17	4.462 ± 0.10	1.039	<0.0001
		Median (25%, 75%)	3, 4	4, 5
		Strongly confident (%)	0	46.15
Q5	Ability to place an umbilical venous catheter	Mean ± SEM	2.308 ± 0.22	4.000 ± 0.16	1.692	<0.0001
		Median (25%, 75%)	1, 3	4, 4.25
		Strongly confident	0	23.08

Self-assessed improvements in confidence were analyzed using the mean difference, which was calculated by taking the absolute value of the difference of the post-survey mean value from the pre-survey mean value. The largest improvement in confidence among the residents was seen in the ability to treat neonatal respiratory arrest (pre-test mean = 2.192 ± 0.20, post-test mean = 4.038 ± 0.12, p < 0.0001) which had the largest mean difference calculated at 1.846. Following the ability to treat neonatal respiratory arrest, the ability to supervise/run a neonatal resuscitation (pre-test mean = 1.885 ± 0.18, post-test mean = 3.577 ± 0.20, p < 0.0001) and the ability to place an umbilical venous catheter (pre-test mean = 2.308 ± 0.22, post-test mean = 4.000 ± 0.16, p < 0.0001), both had the second largest mean difference calculated at 1.692. Residents showed the least improvement in their confidence in their ability to perform intubations in neonates (pre-test mean = 2.423 ± 0.22, post-test mean = 3.923 ± 0.19, p < 0.0001) and their ability to perform chest compressions (pre-test mean = 3.432 ± 0.17, post-test mean = 4.462 ± 0.10, p < 0.0001), which had calculated mean difference scores of 1.500 and 1.039, respectively.

Finally, the percentage of residents who were strongly confident was examined for all evaluated aspects of the survey. This number was calculated by taking the percentage of residents that answered a “5” on the 5-point Likert-type scale for each individual question on the survey. Prior to the scenarios, the percentage of strongly confident residents was found to be 0%. Following the scenarios, all evaluated aspects of the survey showed an increase in the number of residents who identified as being strongly confident, with the greatest increase seen in the ability to perform chest compressions which was calculated at 46.15%.

Discussion

Neonatal resuscitation is a set of interventions conducted at the time of birth to help establish breathing and circulation. All pediatric residents should have the set of skills to successfully perform these interventions. With the restriction of duty hours and the presence of other clinical personnel, pediatric residents have less procedural opportunities to practice these interventions. Simulation is an important means to obtain these skills. While the effectiveness of simulation in improving skills and teamwork in medicine has been well documented in the literature, this study aims to examine whether simulation improves resident confidence in mock neonatal resuscitation scenarios. A novel aspect of our study was the use of a multidisciplinary team composed of a pediatric faculty member and an NICU transport nurse. Other studies involving resident education utilized fellows, general pediatric faculty or neonatologists to teach the scenarios, although one did acknowledge that prior to their study, it was often nurses who provided informal training prior to or during procedures.[5] Following the sessions several residents commented that it was particularly useful having the nurse present to explain the details of practical matters such as plugging suction into the wall and the functioning of a stop-cock. It has been shown that direct, hands-on practice with simulators improves skill effectiveness.[16] Further study may be needed regarding the efficacy of including nurses versus physician only teaching related to hands-on bedside skills.

This study shows that the use of high-fidelity simulation improved the confidence of residents with scenarios. More specifically, the use of high-fidelity simulation improved the confidence of residents when it came to performing intubations in neonates, running a resuscitation, treating neonatal respiratory arrest, performing chest compressions, and placing an umbilical venous catheter. We found that our residents gained the most confidence in their ability to treat neonatal respiratory arrest and that residents were most strongly confident in their ability to perform chest compressions. There is limited research specifically related to residents and neonatal resuscitation. Our study is most similar to one by Surcouf et al.,[1] which also used high-fidelity simulation training and evaluated residents’ self-confidence with neonatal resuscitation. While the question items were worded slightly differently, both studies basically showed significant improvement in confidence in relation to the ability to perform intubation in neonates, ability to supervise/run a neonatal resuscitation, ability to treat neonatal respiratory arrest, and the ability to perform chest compressions. Of note, our study showed significant improvement in confidence in the ability to place an umbilical venous catheter while the Surcouf study did not. Further study is indicated to evaluate if this may be related to our use of a multidisciplinary approach with much attention to detail by our neonatal transport nurse.

There are several limitations to this study that are worth mentioning. First, the results of our study may be limited by the use of self-reported measures of confidence. It is possible that residents may have over-reported their confidence. Second, this study is an analysis of simulated scenarios and may not represent responses during a real resuscitation. However, the low volume of real scenarios makes it difficult to assess resident confidence in these scenarios. Third, this study did not include a control group; however, residents were used as their own controls using pre- and post-intervention surveys. Finally, this study only assessed the residents’ self-reported confidence, but did not examine whether or not this correlated with clinical competence.

In conclusion, our study found that high-fidelity simulation is valuable in that it can improve resident confidence during mock scenarios. This increase in confidence could have benefits in terms of patient care if it correlates with an increase in clinical skills. In future studies, it would be important to examine this correlation in more detail.