Rolling-refresher simulation improves performance and retention of paediatric intensive care unit nurse code cart management.

INTRODUCTION: Paediatric Intensive Care Unit Nurses (PICU RNs) manage the code cart during paediatric emergencies at the Children's Hospital at Dartmouth-Hitchcock. These are low -frequency, high-stakes events.
METHODS: An uncontrolled intervention study with 6-month follow-up. A collaboration of physician and nursing experts developed a rolling-refresher training programme consisting of five simulated scenarios, including 22 code cart skills, to establish nursing code cart competency. The cohort of PICU RNs underwent a competency assessment in training 1. To achieve competence, the participating RN received immediate feedback and instruction and repeated each task until mastery during training 1. The competencies were repeated 6 months later, designated training 2.
RESULTS: Thirty-two RNs participated in training 1. Sixteen RNs (50%) completed the second training. Our rolling-refresher training programme resulted in a 43% reduction in the odds of first attempt failures between training 1 and training 2 (p=0.01). Multivariate linear regression evaluating the difference in first attempt failure between training 1 and training 2 revealed that the following covariates were not significantly associated with this improvement: interval Paediatric Advanced Life Support training, interval use of the code cart or defibrillator (either real or simulated) and time between training sessions. Univariate analysis between the two trainings revealed a statistically significant reduction in first attempt failures for: preparing an epinephrine infusion (72% vs 41%, p=0.04) and providing bag-mask ventilation (28% vs 0%, p=0.02).
CONCLUSIONS: Our rolling-refresher training programme demonstrated significant improvement in performance for low-frequency, high-risk skills required to manage a paediatric code cart with retention after initial training.

Introduction

Simulation training has been widely accepted as a promising method for training healthcare professionals. Multiple systematic reviews of simulation training for procedures suggest that simulation improves procedural performance with patients, increases provider confidence and has the potential to reduce patient complications.1–11 Despite these findings, the most effective and efficient methods of simulation training remain unknown. At the Children’s Hospital at Dartmouth-Hitchcock (CHaD), the Paediatric Intensive Care Unit Nurses (PICU RNs) manage code carts during emergencies. Paediatric emergencies are rare and therefore it is necessary to simulate these scenarios to ensure adequate response times and competency in both individual and team tasks when an event does occur. A multicentre cohort study of in-hospital paediatric cardiac arrest demonstrated that a delay in administration of epinephrine was associated with a decreased chance of survival, decreased chance of return of spontaneous circulation, 24 hours survival and survival to hospital discharge with favourable neurological outcome in patients initially with a non-shockable rhythm. The risk increased with each minute in delay in receiving epinephrine.12 This study highlights the importance of using optimal training methods for paediatric nursing staff for achieving emergency skills, such as preparing and delivering epinephrine. Currently, there are no simulation studies that examine the role of rolling-refresher simulation training on nurse knowledge or competency on managing the code cart either in the paediatric or adult critical care setting.

Our research project was an uncontrolled intervention study with 6-month follow-upwith the study subjects being the entire PICU nursing staff. This method ensured that all PICU RNs were trained to mastery in managing the code cart during emergencies. We specifically decided against randomisation to training as we felt all nurses in our PICU should receive this assessment and training. ‘Managing the code cart’ consisted of preparing and delivering weight based emergency medications, preparing appropriate delivery systems and applying emergency equipment correctly and efficiently. Several time-dependent and life-saving skill-training scenarios were developed and used to train the PICU RNs to mastery. The in situ rolling refresher training allowed nurses to be trained during regular work hours, thus optimising time utilisation and improving overall value. The training programme also allowed nurses to familiarise themselves with the exact equipment that they would be expected to use in live situations on real patients.

Our Advanced Practice Nurse Practitioners (APRNs) served as expert instructors for this project and serves as another example of how APRNs improve patient safety for paediatric patients at CHaD. Our overarching hypothesis is that expertly trained nurses managing the code cart during emergencies will improve overall emergency team performance and this can be achieved using a rolling refresher in situ training programme with continued assessment and training every 6 months. Our specific hypothesis was that there are significant deficiencies in current PICU RN nursing competency on managing the paediatric code cart during emergencies and our training programme can identify and train to mastery on these life-saving skills with retention of these skills at 6 months.

Methods

The Dartmouth College Institutional Review Board approved this research project.

Design

We performed an uncontrolled intervention study with 6-month follow-up. Five simulated scenarios, including 22 code cart skills, were developed based on Paediatric Advanced Life Support (PALS) algorithms and in collaboration with paediatric critical care physicians and nurse practitioners using a modified Delphi process.13 Scenarios included shock, hypoglycaemia, respiratory distress/failure, ventricular fibrillation and supraventricular tachycardia. The specific skills within each scenario varied in complexity. All training sessions were performed from May 2014 through June 2015. Scenarios were piloted by several members of the study team who were considered paediatric code cart experts and refinements were made to establish content and face validity of the scenarios and skills. Given no national standards for time limits exist for any of the skills we were testing, we choose times based on the expert teams experience. For each task, expected actions were defined based on nursing standards.

Sample and setting

See online supplementary appendix 1 for all of the details of our training programme including descriptions of the scenarios as well as each task within the scenario with the expected actions and time limits for all tasks. Training sessions were held during scheduled clinical shifts, in an empty PICU patient room, with the trainee’s patients being cross-covered by another PICU RN. On a day where staffing and acuity allowed, an unoccupied patient room would be set up with a Paediatric Code cart, defibrillator, a syringe pump and infusion pump and both an infant and child manikin in addition to the normal supplies in each individual room.

Detailed description of rolling refresher simulation training

Training 1

Prior to each training session, participants completed a pretraining questionnaire to assess nursing characteristics as well as prior code cart exposure and training (see online supplementary appendix 2). The following is an example of how the assessment and training programme were performed. The shock scenario started with the following description read out loud, ‘A 15 kg 1 year old is presenting with tachycardia, hypotension and fever. You will be responsible for using the code cart to manage this patient.’ There were four tasks in this scenario. Task 1 read, ‘obtain supplies for IV placement in a 15 kg child. Also obtain equipment for drawing a blood gas, CBC, and BMP.’ It was expected that the nurse would find gloves, an intravenous start kit which contained tourniquet, gauze, small tegaderm, alcohol swab/chloraprep, microbore tubing), flush, angiocatheter, laboratory tubes and a blood gas syringe all within 1 min. For task 2, nurses were asked to ‘assemble and set up the necessary equipment to manually give a 300 mL normal saline via the push pull method using a 3-way-stopcock.’ The nurse had 3 min to complete this task. For the third task, the nurse needed to ‘assemble and set up the necessary equipment to start a dopamine infusion for this 15 kg child at 5 mcg/kg/min’. Successful completion of this task included finding the premixed dopamine infusion in the code cart, priming the infusion and setting the infusion pump appropriately to deliver dopamine at 5 mcg/kg/min. For the final task in this scenario, the nurse was asked to ‘prepare an epinephrine infusion for this 15 kg child at 0.1mcg/kg/min’. For this particular task, successful completion was determined if the PICU RN was correctly able to prepare an epinephrine infusion by placing 6 mg of 1:1000 epinephrine into 44 mL of D5W, label the medication and programme the pump successfully in less than 5 min. For any of these tasks, if the PICU RN was not able to find the equipment, or know how to perform a particular task in the time period allotted, then this was considered a first attempt failure and the APRN then instructed the PICU RN on the location of the equipment and correct method to prepare the equipment or medications and then the PICU RN was asked to repeat the task again within the time limits set.

This was done in an iterative fashion until all of the tasks for the shock scenario were completed successfully. Each task had required steps that needed to be done to ensure a first attempt success. The participant then moved onto the next scenario and the specific tasks within the next scenario. RNs received individualised education based on their baseline knowledge and experience. Following completion of training 1, nurses completed a survey assessing their experience with the code cart training (see online supplementary appendix 2).

Training 2

A second identical training session including an assessment and training to mastery for skills not initially performed successfully (training 2) was completed after a 6-month interval and was also done by one of the APRNs. The second training used the data collection tool that is shown in online supplementary appendix 1 and the pretraining and post-training questionnaires that are shown in online supplementary appendix 2. The goal of the second training was to refresh these infrequently used skills, assess retention of training effectiveness from training 1 and ensure continued competency. Given that we chose not to randomise training, we attempted to measure cofounders that may have influenced code cart training between training 1 and training 2 by administering a survey immediately prior to training 2 that included questions about interval PALS training, number of code cart uses in both simulated real patients environments and number of simulated codes and real codes. In addition, following completion of each training, participants were given a survey to complete. The survey examined the individual’s feelings of preparedness as well as provided the investigator with insight into future training needs.

Data collection and analysis

Study data were collected and managed using REDCap14 (Research Electronic Data Capture), an electronic data capture tool hosted at Dartmouth College. Data were analysed using Stata V.13.1 and SAS V.9.4. Two-sided significance level was set at 5%. Univariate logistic and multivariate linear regression analyses were performed to identify covariates associated with first attempt failures during training.

Results

Characteristics of participants are displayed in table 1.

Table 1

Characteristics of nursing participants (n=32)

Characteristic	Summary Value
Total PICU nursing experience years (median, IQR)	3 (1.4, 8)
DHMC PICU nursing experience years (median, IQR)	3 (1.4, 6)
Total nursing experience years (median, IQR)	8 (5, 20.3)
Number of paediatric codes attended last year (median, IQR)	1 (0, 2)
Number of paediatric mock codes attended in last year (median, IQR)	1 (1, 2)
Time since last paediatric code (%)
<1 month	10%
1–6 months	35%
7–12 months	16%
>12 months	39%
Time since last paediatric mock code (%)
<1 month	6%
1–6 months	47%
7–12 months	22%
>12 months	25%
Time since last Paediatric Advance Life Support training (%)
<1 month	13%
1–6 months	34%
7–12 months	19%
>12 months	34%
Number of code cart used last year (median, IQR)	2 (1,3.5)
Comfort in finding equipment on paediatric code cart
Scale 1–5, 1–uncomfortable, 3–neutral, 5–comfortable (median, IQR)	3 (2,3)
Preparedness in managing the code cart during codes.
Scale 1–5, 1–unprepared, 3–neutral, 5–prepared (median, IQR)	3 (2,4)
Preparedness in managing defibrillator during codes.
Scale 1–5, 1–unprepared, 3–neutral, 5–prepared (median, IQR)	3 (2,3)

PICU, Paediatric Intensive Care Unit.

DHMC, Dartmouth Hitchock Medical Center

Thirty-two RNs participated in the pilot quality improvement project with 32 completing the initial training and 16 completing both the initial (training 1) and the follow-up (training 2) sessions. All active PICU staff nurses completed the training. The reasons for not completing follow-up training were variable, including nurse turnover and leave of absence. Median PICU RN experience was 3 years (IQR 1.4, 8). RNs had attended a median of one paediatric code in the preceding year (IQR 0, 2) and the median number of paediatric mock codes was 1 (IQR 1, 2). The median number of code cart used in the prior year was 2 (IQR 1, 3.5). Sixty per cent of RNs had participated in a code in the year prior to training and 75% had participated in a mock code in the previous year. Sixty-six per cent had completed their PALS certification within a year prior to this training.

Table 2 displays the comparison of first attempt success rates for the scenario and skills for the RNs that completed both training 1 and training 2.

Table 2

Comparison of first attempt success rates between first and second training for each task (n=16)

Scenario and skills	Training 1 Frequency of first attempt success (%)	Training 2 Frequency of first attempt success (%)	p*
Shock: 1 year old (15 kg)
Obtain supplies to place intravenous and send laboratories	72%	88%	0.19
Assemble equipment to manually deliver a 300 mL bolus of normal saline	84%	100%	0.09
Assemble and prepare equipment to initiate a dopamine infusion at 5 mcg/kg/min	88%	88%	0.94
Assemble and prepare equipment to initiate an epinephrine infusion at 0.1 mcg/kg/min	28%	59%	0.04†
Hypoglycaemia: 1 month (5 kg) and 4 years old (20 kg)
Draw up 2 mL/kg of D10W	66%	88%	0.09
Draw up 1 mL/kg of D50W	100%	100%	–
Respiratory distress: 1 year old (10 kg)
Provide bag-mask ventilation with oxygen	72%	100%	0.02†
Find Yankauer and flexible suction catheters	66%	59%	0.64
Draw up 100 mcg atropine	84%	82%	0.86
Draw up 20 mg propofol	97%	100%	0.46
Draw up 10 mg rocuronium	100%	100%	–
Obtain supplies to place a nasogastric tube	94%	100%	0.29
Ventricular fibrillation: 4 years old (20 kg)
Place bed in CPR mode, place backboard	97%	100%	0.46
Obtain supplies for intraosseus catheter	97%	100%	0.46
Obtain paediatric defibrillator pads and prepare for delivery of 40J shock	97%	100%	0.46
Draw up 0.2 mg of epinephrine	78%	88%	0.38
Draw up 20 mg of lidocaine	100%	100%	–
Draw up 100 mg of amiodarone	94%	100%	0.29
Draw up 20 meq of sodium bicarbonate	91%	82%	0.4
Draw up 400 mg of calcium chloride	100%	100%	–
Supraventricular tachycardia: 5 years old (18 kg)
Draw up 1.8 mg of adenosine and describe administration	78%	82%	0.73
Charge defibrillator for 10J synchronised cardioversion	91%	82%	0.4

*p values calculated via χ2 test.

†Bold values are statistically significant. CPR, cardiopulmonary resuscitation; J, joules; W, water.

On first attempt, only 28% of RNs were able to assemble and prepare equipment to initiate an epinephrine infusion for a 1-year-old child in the allocated time frame. Furthermore, less than 80% RNs were able to perform the following skills on the first attempt in the allocated time: find supplies to place an intravenous, draw up D10 for a dextrose bolus, find the equipment to provide bag-mask ventilation, find suction catheters, draw up a code dose of epinephrine and draw up and appropriately deliver adenosine.

Median number of first attempt failures was 2 (IQR 1, 5). No participant completed all tasks successfully on their first attempt.

Table 3 displays the multiple logistical regression model comparing odds of first attempt failure during training 1 with various nursing characteristics.

Table 3

Multiple logistic regression model* comparing odds of first attempt failure during initial training 1 (n=32) with nursing characteristics

Characteristic	OR of first attempt failure (95% CI)
Total years of PICU nursing experience	0.98 (0.94 to 1.03)
Number of paediatric codes attended last year	0.91 (0.72 to 1.16)
Number of paediatric mock codes attended last year	1.18 (0.78 to 1.76)
Number of code cart used last year	0.91 (0.71 to 1.16)
Time since last paediatric code
≤6 months	ref
>6 months	0.72 (0.39 to 1.33)
Time since last paediatric mock code
≤6 months	ref
>6 months	0.74 (0.29 to 1.87)
Time since last code cart use
≤6 months	ref
>6 months	0.96 (0.45 to 2.03)
Time since last Paediatric Advanced Life Support training
≤6 months	ref
>6 months	3.35 (1.49, 7.53)†
Comfort in finding equipment on paediatric code cart (Self-reported scale 1–5, 1–uncomfortable, 3– neutral, 5–comfortable)
1 or 2	ref
3, 4, or 5	3.63 (1.14, 11.58)†
Preparedness in managing the code cart during codes (Self-reported scale 1–5, 1–unprepared, 3–neutral, 5–prepared)
2	0.29 (0.11, 0.76)†
3	0.11 (0.03, 0.47)†
4 or 5	0.09 (0.02, 0.45)†

*All covariates listed were included in the final logistic regression model.

†Bold values were statistically significant.

PICU, Paediatric Intensive Care Unit.

The feeling of preparedness was significantly associated with decreased first attempt failures: feeling of preparedness 4 or 5 (OR 0.12, p=0.02), 3 (OR 0.13, p=<0.01) and 2 (OR 0.32, p=0.02), versus 1 (1–unprepared, 3–neutral, 5–prepared). The factors significantly associated with increased first attempt failures included: most recent PALS course >6 months (OR 3.51, p<0.01) versus <6 months, and comfort managing the code cart 3–5 (OR 3.39, p=0.04) versus those less comfortable 1–2 (1–uncomfortable, 3–neutral, 5–comfortable). The following were not significantly associated with first attempt failure: years as a PICU RN, timing or number of previous code or mock code events, last code cart use and number of codes.

Sixteen nurses completed training 2. Overall, there was a 43% reduction (OR 0.57, p=0.01) in first attempt failure between the initial and follow-up training. Table 2 displays the comparison of first attempt success rates for the scenario and skills for the RNs that completed both training 1 and training 2. Univariate analysis revealed significant improvement in the following skills: assembling and preparing equipment to initiate an epinephrine infusion for a 1-year-old child and finding and demonstrating bag-mask ventilation for a 1-year-old child. No skill had a statistically significant decrease in first attempt success during training 2.

A multivariate linear regression model comparing the difference in first attempt failures between the initial and follow-up training for each participant is shown in table 4.

Table 4

Multiple linear regression model* comparing difference in first attempt failure between first and second trainings (n=16) with interval nursing characteristics

Characteristic	Regression coefficient (95% CI)†
Interval PALS training
No	ref
Yes	0.09 (−0.02, 0.2)
Interval code cart use (real or simulated)
No	ref
Yes	−0.02 (−0.2, 0.16)
Interval defibrillator use (real or simulated)
No	ref
Yes	−0.15 (−0.33, 0.02)
Months from prior training	0.02 (−0.01, 0.05)

*All covariates listed were included in the final linear regression model.

†No values were statistically significant.

PALS, Pedaitric Advanced Life Support.

In our model, there was no significant association between the reduction in first attempt failures with the following characteristics: interval PALS training, use of the code cart or defibrillator (either on real patients or in simulation) and the time between training sessions.

In addition to training, participating RNs were surveyed to gauge their response to the training programme. Post-training surveys were collected from participants after training 1. Eighty per cent of RNs found the programme to be ‘very helpful’ (5/5 on a 5-point Likert scale) and that it ‘greatly’ improved confidence in finding and using medical equipment on the paediatric code cart (5/5 on a 5-point Likert scale). See online supplementary appendix 2 for details of participant surveys.

Discussion

Many current resuscitation improvement projects focus on crisis resource management, which includes a focus on non-technical skills such as teamwork, leadership and communication.15–17 While our institution continues to work to improve teamwork and team training, we also seek to evaluate and improve individual roles during an emergency situation. In this programme, we focused on the nursing role of managing the code cart during emergencies. Given the low frequency of emergency events, achieving nursing excellence in these skills is a long-term goal. For this project, we demonstrated that many of the selected code cart skills were initially deficient in our nursing staff.  With our rolling refresher training programme, we were able to train all active nursing staff to skill mastery; we were able to demonstrate statistically significant improvement in the overall performance of our nurses as well as achieve retention of improved performance demonstrated during follow-up assessment and refresher training.

While the number of participants who completed both the initial and follow-up training was small (16), we were able to show a significant reduction in odds of first attempt failure between the two groups. Using multivariate linear regression to evaluate the association between the difference in first attempt failure between initial and follow-up training, we did not find a significant association with any of the covariates we measured for odds of first attempt failure (PALS training between sessions, code cart or defibrillator use in simulated or real environments or time between training sessions). While there may have been other factors we did not measure that contributed to improved performance, our conclusion is that the training itself led to significant improvement in these skills and retention of knowledge.

We acknowledge several limitations to our study. This was a single-centre programme with a limited number of participants. Our goal was to improve our team’s response to paediatric emergencies in our institution. While this is a daunting task, we chose to focus on nursing education and attempted to improve our PICU RNs training for managing the code cart during emergencies. With this stated goal, we intentionally choose a non-randomisation method as we felt it important to ensure that all of our PICU RNs were assessed and trained. We acknowledge all the limitations of a non-randomised study including other unmeasured cofounders leading to improvement in code cart management between trainings 1 and 2. We attempted to measure important cofounders between training sessions including interval PALS training, number of code cart uses in both simulated environments and on real patients, number of simulated codes and real codes, but there may have been other cofounders we did not measure that influenced our results. While we trained all active nurses during the study period, our unit did experience nursing turnover and illness/personal leaves resulting in only 50% of RNs able to complete training 2 by the end of the study period and, thus, only 16 nurses were included in the final comparative analysis. We provided this training while nurses were staffing and caring for patients, which may have led to interruptions and distractions and may have affected performance.

Another unavoidable limitation of our training programme was the time required to master 22 tasks. The training time is dramatically increased when training new graduate RNs, new PICU RNs or nurses who have not had recent exposure to the code cart or have not had to manage the code cart in other nursing roles at other facilities. Based on our post-training survey data and observations of the training process, our plan is to modify the original five scenarios and 22 tasks into six scenarios with 30 tasks with the goal of completing one scenario per month with all of our staff on a monthly basis. Having more frequent, shorter sessions will allow for RNs to complete these trainings during their regularly scheduled shifts. The shorter scenarios will also allow the study team members to complete more trainings for the nursing staff, possibly expanding into other paediatric care areas such as the Emergency Room and Inpatient Unit.

Conclusions

Our rolling refresher code cart training programme for PICU nurses resulted in identification of initially deficient life-saving PICU nursing skills, allowing for RN and skill specific training to skill mastery. Follow-up training revealed significant improvement in performance with a 43% reduction in odds of first attempt failure. Our rolling-refresher training programme can serve as a model for the training of low frequency but high risk nursing skills. Given the precedent set for timely delivery of life-saving medications during resuscitation, and marked improvement observed with our training exercises, there is promise these training efforts may aide in improving overall code performance. Further evaluation and refinement of our programme is needed to determine the most efficient and effective training methods.