Longitudinal effect of high frequency training on CPR performance during simulated and actual pediatric cardiac arrest.

STUDY AIM: To determine the impact of high-frequency CPR training on performance during simulated and real pediatric CPR events in a pediatric emergency department (ED).
METHODS: Prospective observational study. A high-frequency CPR training program (Resuscitation Quality Improvement (RQI)) was implemented among ED providers in a children's hospital. Data on CPR performance was collected longitundinally during quarterly retraining sessions; scores were analyzed between quarter 1 and quarter 4 by nonparametric methods. Data on CPR performance during actual patient events was collected by simultaneous combination of video review and compression monitor devices to allow measurement of CPR quality by individual providers; linear mixed effects models were used to analyze the association between RQI components and CPR quality.
RESULTS: 159 providers completed four consecutive RQI sessions. Scores for all CPR tasks during retraining sessions significantly improved during the study period. 28 actual CPR events were captured during the study period; 49 observations of RQI trained providers performing CPR on children were analyzed. A significant association was found between the number of prior RQI sessions and the percent of compressions meeting guidelines for rate (β coefficient -0.08; standard error 0.04; p = 0.03).
CONCLUSIONS: Over a 15 month period, RQI resulted in improved performance during training sessions for all skills. A significant association was found between number of sessions and adherence to compression rate guidelines during real patient events. Fewer than 30% of providers performed CPR on a patient during the study period. Multicenter studies over longer time periods should be undertaken to overcome the limitation of these rare events.

Introduction

Approximately 16,000 children suffer cardiac arrest in the United States per year. Cardiopulmonary resuscitation (CPR) remains the cornerstone of treatment for cardiac arrest, and guidelines for CPR performance have existed for decades. Despite this, clinical studies continue to show that CPR is frequently performed in a manner inconsistent with published guidelines.2, 3 It has been increasingly recognized that improving CPR education and skill training is an essential link to improving outcomes from cardiac arrest.

Evidence has shown that skills and knowledge in CPR begin to decay in months following life support courses such as Basic Life Support and Pediatric Advanced Life Support.5, 6, 7 The 2015 Consensus on Science from the International Liaison Committee on Resuscitation (ILCOR) Education Task Force states that there is “evidence that frequent training improves CPR skills [and] responder confidence” and “suggest[s] that individuals likely to encounter cardiac arrest consider more frequent retraining” in BLS. Based on these observations, in 2015 the American Heart Association launched the Resuscitation Quality Improvement (RQI) program, a novel educational approach to CPR skills where students undergo quarterly (every 3 month) self-directed refresher training in CPR skills.

Our group has published several studies examining the use of video review as a technique to assess resuscitative procedures, including CPR.2, 9, 10, 11 A recent report on infant CPR used a combination of chest compression monitor devices and video review to measure chest compression quality by single providers (rather than at the level of entire CPR events), allowing us to characterize individuals’ CPR performance as opposed to an aggregate description of CPR quality. For the current study, we report on actual CPR events using a similar method to measure the impact of the deployment of RQI training across healthcare providers in a tertiary pediatric ED. We hypothesize that, using a combination of video review and chest compression device data, we can demonstrate a direct link between high-frequency CPR training and improved chest compression quality at the level of individual providers during both simulated and real patient events.

Methods

This was a single center observational study conducted from August 2016 to December 2018. The study was approved by the Institutional Review Board of the Children’s Hospital of Philadelphia.

The study period consisted of a baseline period between August 2016 and July 2017 followed by an intervention period from October 2017 to December 2018.

RQI training

All healthcare providers in the ED who maintain certification in either BLS or PALS and who may be required to perform chest compressions during a patient event in the ED were enrolled in RQI in October 2017. Physicians, nurses, paramedics, and emergency medical technicians comprised the eligible pool of enrollees. A master list of providers, along with individual specific identification numbers, was maintained by the study team.

Following a baseline assessment of CPR skill during the month of October 2017, all providers were required to complete skill training in infant compressions, adult compressions, infant ventilations, and adult ventilations on a quarterly basis throughout the study period. Scores for each training session were assigned using a proprietary weighted algorithm based on AHA CPR recommendations and scaled from 0% to 100%. Subjects were required to achieve a passing score (75%) for each activity, per the standards set by the platform. If participants did not pass, skills stations are programmed to provide feedback on skills so that participants can remediate and complete the exercise; unlimited attempts are permitted until a passing score is achieved. Through this mechanism, all clinical staff in the ED were retrained in CPR each quarter.

CPR performance during training sessions over time

Performance data during training sessions was collected using a cloud-based learning and training management system (RQI, American Heart Association, Dallas, TX and Laerdal Medical, Stavanger, Norway). The compression score is calculated using the parameters of compression rate, compression depth, chest recoil, and correct hand position. The ventilation score is calculated using ventilation volume and rate. Median overall scores for compressions and ventilations were tabulated by quarter; the change in scores between the first quarter and the final quarter of the study period was analyzed.

Data were also collected for each individual compression completed by learners in all sessions without using the overall scoring algorithm. This allowed each learner’s training sessions to be assessed for the percent of training compressions that met guidelines for rate and for depth. This historical training performance data was then linked with the clinical database for providers participating in real patient events to determine their time since last training and performance measured in a similar way to performance measured during real patient events (described below).

CPR performance during real patient events

Enrollable events

All events where a patient received chest compressions (CCs) for > 1 min were eligible for inclusion. Events where a CC monitor device was not used, or where videorecording did not occur, were excluded. Data collected on patients included age, out-of-hospital versus in-hospital cardiac arrest, event duration, and outcomes according to Utstein definitions that could be determined from video, i.e., return of spontaneous circulation (ROSC) and survival to hospital admission.

During actual resuscitations, the ED response team in our center uniformly includes at least one respiratory therapist who has primary responsibility for assisted (bag-valve mask) ventilation in patients. Because this task is separately assigned to a specific provider, our outcomes of interest from actual patient events were limited to chest compression quality only.

Data collection: video review

Resuscitative care in the study site ED is videorecorded as part of an intradivisional continuous quality assurance program. Patient/parent consent is obtained at the time of consent for treatment. Videorecording is done using multiple synchronized camera views plus a view of the patient monitor. Videos are reviewed and de-identified data is collected on common resuscitative procedures, including CPR. Following data collection, videos are deleted within a mandatory 30 day period and are not part of the patient medical record.

Videorecorded events were reviewed by study team members. CC rate was expressed as compressions per minute (cpm). Start and stop times for each providers’ CC period(s) were measured to the nearest second. Compressors were individually identified and their ID numbers and provider category were recorded for each CC period performed; additionally, the date of that provider’s most recent CC training (either RQI or BLS/PALS) was entered in order to calculate the number of days elapsed since their last training session (referred to as ‘time since last training’, in days). The cumulative total number of RQI sessions completed by the provider up until that date was also recorded.

Data collection: compression monitor device

CC rate and depth were measured by a CC monitor device (R-series®, ZOLL Medical, Chelmsford, MA, USA) with dual sensor electrode pads in anterior-posterior position. Code Review™ software (ZOLL Medical, Chelmsford, MA, USA) was used to extract and analyze CC data; using start and stop time points for each compressor segment as determined by video review, the software allows for the selection of identical time periods and summarization of CC parameters for those time periods (i.e. CPR performance by individual providers). CC parameters were summarized by quarter; during the third quarter of data collection, there was a period of 11 weeks during which no CPR events occurred in the ED, and the third and fourth quarters of the study period were combined for data displaying purposes.

Chest compressions were classified as ‘guideline compliant’ (GL) when they met age-based recommendations for rate and depth as per American Heart Association Pediatric Basic Life Support 2020 guidelines (rate 100–120 cpm; depth 1.5–2.0 in for infants and 2.0–2.5 in for children). We performed a separate analysis with an outcome of interest of chest compressions within a rate of 90–130 cpm and a depth range of +0.5 in. of AHA recommendations (‘guideline plus’, GL-Plus).

Analysis

Quantitative data were analyzed using Stata statistical software (Version 13.1, StataCorp., College Station, Texas). RQI training data were analyzed for continuous learners for the four quarters of the intervention period. The overall scores for each quarter for continuous learners were analyzed for compressions and ventilations. The overall scores were then analyzed between Q1 and Q4 using a Wilcoxon signed-rank test.

Patient event data were analyzed with linear mixed effects models to determine the contribution of specific RQI training components on chest compression performance in pediatric EDs. The outcomes in these models were (1) the percentage of provider compressions in the ED that met guidelines for depth and rate for each event, and (2) the percentage of compressions either meeting guidelines or within the additional margin of upper or lower limits (GL-Plus). The models included time since RQI training, total number of previous RQI training sessions, and performance on previous RQI sessions. Cross-classified mixed effects models were used with random effects for event and provider as some providers participated in multiple patient events. Provider category, patient age category, total number of previous RQI sessions, number of days since most recent RQI training, and the percentage of compressions meeting guideline requirements (or GL Plus) on most recent RQI training were fixed effects.

Results

Baseline performance

During the baseline period from August 2016 to July 2017, 10 children received chest compressions under video recorded conditions with a compression monitor in place for at least part of the event; complete data was available on 86 compressor segments during this period (27 in infants, 59 in children). 2/10 patients achieved ROSC and survived to hospital admission. Average compression rate and depth per segment during this period are shown in Fig. 1, stratified by age category.

Fig. 1

Each dot on these scatterplots represents one segment of compressions by a single compressor. Shaded bars correspond to ranges of rate and depth compliant with AHA guidelines.

RQI training and simulated CPR performance

A total of 253 providers enrolled in RQI at the start of the study period. 159 providers completed all 4 quarterly RQI sessions over the study period. Table 1 shows the median scores across these 159 providers for all four tasks by quarterly session. A statistically significant improvement was seen for all four tasks between the first and last quarters of the study period.

Table 1

RQI scores by quarter.

	Median Overall Daily RQI Score by Quartera, b				Median Difference Score Q1 to Q5	Wilcoxon Signed-Rank Test P-value
	Q1	Q2	Q3−4	Q5
Infant Compressions	91.5 (76.8−97.9)	97.9 (90.1−99.7)	95.5 (82.7−99.3)	95.0 (84.0−99.4)	1.5 (-7.4–13.7)	0.03
Infant Ventilations	82.9 (65.1−92.3)	94.9 (84.5−99.1)	93.5 (85.1−96.4)	95.5 (87.5−98.1)	10.2 (2.3–25.7)	<0.001
Adult/Child Compressions	84.3 (71.6−96.1)	90.2 (80.3−96.3)	92.9 (84.8−98.1)	96.2 (88.7−98.8)	7.3 (0.6–21.3)	<0.001
Adult/Child Ventilations	79.8 (55.2−93.4)	84.6 (73.9−95.5)	85.5 (77.9−95.4)	95.3 (84.4−98.8)	13.35 (0.8–33.5)	<0.001

Values are score or difference score (IQR).

If learners completed more than one day of training in any quarter, the overall daily RQI scores have been averaged for that quarter.

Overall daily score is the mean of all RQI attempts for a user on a given day.

CPR performance in real patient events

During the study period from October 2017 to December 2018, 28 children received chest compressions under video recorded conditions. 11/28 patients achieved ROSC and survived to hospital admission. Data was collected from a chest compression monitor device during at least one compression segment for 16 CPR events (7 infants, 9 children). A total of 49 observations of chest compressions by individual providers had complete data for analysis.

Based on identifying providers from video review, there were 49 observations of chest compressions performed by a total of 32 RQI trained providers nested within individual cardiac arrest events in the ED. 30/49 (61%) of segments were performed by paramedics or EMTs; 14/49 (29%) by nurses, and 5/49 (10%) by physicians. The number of events each provider participated in ranged from one to five; the range of total number of compressions performed per event for each provider ranges from 40 to 1313 chest compressions. Descriptive statistics for adherence to AHA guidelines for rate and depth are shown in Table 2. Fig. 2 shows rate and depth per compressor segment stratified by patient age and by quarter of the study period.

Table 2

Chest compression performance in patient events.

Outcome Variables	All CC segments (n = 49)	Peds (n = 39)	Infant (n = 10)
	Median (IQR)	Median (IQR)	Median (IQR)
Percent of ED Compressions meeting GL for Ratea	57.7 % (22.8 %–83.4 %)	69.7 % (50.6 %–88.2 %)	16.0 % (8.4 %–51.4 %)
Percent of ED Compressions meeting GL Plus for Rateb	91.1 % (71.2 %–97.5 %)	94.1 % (82.4 %–98.0 %)	64.2 % (32.4 %–86.5 %)
Percent of ED Compressions meeting GL for Deptha	12.0 % (0−46.4%)	22.6 % (0 %–60.2 %)	0.7 % (0 %–7.2 %)
Percent of ED Compressions meeting GL Plus for Depthb	77.4 % (31.7 %–97.8 %)	84.2 % (27.5 %–98.3 %)	54.9 % (37.1 %–94.5 %)
% of Compressions meeting GL Plus for Depth at last RQI Trainingb	100 % (96.7 %–100 %)	98.8 % (95.8 %–100 %)	100 % (100 %-100 %)

GL for rate is 100–120 compressions per minute; GL for depth is 1.5–2.0 inches for infants and 2.0–2.5 inches for children.

GL plus for rate is 90–130 compressions per minutes; GL for depth is 1.0–2.5 inches for infants and 1.5–3.0 inches for children.

Fig. 2

Across all observations, the median time between the most recent RQI session and the clinical event in which a given provider participated was 72 days (IQR 28–95 days). The median number of cumulative quarterly RQI sessions a given provider had completed at the time of the relevant event was 2 (IQR 1−4 sessions). Descriptive statistics for elapsed time between training and clinical events are shown in Table 3, along with the percent of CCs compliant with guidelines during the most recent RQI manikin sessions for those providers. Fig. 3 shows scatterplots displaying points corresponding to individual providers and the relationship between CC guideline adherence and time since last training and cumulative number of training sessions.

Table 3

Elapsed time and most recent score from RQI session at time of clinical event.

Predictor Variables	All CC segments (n = 49)	Peds (n = 39)	Infant (n = 10)
	Median (IQR)	Median (IQR)	Median (IQR)
Number of Days since last RQI Training	72 days (28 days-95 days)	61 days (28 days-95 days)	92 days (28 days-103 days)
Number of Prior Quarterly RQI Trainings	2 (1–4)	2 (1–4)	2 (1–3)
% of Compressions meeting GL for Rate at last RQI Traininga	85.0 % (71.2 %–94.3 %)	83.3 % (66.2 %–93.2 %)	94.7 % (82.3 %–96.6 %)
% of Compressions meeting GL Plus for Rate at last RQI Trainingb	100 % (98.5 %–100 %)	100 % (98.3 %–100 %)	100 % (100 %-100 %)
% of Compressions meeting GL for Depth at last RQI Traininga	79.7 % (62.9 %–98.3 %)	77.9 % (58.3 %–91.7 %)	98.7 % (97.1 %–99.2 %)
% of Compressions meeting GL Plus for Depth at last RQI Trainingb	100 % (96.7 %–100 %)	98.8 % (95.8 %–100 %)	100 % (100 %-100 %)

GL for rate is 100–120 compressions per minute; GL for depth is 1.5–2.0 inches for infants and 2.0–2.5 inches for children.

GL plus for rate is 90–130 compressions per minutes; GL for depth is 1.0–2.5 inches for infants and 1.5–3.0 inches for children.

Fig. 3

On multivariate analysis, a significant association was found between the number of prior RQI sessions and the percent of CCs meeting guideline-plus for rate (β coefficient −0.08; standard error 0.04; p = 0.03); no other independent associations were found (Table 4).

Table 4

Multivariate analysis.

Variables Predicting Meeting GL or GL Plus for particular skill (depth or rate) during ED events	β Coefficient (Standard Error)	P-Value
% of compressions meeting GL for Rate
Number of Days since last RQI Training	.0007 (.0009)	0.45
Number of Prior Quarterly RQI Trainings	.0067 (.04)	0.87
% of Compressions meeting GL at last RQI Training	−.11 (.19)	0.57
% of compressions meeting GL for Depth
Number of Days since last RQI Training	.001 (.0007)	0.09
Number of Prior Quarterly RQI Trainings	−.06 (.03)	0.06
% of Compressions meeting GL at last RQI Training	.078 (.11)	0.49
% of compressions meeting GL Plus for Rate
Number of Days since last RQI Training	.0006 (.0005)	0.19
Number of Prior Quarterly RQI Trainings	−.009 (.03)	0.76
% of Compressions meeting GL at last RQI Training	−.053 (.41)	0.90
% of compressions meeting GL Plus for Depth
Number of Days since last RQI Training	.0003 (.0008)	0.72
Number of Prior Quarterly RQI Trainings	−.08 (.04)	0.03
% of Compressions meeting GL at last RQI Training	−.45 (1.04)	0.67

Discussion

In our study, we showed that high-frequency CPR skill training results in improved performance during subsequent retraining sessions among healthcare providers in a pediatric ED. We found an independent association between a greater number of RQI sessions and GL-Plus adherence for compression rate, but were unable to demonstrate an improvement in chest compression depth during actual CPR events throughout a one year period. We also found that compression depth adherence was especially poor in infants, a finding in keeping with prior studies by members of our group.2, 3

Importantly, our additional findings include the fact that exposure to performing CPR on children, even at a high-volume tertiary center, is very rare. Only a small proportion of providers performed any chest compressions on a child throughout the study period, and the majority of the observations in our analysis represented the only time throughout the study period that a given provider performed any chest compressions. These epidemiologic constraints resulted in a very limited sample size for our study.

Multiple studies have demonstrated that CPR for children is frequently performed in a manner inconsistent with published guidelines.2, 3, 13 Optimizing CPR training for healthcare providers is an essential component in ILCOR’s formula for survival, bridging the gap between medical science and implementation. AHA life support courses have traditionally been readministered to learners at intervals ranging from every 1–2 years; this schedule typically yields effective short term knowledge and skill acquisition. However, it has been shown in published studies that CPR skill begin to deteriorate in a matter of months after the completion of a typical course.5, 6, 7 At the level of individual provider exposure, performing chest compressions on a child is a rare occurrence. Our group previously reported a descriptive summary of time performing chest compressions among individual providers on the ED, and found that the median amount of time in one year that a single ED provider performs chest compressions on a child is approximately 3 min. Pediatric cardiac arrest represents a combination of a high-stakes clinical event with infrequent occurrence, making skill maintenance a particular challenge.

The term ‘spaced practice’ has been used to refer to training delivered in multiple separate sessions over a period of time, where the content is either distributed across multiple sessions or is practiced repetitively. Pediatric CPR education may be a particularly good fit with spaced practice, given that events are infrequent and that success depends on psychomotor skill as well as knowledge. Members of our group have previously reported that bedside providers in a pediatric hospital demonstrate improved CPR performance on manikins when they are provided repetitive short-duration practice sessions for chest compressions. Based on these and other studies, the AHA RQI program was designed to integrate spaced practice into routine BLS training, in addition to other advanced educational techniques such as real-time feedback and deliberate practice. Published studies on the effectiveness of RQI have demonstrated improved simulated CPR performance during training sessions as well as improved chest compression fraction during actual CPR events at an adult centers.17, 18 To our knowledge, this is the first study to measure CPR performance by individual learners during real patient events and to attempt to examine the direct relationship with their CPR training.

Studies that allow the analysis of individual providers’ CPR performance during cardiac arrest management are lacking in published literature, with the vast majority of studies reporting on CPR quality in aggregate across entire events.3, 13 Video recording and review has been reported as a reliable method of assessing CPR performance in adults and children.2, 19, 20, 21 Our group has previously reported on pediatric CPR performance in the ED quantified by video review; among our significant findings from those studies was that compression depth is not accurately assessed from video review alone.2, 22 The combination of video review and chest compression monitor devices allows quantifying chest compression depth and rate in time segments that correspond to segments of CPR provided by individuals.10, 19 The present study used the same methodology, and we believe this method has the potential to directly link training and performance.

Several limitations to the current study should be mentioned. Our data was collected at a single tertiary pediatric hospital. The majority of pediatric CPR events in hospital EDs occur at non-tertiary centers. While the epidemiology of pediatric cardiac arrest is likely to be very similar between this site and other hospitals in the US, it remains possible that the generalizability of our findings to other centers is limited.

Where possible, we attempted to accurately quantify the timing of providers’ training outside of their enrollment in RQI (e.g. if newly hired personnel had recently completed BLS or PALS training). This data was not complete across the entire pool of providers. Additionally, the use of the Zoll R device is meant to allow a compressor to receive real-time visual feedback on compression depth and rate. The configuration of our resuscitation room does not allow a line of sight to the device when CPR is in progress; however, in some cases, a supervising clinician may give prompts to a provider based on the feedback. We did not account for the influence of device feedback or verbal prompts in our analysis.

The overall survival of the cohort was poor, in keeping with most published studies of pediatric CPR in the ED. Most of our patients were OHCA patients, representing a group for whom outcomes have remained poor and improved negligibly over several decades. There are multiple examples of surrogate outcomes in literature on CPR quality such as ROSC and first defibrillation success23, 24; these studies provide evidence that optimized CPR performance has the potential to influence clinical outcomes, and that measuring CPR performance objectively is desirable. We believe video review is a feasible methodology for quantifying CPR performance and may yield a measurable process that can be studied prospectively as a metric of care delivery independent of clinical outcomes, with the eventual goal of improving pediatric CPR training on a large scale to improve survival from pediatric cardiac arrest.

Conclusions

In this single center study, we demonstrated that high-frequency CPR training among pediatric ED healthcare providers led to consistently increased performance of high quality CPR in ongoing training sessions. Simultaneous data collection in real events showed an association between number of sessions completed and guideline adherence for compression rate. Future studies should use similar methodology to examine individual providers CPR quality during real patient events in a multicenter and longitudinal fashion to determine the impact of this novel training strategy on CPR delivery and patient outcomes.

CRediT authorship contribution statement

All listed authors have made substantial contributions to the conception and design of the study, acquisition of data, and/or analysis and interpretation of data. The drafted manuscript was reviewed and commented on in its entirety by all coauthors and the presented final version has been approved by all coauthors.

Funding

This study was supported by an unrestricted research grant to the Children’s Hospital of Philadelphia from RQI Partners, LLC and funding from the Pediatric Critical Care Medicine Endowed Chair Funds.

Declaration of interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests:

Russell E. Griffin is an employee of RQI Partners, LLC.

Debra G. Heard is a paid consultant for the American Heart Association.