Paediatric chest compression performance improves via novel augmented-reality cardiopulmonary resuscitation feedback system: A mixed-methods pilot study in a simulation-based setting.

Aim: More than 20,000 children experience a cardiac arrest event each year in the United States. Most children do not survive. High-quality cardiopulmonary resuscitation (CPR) has been associated with improved outcomes yet adherence to guidelines is poor. We developed and tested an augmented reality head mounted display chest compression (CC) feedback system (AR-CPR) designed to provide real-time CC feedback and guidance.
Methods: We conducted an unblinded randomized crossover simulation-based study to determine whether AR-CPR changes a user's CC performance. A convenience sample of healthcare providers who perform CC on children were included. Subjects performed three two-minute cycles of CC during a simulated 18-minute paediatric cardiac arrest. Subjects were randomized to utilize AR-CPR in the second or third CC cycle. After, subjects participated in a qualitative portion to inquire about their experience with AR-CPR and offer criticisms and suggestions for future development.
Results: There were 34 subjects recruited. Sixteen subjects were randomly assigned to have AR-CPR in cycle two (Group A) and 18 subjects were randomized to have AR-CPR in cycle three (Group B). There were no differences between groups CC performance in cycle one (baseline). In cycle two, subjects in Group A had 73% (SD 18%) perfect CC epochs compared to 17% (SD 26%) in Group B (p < 0.001). Overall, subjects enjoyed using AR-CPR and felt it improved their CC performance.
Conclusion: This novel AR-CPR feedback system showed significant CC performance change closer to CC guidelines. Numerous hardware, software, and user interface improvements were made during this pilot study.

Introduction

More than 20,000 children experience a cardiac arrest event per year in the United States.1., 2., 3. Although there has been an increase in survival in recent years, most children do not survive.4., 5. There is significant variability in survival across institutions, implying that cardiopulmonary resuscitation (CPR) performance largely impacts survival.6., 7. Additionally, the differences of patient size, chest wall compliance, and the emotional stress of a potentially dying child add to the difficulty of delivering high-quality CPR to children.8., 9.

High-quality CPR has been associated with improved outcomes. Yet, adherence to guidelines is poor with most reporting 20–40% adherence.11., 12., 13. The current gold standard for paediatric CPR quality is a quality CPR (qCPR) coach, which has been shown to improve guideline adherence up to 64% in a simulation-based environment. While effective, qCPR coach utilization can be limited by personnel and training deficiencies.15., 16.

Several CC feedback devices have been described in the literature with most showing improvement but with limitations.17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27. Some rely solely on visual feedback, which can be difficult to see in clinical events due to display limitations and room crowding.20., 22., 23. Most devices provide CC information with no direct guidance to improve CCs in real-time, necessitating the user to interpret the information and make self-directed changes.

Augmented reality (AR) is a burgeoning technology that has shown early promise in healthcare.28., 29., 30., 31. By utilizing a head mounted display (HMD) that layers relevant data directly in the user’s field of view, there is potential to overcome the constraints of current feedback devices. An AR HMD cannot be obstructed, is highly visible, and constantly tracks with head movement, allowing for use in crowded environments.

We have developed a novel AR CPR feedback HMD system (AR-CPR) with the primary mission of improving paediatric chest compression (CC) performance. A secondary mission was to overcome barriers associated with current CC feedback systems. The purpose of this mixed-methods pilot study was to 1. Determine the impact AR-CPR has on CC performance in real-time. 2. To identify and fix software and hardware issues. 3. To identify themes around user feedback on the AR-CPR system to inform future design.

Methods

We conducted a mixed-methods study to determine whether AR-CPR changes CC performance. The quantitative component was an unblinded, randomized crossover simulation-based (SBL) study. The qualitative component was a thematic analysis of users’ experience with AR-CPR. The study was conducted from September to October 2021 at the Johns Hopkins Children’s Center Paediatric Emergency Department (PED), an urban academic setting in Baltimore, MD. The study was approved by the Johns Hopkins Medicine Institutional Review Board (IRB00264313).

Study subjects, inclusion and exclusion criteria

A convenience sample of healthcare providers who perform CCs on children during their routine clinical care was included. This included nurses, clinical technicians, medical and physician assistant students and residents, physician assistants, paediatric emergency medicine fellows, and attending physicians. Subjects were recruited during their clinical shifts or agreed to participate prior or after their clinical shifts. Each subject had previously received standard basic life support training. There were no exclusion criteria. Subject remuneration via gift card was provided for completion of the study.

Sample size

A sample size of 28 subjects (14 per group) was needed to detect a 40% change in mean percentage of one-minute epochs with high quality CC between the two groups, assuming standard deviation of 35%, a significance level of 0.05 and a power of 0.8. Prior studies have demonstrated that with no feedback, AHA PALS algorithm compliance is around 20% with a standard deviation of 35%.14., 32., 33.

AR-CPR device development and specifications

The AR-CPR system was developed as a collaboration between the Johns Hopkins University (JHU) School of Medicine (SOM) and the JHU Applied Physics Laboratory (APL) Mixed Reality Collaboration Center. An Adafruit Feather STM32F405 Express and Adafruit ISM330DHCX – 6 DoF IMU (Adafruit Industries LLC, New York, NY) connected to a Dell Latitude 7420 (Dell Inc, Round Rock, Tx) running the AR-CPR Coach application developed by APL was wirelessly in communication via a TP-Link AC750 (TP-Link Technologies Co, Shenzhen, China) to a Vuzix M400 Headset (Vuzix Corporation, Rochester, NY) running the AR-CPR software to measure rate and depth and provide AR feedback in the users' field of view in real-time (Fig. 1, Fig. 2).

Fig. 1

Schematic diagram of AR-CPR. An Adafruit Inertial Measurement Unit (A) and Adafruit Microprocessor (B) connected to a Dell Latitude notebook (C) running the AR-CPR Coach Software developed by the Johns Hopkins University Applied Physics Laboratory Extended Reality Collaboration Center was wirelessly in communication via a TP-Link router (D) to a Vuzix Augmented Reality Headset (E) running the AR-CPR software.

Fig. 2

AR-CPR feedback user interface examples. If AR-CPR does not detect compressions for 3 seconds, an alert stating “”START COMPRESSIONS“” displays (A). In-range compressions are displayed with markers within the larger target box and display of the word “”GOOD“” (B). The colored circle integrates both rate and depth information. Depth is displayed with a vertical marker in centimeters (cm) with deeper depths displayed with a line lower to the bottom of the display. Rate is displayed with a horizontal marker in compressions per minute (CPM), with faster rates displayed with a line farther to the right of the display. Compressions that are detected outside of the goal rate or depth result in the appropriate marker changing from green to red, the 'marker's movement outside of the green target box, and a text box displaying to user’s deficit, as depicted by the “”TOO FAST“” prompt (C). If the user continues to compress sub optimally with any metric out of range for greater than 3 seconds, a larger visual prompt displays with guided feedback to correct the compressions, as depicted by the “”GO SLOWER“” prompt (D).

Study procedure

Prior to the SBL scenario beginning, informed consent was obtained, and demographics were recorded. Subjects were then individually randomized using an online random number generator to use AR-CPR in either cycle two (Group A) or AR-CPR in cycle three (Group B) (Fig. 3). Subjects were in teams of one to three compressors. There was no orientation to AR-CPR prior to use.

Fig. 3

Study protocol and randomization.

Each group was told the patient was a five-year-old child brought to the PED in asystole with a laryngeal mask airway in place. They were instructed to perform the best continuous CCs they could. Each subject performed three two-minute cycles of CCs with four minutes in-between each cycle to simulate best practice of three compressors.

AR-CPR continuously recorded rate and depth data for the entirety of the 18-minute event. Data collection via AR-CPR included three data points per second. A Laerdal SimJunior (Laerdal Medical, Stavanger, Norway) was connected to a Zoll R Series via Zoll OneStep Paediatric CPR Electrodes (Zoll Medical, Chelmsford, Ma) to simulate standard of care at JHCC. The defibrillator was specifically positioned out of subject sight.

The qualitative study entailed a semi-structured interview that explored users’ feedback and experience. Investigators asked closed-ended and open-ended questions about the overall experience, perception of AR-CPR’s effect on CCs, the visual display, and emotional or physical changes. Verbal responses were recorded and transcribed by Otter.ai (Otter.ai, Mountain View, Ca) and stored in a RedCap database.34., 35..

Outcomes

Data was collected via AR-CPR for all compressions. The primary outcome was measured by the percentage of time a subject was at the optimal rate of CCs (100–120 compressions per minute), CC depth of five cm (2.54–7.64 cm; representing upper and lower limits of the user interface), and a combination of rate and depth. Secondary outcomes included evaluating potential retention of CC performance after removal of AR-CPR, identifying hardware and software issues, and to qualitatively assess the AR-CPR experience. Qualitative outcomes were themes on user experience, identified from the semi-structured interview data.

Statistical and qualitative analysis

We summarized the demographic characteristics for participants in both Group A and B with descriptive statistics. We compared epochs of one second, three seconds, and 15 seconds, which showed similar patterns across cycles and groups. Because an optimal rate is two CCs per second, group consensus was that three seconds was the optimal epoch.

We first derived a mean rate and depth for each three second epoch. We assigned rate and depth at each epoch as a binary measurement: “at goal” or “not at goal”. “Goal CC” was defined as being “at goal” for both rate and depth during an epoch. We calculated the percentage of epochs “at goal” divided by all epochs a subject completed.

We performed unpaired t-tests to compare means between Groups A and B at each cycle. We used paired t-tests to calculate changes within each group between cycles two and three. Where appropriate, Cohen’s d is provided as a standardized measure of effect size. Analysis was performed using Stata Statistical Software: Release 13 (StataCorp LLC, College Station, TX).

Content analysis was applied to the transcript data. A coding guide was developed (TC) which was modified by team consensus. Each transcript was coded by at least two raters (TC, EB, BL). Coding proceeded iteratively. The coding team periodically debriefed and modified the coding guide with group consensus. The team coded all transcripts, observing for thematic saturation, or the point at which no new themes or codes were identified. Thematic analysis was applied to the coded text to determine themes related to subject AR-CPR feedback and experience.

Results

Quantitative data

There were 34 subjects enrolled. Sixteen subjects were assigned to have AR-CPR in cycle two (Group A) and 18 subjects in cycle three (Group B). Their characteristics are shown in (Table 1). Two cycles (16 and 20 epochs) had less than 35 epochs (105 s) recorded.

Table 1

Subject Demographic Information. Group A: AR-CPR in Cycle 2. Group B: AR-CPR in Cycle 3.

Subject Routine Clinical Role	Group A (%)	Group B (%)	Total
Technician	1 (6)	2 (11)	3 (9)
Nurse	8 (50)	10 (56)	18 (53)
Resident	2 (13)	2 (11)	4 (12)
Fellow	0 (0)	2 (11)	2 (6)
Clinical Associate	0 (0)	2 (11)	2 (6)
Physician's Assistant/Nurse Practitioner	2 (13)	0 (0)	2 (6)
Medical Student	3 (19)	0 (0)	3 (9)
Subject Age	Group A (%)	Group B (%)	Total
20–30	8 (50)	9 (50)	17 (50)
31–40	7 (44)	8(44)	15 (44)
>40	1 (6)	1 (6)	2 (6)
Were Nurse Subjects Trained as qCPR Coach	Group A (%)	Group B (%)	Total
Yes	3 (38)	7 (70)	10 (56)
No	5 (63)	3 (30)	8 (44)
Had the Subject Ever Done CPR on a Patient Before?	Group A (%)	Group B (%)	Total
Yes	12 (75)	16 (89)	28 (82)
No	4 (25)	2 (11)	6 (18)
Had the Subject Ever Used Any form of AR Before?	Group A (%)	Group B (%)	Total
Yes	0 (0)	1 (6)	1 (3)
No	16 (100)	17 (94)	33 (97)

Table 2 shows the percentage of time spent at goal CC for each cycle. No differences were seen between Group A and Group B at baseline (cycle one). In cycle two, Group A performed significantly better with CC depth but not rate compared to Group B. Subjects in Group A had 73% (SD 18%) perfect CC epochs compared to 17% (SD 26%) in Group B (p < 0.001). Comparing performance in cycle three, there was no differences in overall performance.

Table 2

Comparison of the percentage of time with perfect rate, depth, and chest compressions (CC) between Group A (AR-CPR in Cycle two) and Group B (AR-CPR in Cycle three).

		Group A	Group B	P Value	Cohen’s d
		Mean (SD)	Mean (SD)
		(n = 16)	(n = 18)
Cycle 1 (Baseline)	% Time with Perfect Rate	61 (36)	66 (41)	0.725	−0.12
	% Time Perfect Depth	12 (26)	13 (30)	0.894	−0.05
	% Time Perfect CC	4 (10)	6 (20)	0.642	−0.16
Cycle 2 (Group A AR, Group B no feedback)	% Time with Perfect Rate	90 (8)	76 (27)	0.056	0.68
	% Time Perfect Depth	79 (15)	21 (34)	<0.001	2.23
	% Time Perfect CC	73 (18)	17 (26)	<0.001	2.49
Cycle 3 (Group B AR, Group A no feedback)	% Time with Perfect Rate	86 (27)	87 (20)	0.896	−0.05
	% Time Perfect Depth	54 (42)	75 (36)	0.142	−0.52
	% Time Perfect CC	42 (38)	66 (36)	0.059	−0.67

Table 3 shows comparisons of differences between each cycle for individuals within each Group. When using AR-CPR both Group’s percentage of perfect rate, depth, and chest compressions (CC) statistically increased compared to no AR-CPR (p < 0.005). Group A had a statistically decreased percentage of perfect depth and CC during their cycle three, as compared to their cycle two (p < 0.05). No difference was detected between Group B cycle one and two.

Table 3

Comparison of the intra-group differences between cycles. Group A: AR-CPR in Cycle two. Group B: AR-CPR in Cycle three.

		Group A	Group B
		Mean (SD)	Mean (SD)
		(n = 16)	(n = 18)
Difference Between Cycle 2 and 1	% Time with Perfect Rate	29 (39)*	10 (34)
	% Time Perfect Depth	67 (28)*	7 (20)
	% Time Perfect CC	69 (22)*	10 (24)
Difference Between Cycle 3 and 2	% Time with Perfect Rate	−4 (24)	11 (33)
	% Time Perfect Depth	−25 (45)*	54 (38)*
	% Time Perfect CC	−31 (35)*	50 (33)*

p < 0.05.

Qualitative data

Thirty-four transcripts were analyzed. Saturation was reached after 17 transcripts. Themes focused on usability of the headset, anticipated barriers to AR-CPR use in clinical environments, and emotional response to CC performance. See Supplementary Materials for full thematic report.

Theme one pertained to usability of AR-CPR. The subthemes touched on cognitive offload and ease of use (subtheme 1.1), time to orient to AR-CPR (1.2), display visibility (1.3), and cognitive effort to focus on the display (1.4).

In subtheme 1.1, subjects described the lack of audio feedback and simple display enabled cognitive offloading. Subjects described the interface like playing a game. They foresaw the utility if no qCPR coach is available. The real-time feedback was an important feature. Subjects described a preference for the visual feedback with AR-CPR. They noted that it was easy to use. They appreciated the specific adjustments that needed to be made, beyond just an audible 'beep' from the Zoll.

“I actually found it really helpful because I'm more of a visual person so I kind of like it like in a critical care situation when everybody's kind of like talking over one another it's hard to like focus on, like the one voice, and the person that's talking to you, so it's kind of nice to have the visual feedback.“ – 33

Subtheme 1.2 described orienting to the system. Most subjects described a brief learning curve, taking several seconds to orient. Subjects noted that despite a brief learning curve they desire training on how to use the device.

“I kind of understood it… after doing it for, like, 10 seconds… I figured out… what I was supposed to do.” − 12

In subtheme 1.3, subjects noted challenges of display visibility. While most subjects thought AR-CPR was easy to use, there was criticism related to the small display and the monocular eye piece. Subjects described challenges in interpreting the display.

“Sometimes I felt like I had to shut one eye to see it really well and that I like it got a little blurry.” − 27

In subtheme 1.4, subjects explained the cognitive effort needed to interpret the visual display data. Some described that focusing on the display may distract from the act of performing CCs or distract from the other patient interventions.

“I guess it was a little difficult. Just getting it from, like, trying to focus in on that one spot in front of me that was giving the feedback…It took me some time to like focus it on the words that were there.” − 10

Theme two described anticipated barriers to clinical use. Subthemes noted comfort in traditional methods of CPR feedback (2.1), appreciation of the human touch of a qCPR coach (2.2), and trust in AR-CPR (2.3).

In subtheme 2.1, subjects mentioned a preference for the Zoll defibrillator with the CC feedback features, over an unfamiliar system.

“I just like looking at the [depth and rate] numbers [on the Zoll] like because you can tell … when you're doing compressions [and] once you see that the number on the Zoll is right, and you can, like, feel that, then you know … you stay where you are but …trying to keep that circle inside the rectangle is like a constant [task].” – 12

Subtheme 2.2 was that humans are needed to interpret CC performance data, and an alliance with a staff member can overcome technical failures of hardware.

“Just having a green or a red, or too fast or too slow, like, that's what it would do if we didn't go manual mode [on the Zoll]. And that's why we have our CPR coaches, because they're able to actually… in real time… evaluate the data.” − 15

In subtheme 2.3, subjects recognized that the depth gauge on the AR-CPR was inconsistent with their past experiences. They would desire more data to understand accuracy to trust the device.

“If I knew I was using it correctly, then I think it would be [a] good [experience].” − 7

Theme three described emotional responses to CC performance, such as that AR-CPR decreased anxiety (3.1), and that there is general stress around CC (3.2).

In subtheme 3.1, subjects mentioned that the instant, closed-loop feedback provided by AR-CPR reduced anxiety. Their ability to modify their technique without external stimulus brought relief, relative to listening to feedback from another person.

“I like that it provides … personal feedback, … in real time, instead of having someone having to … shout the things that you know you need to correct, while you're …providing CPR… It's a more closed loop system.” − 34

In subtheme 3.2, the stress around CC and the high-pressure situation of CPR was noted. They also described the pressure to deliver high accuracy of CCs was distressing, and some providers seek positive encouragement for good performance.

“[I liked] … the real time feedback and trying to keep it in that … good range and not just beeping at you if you're wrong.” − 31

Discussion

This pilot study showed significant performance changes relating to paediatric CCs with AR-CPR. At goal CCs improved from 17% to 73% with AR-CPR. This was largely due to improved depth adherence, which is the most challenging metric to do well.13., 36. This is on par with the current gold standard of a qCPR coach (64%). However, implementing qCPR coaches requires training and personnel that many environments cannot provide, such as some community hospitals and pre-hospital settings.15., 16. In these settings, AR-CPR could demonstrate even greater benefit. Additionally, AR-CPR overcomes many limitations.17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27. associated with feedback devices by providing real-time CC guidance in the user’s direct field of view that actively drives improvement in CC performance, is easy to see and use for most subjects, and provides multiple CC metrics.

Given the prototype nature of this project, we expected to uncover software and hardware issues. The two most noteworthy were: 1. CC data not given if full chest recoil wasn’t performed and 2. The potential inaccuracy and imprecision of the depth measurement. When the chest was not fully recoiled, the IMU would undercount rate and mismeasure depth. Once this error was identified, all subjects going forward were instructed to allow full recoil prior to starting their compressions, which resolved this issue. After the completion of this study, we corrected the software. The determination of accuracy and precision for rate and depth measurements by AR-CPR remains a key element to further development. Current work is under way to validate these metrics.

Thematic analysis revealed that subjects enjoyed using AR-CPR, felt it improved the quality of their CCs and was easy to use. The subjective cognitive offload of analyzing one’s own CC rate and depth in real-time was appreciated. This differs from published literature and should be explored further. Constructive comments mentioned time to orient to the device. While training may be necessary for future use of AR-CPR, this study demonstrated that with no orientation, users required merely seconds to comprehend the AR display.

Some subjects described an appreciation of the human touch of a qCPR coach. This speaks to the importance of the human–computer interaction and how computer systems are rarely sufficient to replace a human who must apply judgement, knowledge, and intuition. Rather, a symbiosis that combines a computer’s computation ability with a human user is the optimal outcome for high-quality CPR.

This pilot project has discerned several future directions for study. First, determining the accuracy and precision of the system. Second, focused user experience, workload assessment, and user interface (UI) testing with human factors engineers will be paramount to the adaptation of AR-CPR. Third, integrating more CC metrics such as recoil into the UI. Lastly, larger scale simulation-based testing is needed prior to moving towards clinical testing.

Beyond standard SBL research limitations, there are several limitations worth mentioning. First, the variation of team size from one to three subjects may have impacted performance due to an individual performance being improved or worsened by watching others. Although not directly analyzed, we believe due to the random nature of subject availability and recruitment, any performance change would be equivocal between groups.

Second, the Zoll defibrillator has an audio metronome feature that investigators could not turn off. Subjects were told prior to starting that this sound is due to “poor pad contact” related to the manikin, and to ignore it as best as possible. Given there was no difference in rate between Groups, there is a distinct possibility that this metronome feature impacted rate performance and potentially masked a benefit of AR-CPR.

Third, there were a variable number of datapoints per subjects, due to crashes related to software bugs in the AR-CPR prototype. These software crashes are believed to be related to momentary losses in communication between the notebook computer and the AR headset due to wireless transmission issues. While this does limit the utility of AR-CPR in the current prototype state, we believe that the randomness of these occurrences in both frequency and timing do not alter the statistical analysis. We have since solved the software issue.

Conclusions

We have developed a novel AR-CPR feedback system that significantly changed the CC performance of providers from 17% to 73% time at goal. This system was described as easy to use and overall beneficial to performance. Numerous areas for refinement and improvement were discovered.

Conflicts of interest

Authors have no conflicts of interest to declare.