Improving newborn heart rate assessment using a simple visual timer.

OBJECTIVE: Newborn resuscitation relies on accurate heart rate (HR) assessment, which, during auscultation, is prone to error. We investigated if a 6 s visual timer (VT) could improve HR assessment accuracy during newborn simulation.
DESIGN: Prospective observational study of newborn healthcare professionals.
SETTING: Three-phase developmental approach: phase I: HR auscultation during newborn simulation using a standard clock timer (CT) or the VT; phase II: repeat phase I after using a bespoke training app (NeoRate); phase III: following the Newborn Life Support course, participants assessed random HRs using the CT or VT. MAIN OUTCOME MEASURES: HR accuracy (within ±10 beats/min, correct HR category, i.e. <60, 60-100 and >100 beats/min), assessment time and error-free rates were compared.
RESULTS: Overall, 1974 HR assessments were performed with participants more accurate using the VT for ±10 beats/min (70% CT vs 86% VT, p<0.001) and correct HR category (78% CT vs 84% VT, p<0.01). The VT improved accuracy across all three phases. Additionally, following app training in phase II, the HR accuracy of both the CT and VT improved. The VT resulted in faster HR assessment times of 11 s (IQR 9-13) compared with the CT at 15 s (IQR 9-23, p<0.001). Error-free scenarios increased from 24% using the CT to 57% using the VT (p<0.001), with a shorter assessment time (CT 116 s (IQR 65-156) vs VT 53 s (IQR 50-64), p<0.001).
CONCLUSION: Using a VT to assess simulated newborn HR combined with a training app significantly improves accuracy and reduces assessment time compared with standard methods. Evaluation in the clinical setting is required to determine potential benefits. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Auscultating newborn heart rates (HRs) with a stethoscope has a high rate of error.

Performing parallel tasks increases response time and reduces accuracy.

Healthcare professionals are unable to consistently estimate HR accurately during prolonged resuscitations.

A simple six-second visual timer (VT) improves accuracy and reduces the time taken to make HR assessments during simulations.

Training using the NeoRate App improves HR estimation.

Using the VT during a prolonged resuscitation could double the accuracy and more than halve the assessment time.

Introduction

Heart rate (HR) is an important indicator of a newborn’s condition at birth, determining if resuscitation is required and the effectiveness of any intervention.1 Auscultation using a stethoscope is the quickest and most common method of HR assessment.2 However, incorrect HR estimates occur in 28%–46% using this method; these could result in the incorrect management and delay of stabilisation potentially affecting outcome.3 4

Technological approaches for HR measurement may not be available or initially inaccurate,5–7 so healthcare professionals (HCPs) need to be able to accurately estimate HR using a stethoscope especially in the resource-poor setting or with the lone resuscitator.7

The Neonatal Resuscitation Program suggests using the 6 s rule to estimate HR, but this is not universal; for example, it is not a taught component of the UK Newborn Life Support (NLS).1 Using this rule, the assessor counts the number of heart beats heard in 6 s and multiplies this by 10.8 This introduces two parallel tasks, counting the heart beats while counting 6 s on a clock. Performing parallel tasks of this nature in a stressful environment can increase response time and decrease accuracy.9 10 We aimed to reduce this cognitive burden by using a 6 s visual timer (VT) (see online supplementary media file or https://youtu.be/gB9H1F8qWD4). We hypothesised that HCPs would improve their HR assessment accuracy. The secondary objective was to quantify HR assessment time.

Methods

This was a three-phase developmental study using three different participant groups.

Phase I: VT testing

A simple electronic VT device was designed and built by the team engineer (DEM), incorporating a LED into a 3-D printed plastic mould that flashed every 6 s. The user estimated the HR by counting the number of heart beats heard between flashes and multiplying by 10 to obtain the HR in beats/min.

Participants undertook HR assessments using the Laerdal SimBaby manikin (Laerdal, Stavanger, Norway). This involved placing an electronic stethoscope (ThinkLabs ANR2; ThinkLabs LLC, Colorado, USA) on the chest of the manikin. Heart beat recordings were made from the Laerdal SimBaby manikin using an electronic stethoscope. Audio files were played to each participant when assessing the HR through the electronic stethoscope using an iPod Nano (Apple, California, USA) at a standardised volume. During each simulation, the time taken to assess the HR was from stethoscope placement on the chest of the manikin to when the participant verbalised the result. Participants performed five HR assessments for each technique using a digital clock displaying seconds and minutes from time zero (clock timer (CT)), or the 6 s VT. They were given a brief explanation of how to use the VT. For each technique, participants were randomised to one of four audio lists providing HRs in a different order. The HRs assessed were 50, 70, 90 and 110 beats/min, to represent the range of categorical boundaries defined by NLS1 (very slow, <60 beats/min; slow, 60–100 beats/min; and normal, >100 beats/min); participants were unaware of the HRs chosen.

Phase II: training app testing

This phase studied an in-house-designed newborn resuscitation training app, NeoRate. The app incorporates the same 6 s VT (see online supplementary media file) and includes training sections. These allow the user to listen to a range of HRs (50–145 beats/min in 5 beats/min stages) and to test themselves using both the CT and VT methods. The assessment time is measured by the app, from when the HR audio file starts to when the participant enters their estimation into the app, and results are logged for download. Each participant was provided with a Samsung Galaxy Tab3 Lite (Samsung, Seoul, South Korea) with the NeoRate app for a period of at least 2 weeks prior to study testing to allow familiarisation with both assessment methods. Participants then conducted the identical simulation protocol as described in phase I using the app CT and 6 s VT.

Phase III: NLS course app testing

On the day of successful completion of their NLS course, participants were provided with the NeoRate app as described in phase II and were given a 5 minute introduction to the VT method. The app has a testing section in which HRs (between 50 and 145 in 5 beats/min steps) and testing methods (CT and VT) are randomly selected and split equally. This series was chosen as it covers the range of HRs (<60–>100 beats/min). All participants undertook 16 HR assessments, 8 for each method. The sequence of method choice and HR was randomly generated by the app so that the participant switched between methods multiple times. The HRs used did not differ significantly according to the methods used (data not shown). To reflect use in actual resuscitations, the CT commenced at a random time (between 0 and 60 s). Similarly for the VT, the first flash appeared randomly between 0 and 6 s. Each participant used headphones (Genius GHP-400A, KYE Systems Corp, Taiwan) at a standardised volume.

Outcome measures

The estimated HR accuracy was determined in three ways: (1) falling into recommended HR categories of <60, 60–100 and >100 beats/min,1 11 (2) within ±10 beats/min and (3) difference between estimated and actual HR. Errors were also classified as overestimates or underestimates.

For all phases, participants completing at least five valid assessments for each method were analysed as if performing a single, prolonged resuscitation with multiple assessments, allowing a cumulative error rate to be calculated. The ±10 beats/min measure was used as this would better reflect the tracking of HR through a resuscitation, for example, a baby improving during resuscitation and their HR increasing from 60 to 90 beats/min.

Patient and public involvement

It was not appropriate to involve patients or the public in the design, conduct, reporting or dissemination plans for the study.

Data analysis

Unpaired categorical data were analysed using χ2 and Fisher’s exact test; data were tested for normality, and non-parametric continuous data were analysed using the Mann-Whitney test. Logistic regression was used to calculate ORs of the percentage error for (1) HR category and (2) within 10 beats/min comparing the two methods. Multiple group analysis used Kruskal-Wallis with Dunn’s multiple comparisons test. GraphPad Software V.8.02 (La Jolla, California, USA) was used to analyse the data. For phase III, assuming a power of 80%, significance of 5% and based on previous studies,3 4 the sample size required showing an improvement of 50% with the VT, compared with the CT, in error reduction within 10 beats/min was 76; to account for any data loss, a convenience sample size of 80 participants was used. Statistical significance was set at p<0.05.

Results

Phases I and II

Participants in both phase I (n=44) and phase II (n=28) had similar clinical roles (online supplementary 1). In phase II, one participant was excluded as they did not complete the study to protocol. Each participant assessed five simulated HRs using both methods; in total, there were 440 and 270 assessments each for phases I and II, respectively (online supplementary 2).

Accuracy of HR assessments

Percentage errors for HR category and those >10 beats/min are shown in figure 1A, B. Overall, there was a significant improvement with the VT compared with the CT, which was most marked for estimating the HR within 10 beats/min (table 1). Compared with phase I, using the app in phase II mostly improved accuracy for both the CT and VT: CT HR category, OR 1.7 (95% CI 1.0 to 2.8, p=0.04); CT±10 beats/min, OR 2.0 (95% CI 1.3 to 3.4, p<0.001); and VT ±10 beats/min, OR 2.7 (95% CI 1.2 to 6.1, p=0.01). However, there was no significant difference between phases I and II for VT category, OR 1.4 (95% CI 0.7 to 2.5, p=0.36). Of note, categorical assessment of an HR of <60 beats/min using the VT was poor but much more accurate assessing within 10 beats/min (figure 1).

Figure 1

Percentage of errors made when assessing the heart rate (beats/min) in the correct category (<60, 60–100 and >100 beats/min) (A) and >10 beats/min (B) using both CT and VT. *P<0.05, **P<0.01. CT, clock timer; VT, visual timer.

Table 1

ORs with 95% CIs for the percentage error for the heart rate category and within 10 beats/min for each method, CT and VT, during phases I and II

Phase	Accuracy criteria	CT (% errors)	VT (% errors)	OR (95% CI)
I	Category	65/220 (29.5)	36/220 (16.3)	2.1 (1.4 to 3.4)
	±10 beats/min	75/220 (34.1)	32/220 (14.5)	3.0 (1.9 to 4.8)
II	Category	27/135 (20.0)	17/135 (12.6)	1.7 (0.9 to 3.4)
	±10 beats/min	27/135 (20.0)	8/135 (5.9)	4.0 (1.7 to 8.5)
Total category		92/355 (26%)	53/355 (15%)	2.0 (1.4 to 2.9)
Total ±10 beats/min		102/355 (29%)	40/355 (11%)	3.2 (2.1 to 4.8)

CT, clock timer; VT, visual timer.

HR assessment time

In phase I, median assessment times were significantly faster using the VT (12 s, IQR 10–16 s) compared with the CT (18 s, IQR 10–32 s). In phase II, participants showed significant improvement in assessment time after using the training app (CT=10 s, IQR 7–16 s; VT=10 s, IQR 9–11 s; figure 2).

Figure 2

Phase I and phase II violin plots (median and IQR) showing assessment times for the CT and VT methods. **P<0.001. CT, clock timer; VT, visual timer.

Assessments during a prolonged resuscitation

If all five assessments were made during the same newborn resuscitation, the number of participants making no errors: phase I CT, n=14 (32%) and VT, n=28 (63%) (p=0.005); phase II CT, n=12 (46%) and VT n=22 (85%) (p=0.01). Combining assessment times for phases I and II for participants making no errors, there was no significant difference between the VT (n=50) median HR assessment time of 56 s (IQR 49–66 s) and 58 s (IQR 39–104 s) with the CT (n=26) (p=0.8).

Phase III

Eighty participants (online supplementary 1) recruited from five different NLS courses in the UK made 1264 valid assessments (CT, n=632; VT, n=632; online supplementary 2).

Accuracy of HR assessment

Participants were more accurate using the VT for simulated HR assessment compared with the CT method for within 10 beats/min (85.0% vs 69.4%, p<0.001), but there was no difference when identifying the correct HR category (83.0% vs 81.0%, p=0.27). Errors were evenly spread between overestimates and underestimates within 10 beats/min for the VT (6.8% overestimates, 85.0% correct, 8.2% underestimates), whereas when using the CT, participants tended to overestimate HR (21.4% overestimates, 69.4% correct, 9.2% underestimates). Median absolute HR differences significantly reduced variability using the VT method (±5 beats/min, IQR 0–10) compared with the CT method (±5 beats/min, IQR 2–15; p<0.001).

When using the VT, participants were significantly quicker with a median assessment time of 10.9 s (IQR 8.9–12.6 s) compared with the CT method of 15.6 s (IQR 10.1–22.4 s, p<0.001) (figure 3A).

Figure 3

Phase III violin plots (median and IQR) showing (A) heart rate assessment times for each individual assessment using the CT (n=632) and VT (n=632) methods and (B) total time taken to make a series of 5 error-free heart rate assessments (CT, n=19; VT, n=45). **P<0.001. CT, clock timer; VT, visual timer.

A total of 79 participants made at least five valid HR estimations using both methods. If these HR assessments were part of a continuous prolonged resuscitation, the percentage of participants making no errors (defined as within 10 beats/min) for all five assessments in a row significantly improved from 24% (n=19) using the CT to 57% (n=45) using the VT (p<0.001). For participants making no errors, the total time taken using the VT, 53.1 s (IQR 49.8–63.6 s), was significantly faster than that using the CT, 115.9 s (IQR 64.9–156.0 s, p<0.001) (figure 3B). For those with errors, the total time taken using the VT was 55.8 s (IQR 46.2–72.0 s) and 82.5 s (IQR 55.05–114.9 s, p<0.001) using the CT.

Combined results over the three phases

In total, 151 participants performed 1974 assessments. Participants were significantly more accurate using the VT for ±10 beats/min (86% VT vs 70% CT, p<0.001) and the HR category (84% VT vs 78% CT, p<0.01). Median assessment time was also significantly faster using the VT at 11.0 s (IQR 9.0–13.0) compared with the CT at 15.2 s (IQR 9.3–23.0, p<0.001).

Discussion

HR is an important determinant of a newborn’s need for resuscitation at birth, but HCPs estimate it incorrectly in approximately one-third of cases.3 4 Our results demonstrate a similar HR estimation error of 30%–34% when using the standard CT, but using a simple 6 s VT significantly improves both accuracy and reduces assessment time across multiple simulation settings and could potentially improve prolonged resuscitations. Furthermore, training using the NeoRate App improves HR estimation.

To test whether the 6 s VT method significantly improved performance, we designed a three-phase study providing proof of concept (phase I), development of the training app (phase II) and subsequent evaluation by HCPs completing an NLS course (phase III). During phase I, the concept of a 6 s VT was tested and demonstrated an approximate 50% reduction in error rate. Phase II took the concept further with the development of a bespoke HR training app, allowing participants to have HR-focussed training with the standard CT and the VT, resulting in improved performances for both, although using the VT further enhanced assessment accuracy and time. Training apps improve performance in both knowledge-based and scenario-based simulations,12 13 and our findings support this. Furthermore, our study had a comparable categorical error rate as the NeoTapAvancedSupport,14 an HR estimation app.

Phase III explored if the VT within the NeoRate app could be quickly adopted by NLS-trained HCPs. Again, fewer errors were made using the VT compared with the CT (±10 beats/min, 85% vs 69.4%; within correct category 83% vs 81%). Additionally, errors using the VT were equally split between overestimates and underestimates within ±10 beats/min. However, using the CT method, most errors overestimated the actual HR, similar to Voogdt et al3 who found more errors were overestimations,6 potentially failing to recognise the need for resuscitation.

During neonatal resuscitation time, perception has been shown to be inaccurate15 and could affect HR assessment if using a time-based counting method. The VT not only standardises the HR assessment window, thereby reducing variability, but also removes one of the parallel tasks required when using the standard CT method, therefore avoiding dual task interference.9 10 This simplification reduces the cognitive burden and therefore increases accuracy and reduces assessment time. For example, in phase III, this equated to almost a 5 s reduction in assessment time.

The real benefits of the VT could be realised during prolonged resuscitations when multiple HR assessments are made as HCPs are unable to consistently estimate HR accurately.4 Following training with the app in phase II, users learnt to better assess HR using both the CT (46% error-free) and VT (85% error-free). In the final phase, where participants had just completed their NLS certification, their accuracy doubled over a prolonged resuscitation using the VT. Moreover, the HR assessment time more than halved, which could have important implications when following the recommended 30 s reassessment guidance.11 For example, the lone resuscitator could potentially save 50 s if making 10 assessments in the first 5 min of life. Any reduction in the time a baby remains hypoxic is beneficial and good early care reduces neonatal complications.16

The VT could be easily incorporated into resuscitation equipment or stethoscopes, allowing it to be hands-free but would need maintenance such as battery changes. It is available on smartphones, negating this need. However, the parental acceptability of HCPs using smartphones in resuscitation scenarios is not known. The VT compliments other technological approaches for assessing HR (eg, electrocardiographs or pulse oximeters) and would be particularly useful during delayed cord clamping where these other technologies are disadvantaged due to the equipment application time, delay in obtaining signals, the size of the monitors and the need for trailing wires.

The main limitation of our study is that simulation studies lack realism and additional inputs to guide resuscitation, for example, tone, breathing and colour of the baby. The HRs heard were fixed, did not cover the full range of possible HRs and were recorded from a manikin at the same volume, whereas a newborn baby’s HR shows variability in speed and volume at different rates and potentially has added noises from breathing and the environment. However, we believe standardisation of what each participant heard was important. In phases I and II, there was a limited range of HRs used for assessment, particularly low HRs and in multiples of 10 beats/min, although participants were unaware of this. This was addressed in phase III with a wider range of HRs not only in multiples of 10 beats/min and the same improvements in accuracy and timing were observed. Although participants in phase II had access to the app for 2 weeks, it was not possible to standardise the time each participant spent on the app. As with all simulation studies, these results require validation in a clinical study.

Conclusion

With minimal training, using a VT can improve the accuracy and speed while reducing the variability of HR assessments made by HCPs. In all three phases of this study, the VT improved error-free performance in prolonged resuscitations. The VT is a simple, low-cost tool that could easily be incorporated into resuscitation training courses and equipment. To support worldwide access and to allow further evaluation and study, the NeoRate training app used in this study is freely available to download.