Using eye-tracking augmented cognitive task analysis to explore healthcare professionals' cognition during neonatal resuscitation.

AIM: We aimed to describe the cognitive processes of healthcare providers participating as airway leads in delivery room neonatal resuscitations using eye-tracking assisted debriefing to facilitate recall and provide situational context.
METHODS: Delivery room neonatal resuscitations were recorded using eye-tracking glasses worn by participants who acted as airway leads. These glasses analyze eye-movements to produce an audio-visual recording approximating what was "seen" by the participant and marking their visual attention. Participants then reviewed and debriefed their recordings. Debriefing involved a retrospective think-aloud prompted by eye-tracked recordings and an integrated semi-structured interview. Debriefing sessions were transcribed and subjected to thematic analysis.
RESULTS: Eight healthcare providers participated in 10 interviews; two providers participated twice in two separate resuscitations. Most visual attention was directed at the infant (62%), with 16% directed to monitors/gauges, 3% to team members. Five major themes emerged including situation awareness, performance, working in teams, addressing threats to performance, and perception of eye-tracking. Information processing was complex and involved top-down and bottom-up processing of environmental stimuli, integration of knowledge/experience, and anticipation of patient response. Despite the focus on individual cognition, interpersonal interactions and teamwork emerged as key aspects of resuscitation performance. Potential threats to performance include equipment issues, mental stress, distractions, and parental presence. Eye-tracking recordings were well-received by the participants.
CONCLUSION: Retrospective think-aloud prompted by point-of-view eye-tracked recordings is a useful means of examining cognition of healthcare providers during neonatal resuscitation. Themes identified in this project aligned with existing models of clinical reasoning.

Introduction

Effectiveness of neonatal resuscitation depends partly on individual healthcare provider (HCP) and team performance. However, human error occurs in 16–55% of neonatal resuscitations1, 2, 3 and is estimated to contribute to over two-thirds of perinatal mortality and morbidity. Optimizing human factors such as physical resuscitation spaces and training may decrease errors via supporting HCP cognition. A better understanding of human cognition during neonatal resuscitation could therefore inform changes in processes, training, and environments to better support HCP performance and improve neonatal outcomes.

Aspects of HCP cognition in neonatal resuscitation have been examined, including information gathering, situation awareness, stress, and decision-making.5, 6, 7 These studies examined different aspects independently and take place in simulated settings. The investigation of interactions between these elements in a naturalistic setting is limited. As HCPs' cognition and performance is affected by the situational context of each resuscitation, influences of the resuscitation environment should be considered when examining clinical performance.

Cognitive task analysis (CTA) is a group of methods used to identify demands and cognitive skills required to complete a task. CTA often aims to break down complex cognitive processes that drive behaviours and can be applied to clinical and simulated settings. The study of HCP cognition in real-life neonatal resuscitation is limited by researchers’ ability to stay an observer without intervening; therefore, simulations have been used. However, behaviours exhibited in simulations may differ from those that occur in real life.7, 10, 11 Audio-visual recordings is an alternative way to examine real-life HCPs’ performance.12, 13 Compared with environmental recordings, point-of-view recordings with eye-tracking layover may add specificity to recall by indicating where the participant was looking at any given time during, particularly if different objects of interest are within the same field of vision.14, 15

Eye-tracking glasses use near-infrared light to estimate the point of visual gaze. This technology is integrated with point-of-view recordings taken by a camera positioned on the front of the glasses to create an audiovisual recording showing what was “seen” by the wearer along with a gaze marker demonstrating the precise visual attention (VA). Eye-tracking has been used in neonatal resuscitation, including examining HCPs’ VA during simulated and real-life cases and assessing the effect of environmental modifications on VA.6, 16, 17, 18 In this study, we aim to use eye-tracked, point-of-view video augmented CTA to describe HCP cognition in real-life neonatal resuscitations.

Methods

This study took place between July 2019 and January 2020 at the neonatal intensive care unit (NICU) at the Royal Alexandra Hospital, Edmonton, Canada. Health Ethics Research Board, University of Alberta (Pro00077581) approved the study. Approximately 350 infants with a birth weight of <1500 g are admitted to this NICU annually. Here, deliveries are attended by a multi-disciplinary team involving a nurse and transport nurse for infants >32 weeks’ gestation, or a larger 3–6 member team for infants <32 weeks’ gestation. High-risk resuscitations occur within dedicated resuscitation rooms.

A convenience sample of neonatal resuscitations was included. HCPs who acted as airway manager were eligible to participate. Informed consent was first obtained from participants. Resuscitations were then recorded using head-mounted eye-tracking glasses (Tobii Glasses 2, Tobii Technology, Inc., Falls Church, VA), which use reflected near-infrared light to track pupillary movement. Image processing algorithms then superimpose estimated gaze patterns onto an point-of-view recording taken by a camera on the front of the glasses, resulting in an audio-visual recording approximating what was “seen” by the participant and their visual attention (VA) (Fig. 1). After calibration, HCPs perform the resuscitation as usual. Recording ended when the infant was stabilized.

Fig. 1

Example still image from eye-tracked, point-of-view video. (Circle indicating visual attention).

Participants were debriefed as soon as possible following the resuscitation. In think-aloud CTA, participants speak freely about their thought processes during a task. Here, participants performed think-alouds retrospectively. After an orientation to the recording, participants were asked to think back to the resuscitation and verbalize what was “running through their heads” while watching their actions and gaze patterns on the recording. Recordings were paused at set points and semi-structured interview questions asked about aspects of the resuscitation that the participant had not discussed spontaneously (Appendix A, Supplementary data).

Debriefings were audio-recorded and transcribed verbatim, then subjected to thematic analysis. Authors first familiarized themselves with the data through repeated transcript re-readings and review of corresponding eye-tracked recordings for context. Transcripts were then analysed using Nvivo software (Version 12, QSR International). Common themes were identified. While several themes were taken from the literature on non-technical performance in neonatal resuscitation, most emerged during analysis. Transcripts were then re-examined to determine if themes fit across cases. Participants were recruited until a repetition of concepts emerged, suggesting saturation. Eventually, final themes were defined. Square brackets represent sections of codes that have been edited for clarity and ellipses represent omission of words within a quote.

To complement CTA, eye-tracking recordings of adequate quality (gaze capture >60%) were analysed using manual analysis and Tobii Lab analyser software (Tobii Technology, Inc., Falls Church, Virginia, USA) to quantify visual attention (VA). In our experience, when more than 40% of gaze samples are missing, VA quantification cannot be reliably made. VA measures included: i) % cumulative time spent on each area of visual interest (AOIs, including infant, monitors, other HCPs, equipment, etc.) and ii) frequency of gaze-shifts between AOIs. Recordings were analysed until the end of active resuscitation, or upto 10 min.

Descriptive statistical analysis was performed for patient and VA data. Data are presented as mean (standard deviation (SD)) for normally distributed continuous variables and median (interquartile range (IQR)) when the distribution was skewed. Statistical analyses were performed with SPSS 26 (IBM, Armonk, NY).

Results

Eight HCPs participated in 10 interviews; two HCPs participated in two separate resuscitations each. Descriptions of HCPs, patients, and resuscitations are shown in Table 1. Median gestational age was 30.6 weeks (28.2–35.2) and median birth weight was 1428 g (1158−2723). Nine videos could be analysed for VA, with median duration of 526 s (310–601). HCPs primarily focused on the infants (62% of total time, IQR 47–68), with 16% (6–21) VA directed to monitors and gauges, 3% (7–29) on other HCPs (2–6), and 5% (4–7) on equipment and other AOIs, and 11% (8–14) in transition between AOIs. Median gaze-shift frequency was 0.39/second (IQR 0.26−0.53), or the equivalent of 1 gaze-shift every 2.5 s.

Table 1

Characteristics of recorded neonatal resuscitations and participant’s demographics.

Case ID	Participant ID	Professional Designation	Role in Resuscitation	Experience with Neonatal Resuscitation (years)	Infant’s Gestational age (Weeks + days)	Infant’s Weight (g)	Interventions Performed	Infants Condition
V1	1	RN/ TN	Airway Lead	7	30+0	1230	CPAP, PPV	Twin growth discrepancy
V2	2	RN/ TN	Airway Lead	10	33+6	2430	CPAP, PPV	Vasa previa- anterior
V3	3	RN/ TN	Airway Lead	10	31+2	1450	CPAP, PPV	Premature
V4	4	Neonatal Fellow	Team Lead/ Airway Lead	3	26+5	940	CPAP, PPV	Premature, monochorionic, diamniotic twins
V5	2	RN/TN	Airway Lead	10	35+0	3000	CPAP, PPV, Needle Thoracentesis	Hydrops fetalis
V6	5	RT	Airway Lead	2.5	35+6	2630	CPAP, PPV	Not specified respiratory distress
V7	6	RN/ TN	Team Lead/ Airway Lead	15	28+2	1310	CPAP, PPV	Premature
V8	3	RN/ TN	Airway Lead	10	29+0	1405	CPAP	Premature
V9	7	Neonatal Fellow	Team Lead/ Airway Lead	10	28+	860	CPAP, PPV	Premature, Maternal HELLP Syndrome
V10	8	RN	Team Lead/ Airway Lead	12	39+0	3610	No respiratory support	Coarctation of the Aorta

V, video; RN, registered nurse; TN, transport nurse; RT, respiratory therapist; CPAP, continuous positive airway pressure; PPV, positive pressure ventilation.

From the CTA, five themes emerged including situation awareness, performance, working in teams, addressing threats to performance, and perception of eye-tracking (Fig. 2, Appendix B, Supplementary data).

Fig. 2

Summary of Themes and Subthemes.

Theme 1: situation awareness

Participants described a conscious awareness of their patient, team, and resuscitation environment, in keeping with the concept of situation awareness. Subthemes included perception of stimuli, situation monitoring, comprehension (sense-making), and anticipation.

Perception of stimuli

HCP participants described perceiving auditory, visual, and tactile stimuli from patients, team members, and monitoring equipment. The high density of stimuli was apparent from the participant’s recollection and frequenty gaze-shifts in the videos. Participants described stimuli that they were reflectively aware of (HCPs remembered processing the stimuli during the resuscitation) and unreflectively aware of: “I am looking very quickly at this monitor. I’m not sure that I can even see the numbers (V2)).”

Situation monitoring

HCPs reported using “top-down” and “bottom-up” monitoring. In top-down monitoring, HCPs purposefully “scanned” the environment: “I will…follow the same sort of circuit in my vision and be like monitor, monitor, monitor time, check baby, and try to go through all of the little pieces to make sure that things are OK… (V4).” Participants also described bottom-up processing, when cues drew them to a specific stimulus (e.g., audio alarm caused one HCP to look at a monitor).

Comprehension/sense-making

HCPs described making sense of the resuscitation by referencing knowledge, experience, and cognitive aids, such as goal values for infant’s vital signs: “She is [at] 40% oxygen [saturation], but within the 10 min, so I was just checking that with my chart (V2).” Additionally, HCPs noted changes over time: “We are hovering in the low 90’s–100’s for heart rate, there was a brief improvement, and then here we started slowing (V5).”

Anticipation

HCPs described how their anticipations of the infant’s progress guided decision-making. Anticipations resulted from integrations of perceptions, comprehension, knowledge, and experience. One HCP described anticipating resuscitation before the delivery: “If the baby’s head has been stuck… then the baby can be a little bit flatter at birth and take more time to resuscitate(V2).”

Theme 2: performance

HCPs described evaluating their own actions in keeping with the concept of performance. Subthemes included compliance with guidelines, accord with clinical experiences, and comparing to team members.

Compliance with guidelines

HCPs described how guidelines affected their preparations, decision-making, and task performance. HCPs described mentally reviewing resuscitation steps before the infant's arrival, using guidelines to direct their actions. However, intentional protocol deviations can be seen as a sign of experience rather than failure: “…we increase the [fraction of inspired oxygen] before [the arrival of the infant] for a baby that is little and a novice would not do that as it is not part of NRP(V7).”

Accord with clinical experience

HCPs evaluated their actions based on clinical experience. They discussed i) prioritizing tasks(e.g., “Because I've been to so many deliveries, I know that it has not started to pick up [heart rate and saturation readings](V3)” ii) problem-solving (e.g., “We have the probe on but sometimes we need to hold it a little bit [to improve our reading] … (V10)”) and iii) optimizing the use of resources (e.g., “From past experience, the babies typically do better once they are on the [CPAP machine] … if we get her on the [CPAP machine], she may actually improve… (V3).”

Comparing to team members

HCPs described comparing themselves to team members as a valuation of their performance and to learn (e.g. “A lot of the stuff that I do I've modelled off of people who have more experience than me (V6).”) Another HCP described actively learning from their colleague: “… it has been a while since I have done this, so I want to watch how she is doing it(V5).”

Theme 3: working in teams

HCPs described resuscitations as a team practice characterized by trust and familiarity, role-taking, information sharing, and mutual support.

Trust and familiarity

HCPs described trusting team members because of their familiarity: “…we have practiced a lot and have been to many, many deliveries together, so everybody knows what to anticipate(V3)”. When familiarity was lacking, two HCPs compensated by looking around prior to the resuscitation to learn the names, professional designations, and roles of others.

Role taking

HCPs frequently referred to implicit roles that team “fall into.” This was described as both potentially beneficial and detrimental. A newly hired HCP stated: “… in a centre like this you benefit a lot in that people are very flexible in their roles and you do not have to describe to them exactly what it is that they have to do(V4).” However, this might present some challenges: “…when you are coming into a new team that can be very disorienting…I can't get my footing and establish myself as head of the bed if everyone is kind of doing their own thing and it happens without me(V4).”

Navigating leadership

HCPs discussed how team composition affected their function as team leader: “[There is a] balance of being a leader in a scenario and accepting that you have exceptionally talented colleagues that know what they are doing and have an excellent grasp of their roles and your roles(V4).” Another HCP discussed her role as leader of a sub-team: “the team lead…makes sure that everyone is on the same page and coordinates the process, but … I was coordinating with my respiratory therapist with regards to the respiratory part of the baby asking about air entry and pressures (V9).”

Using guidelines to support a shared understanding

HCPs referenced using guidelines to develop a shared team understanding. Specifically, guidelines oriented the team to a step-through process. Guidelines also helped teams develop congruent anticipations of how interventions would progress: “We always assess, intervene, assess and when it wasn't working, we moved on as far as MR.SOPA [acronym for corrective steps](V6).”

Information sharing

Information sharing was discussed as important to teamwork, and occurred both when the information was available to all team members (e.g., verbalizing numbers displayed on the monitor) and when the information was accessible to one individual (e.g., verbalizing auscultated breath sounds). HCPs also described sharing their thoughts to keep everyone “in the loop” (e.g. “I just have a running dialogue so that everyone can hear what I'm thinking(V10).”)

Mutual support

HCPs described providing feedback and managing team workload as additional supportive behaviours. HCPs described noticing errors or opportunities for improvement in their colleagues. Some HCPs provided feedback, for example: “… her shaking the baby's leg like that really irritated …so I politely ask her to not do that (V2).” One HCP stated: “the way that people provide feedback-it's an important indication of how the team is working together(V1).” Other times, HCPs described not vocalizing concerns due to i) being focused on their own performance ii) not wanting to overburden their colleague and iii) being cognizant of their colleague’s expertise (“[Assistant] has been doing this for a very long time…if it had become an issue, then I would have said something(V1)).”

HCPs also discussed monitoring workload, perceiving delays in task completion or decreased task quality as indications of excess workload. HCPs described ways they mitigated excess workload, such as calling for help: “I wanted to make sure that she was not feeling stressed about managing too many things at once so I got [Assistant] to go and see if we had any extra staff…(V1)” Alternatively, they prioritized tasks and provided encouragement: “If you do not have extra help, trying to prioritize what needs to be done at this moment versus what can wait or trying to either relieve the load or give some positive feedback so that [he or she] does not feel as stressed(V7).”

Theme 4: addressing threats to performance

HCPs discussed possible barriers to performance including equipment issues, mental stress, distractions, and parental presence.

Equipment issues

HCPs frequently described equipment issues. Often, delays in obtaining accurate readings from monitoring equipment slowed resuscitations: “She looks pinker than what the number [oxygen saturation] is saying so you know sometimes it's just due to equipment failure versus the actual baby…(V2).” HCPs also discussed challenges with mask ventilation due to the addition of a bulky sensor for the respiratory function monitor: “I may have even done some vagal … by compressing the mask onto the baby's face because I was also trying to balance and keep it upright (V9).” Limited resuscitation space was also discussed as a barrier.

Mental stress

Some HCPs reported mental stress during resuscitations, including i) feeling scrutinized (e.g., “I did feel super self-conscious because of the glasses and there was a lot of people watching(V6)”), ii) unfamiliarity with the setting (e.g., “[there is a] added cognitive load of a being in a spot that is new… (V4)”), iii) lack of experience (e.g., “I have only intubated…five or six times… it was an extra thing that I had to think about in the back of my mind”(V6)) and iv) personal experiences (e.g., “I was very hungry, I'm super busy, and this is going to sound selfish when you have not had a break all day and you are starving…(V8).” One HCP described mitigating tensions using “black humour”: “not making light of the situation by any means, but we are taking some of the stress out of a potentially stressful situation…it is also like team camaraderie… (V3).”

Distraction

Potential distractions was seen in videos and included irrelevant conversations, hospital phone calls, other concurrent resuscitations, and nuisance alarms. HCPs mostly did not discuss these distractions until prompted. When asked, most HCPs were aware of the distractions but dismissed distractions as affecting their performance. For example, one HCP discussed regaining focus: “I do not even remember what was asked in the background… but you can see me taking my attention away, and then coming back. (V3).”

Parental presence

HCPs discussed their perspectives on parental presence during resuscitations. Concerns included i) worrying how parents will cope (e.g.,“I just think that it would be so scary [for parents] so I would prefer if their child is a little bit more stable before they come in(V7).”), ii) extra demands on HCPs (e.g., “… we need everyone to concentrate and if a parent is walking in…and we are [intubating] we need to focus on getting that work done (V10).”) and iii) compromising sterility. In contrast, some HCPs felt that parental presence had little effect on their performance.

Theme 5: perception of eye-tracking

All participants felt that eye-tracking recordings improved their recall. Some noticed that they were behaving in a manner they were not conscious of. For example: “to soothe my own nerves, constant re-checks (of equipment) are what I will often use to remind myself that things are OK. This is something that I did more than I thought that I did(V4).” Some HCPs were surprised by the speed with which their visual focus changed: “I thought the video was going in fast forward, but … I just don't realize how much I'm really looking around(V10).” Several HCPs asked if they could participate again, citing reviews as a learning opportunity.

Discussion

We describe a method of examining HCP cognition during neonatal resuscitation, using retrospective think-aloud CTA prompted by eye-tracked videos. HCPs’ retrospective verbal reports highlighted important resuscitation factors beyond the patient, physical environment, and individual task performance. These themes align with systems-based analysis, conceptualizing complex systems, such as a resuscitation team, as a series of hierarchically layered and interconnected subsystems (Fig. 3). An individual’s interactions with their physical environment lies at the system’s centre, with likely the most immediate implications. This is exemplified by perception of stimuli in the resuscitation environment and physical task performance. Next, individual performance is exemplified by comprehension, anticipation, evaluation of performance, and threats to performance. Finally, interpersonal interactions and organization factors further influences resuscitation performance.

Fig. 3

Conceptualizing neonatal resuscitation cognitive analysis themes within the systems-based analysis.

HCPs described how they collected, interpreted, and utilized information from the resuscitation environment, aligning with Endsley's concept of situation awareness (SA), defined as “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future”. The accuracy of HCPs’ SA was not examined in this study; however, studies suggest that SA is often inaccurate during neonatal resuscitation.6, 24, 25 Failures in SA can result in errors. In our study, HCPs described potential barriers to SA including i) challenges with equipment, ii) personal factors (e.g. fatigue or stress), iii) interpersonal factors (e.g. hesitancy to speak up), and iv) organizational factors (e.g., working without a break due to staffing limitations).

While we aimed to examine individual cognition, and most VA was directed at the infants, “Working in Teams” emerged as an important theme. Subthemes align with existing models of effective teamwork, including Crew Resource Management and TeamSTEPPS.21, 27 First, HCPs highlighted the importance of effective information sharing. A study of simulated neonatal resuscitations demonstrated that each additional information sharing behaviour per minute was associated with a 0.8% increase in a performance score and a 14-s decrease in resuscitation duration. Similarly, a study of NICU resuscitations described “thinking out loud” as vital to effective resuscitation. Second, HCP discussed their willingness to provide feedback during resuscitations. Assertive communication, or “speaking up”, has been shown to improve communication and reduce errors in various healthcare settings.5, 28, 29 Finally, familiarity and trust was discussed frequently as contributing to team performance. This reflects findings by Salih et al., which suggest that working synergistically is a facilitator to optimal team performance during NICU resuscitation.

HCPs all agreed that the eye-tracked recording improved their recall. Participants paused recordings frequently to expand on their thoughts, allowing for a depth of information that might not otherwise have surfaced. Eye-tracking also provided insight by revealing i) the density of environmental stimuli, ii) contrasts between top-down/bottom-up processing, and iii) contrast between reflexively aware and reflexively unaware stimuli. Previous eye-tracking studies in neonatal resuscitation focused on quantifying visual attention16, 17, 18; in contrast, our use of think-aloud CTA provided an in-depth exploration of HCPs thought processes while providing VA context. With eye-tracking, HCPs could easily discern where they were visually focused in the resuscitation environment, even when multiple objects of interest are within their view. Some HCPs explained that eye-tracking review prompted them to reflect on their performance in a novel way. Eye-tracking debrief provides a different experience to learners than traditional video debrief,30, 31 and has been perceived positively by those learning to perform resuscitations. In an emergency room simulation study, debriefs augmented with eye-tracked recordings resulted in significant improvements in students' patient safety practices compared to those observed before the debriefings.

Eye-tracking have been a useful tool in the study of ergonomics in neonatal resuscitation, particularly in examining how HCPs divide their VA during simulated and real-life neonatal resuscitations.6, 16, 17, 18 They have helped to investigate the effect of environmental factors like the presence of respiratory function monitors and monitor positioning on HCPs visual attention. Pairing eye-tracking glasses with CTA further explores HCPs’ cognition to better understand clinical decision-making in neonatal resuscitation. Going forward, these strategies may help to determine how HCPs process and integrate environmental inputs during neonatal resuscitations, and how that may be affected by different physical environments, individual training, and team configurations.

Our study had several limitations. First, participants were experienced—we could not compare novices and experts. Second, we did not capture endotracheal intubations or chest compressions, so cognitive processes during these procedures were not examined. Third, recordings took place in a single institution; themes may be influenced by HCPs' shared experiences, education, and institutional culture. Also, participants and team members were aware that they were being recorded and may have modified their behaviour(Hawthorne effect). Finally, participants' recall may be flawed, despite viewing eye-tracking videos.35, 36

Conclusion

Retrospective think-aloud prompted by eye-tracking can be used to examine the individual HCP cognition and team performance during neonatal resuscitation. Key themes identified include situation awareness, performance, working in teams, addressing threats to performance, and perception of eye-tracking.

CRediT authorship contribution statement

Emily C. Zehnder: Conception and design, Collection and assembly of data, Analysis and interpretation of the data, Drafting of the article, Critical revision of the article for important intellectual content, Final approval of the article. Georg M. Schmölzer: Conception and design, Collection and assembly of data, Analysis and interpretation of the data, Drafting of the article, Critical revision of the article for important intellectual content, Final approval of the article. Michael van Manen: Conception and design, Collection and assembly of data, Analysis and interpretation of the data, Drafting of the article, Critical revision of the article for important intellectual content, Final approval of the article. Brenda H.Y. Law: Conception and design, Collection and assembly of data, Analysis and interpretation of the data, Drafting of the article, Critical revision of the article for important intellectual content, Final approval of the article.

Ethical approval

Health Ethics Research Board, University of Alberta approved the study (Pro00077581).

Declaration of interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.