Can eye-tracking technology improve situational awareness in paramedic clinical education?

Human factors play a significant part in clinical error. Situational awareness (SA) means being aware of one's surroundings, comprehending the present situation, and being able to predict outcomes. It is a key human skill that, when properly applied, is associated with reducing medical error: eye-tracking technology can be used to provide an objective and qualitative measure of the initial perception component of SA. Feedback from eye-tracking technology can be used to improve the understanding and teaching of SA in clinical contexts, and consequently, has potential for reducing clinician error and the concomitant adverse events.

Introduction

Human factors are a major contributor to the outcome of any health care process:1 it has been estimated that as many as 80% of medical errors are the result of human error in one form or another.2 In uncontrolled settings, the risks are likely to be heightened. Paramedics and other health care professionals who practice in uncontrolled and unfamiliar settings have to manage interruptions and distractions while attempting to diagnose, treat, and stabilize patients. They are required to perform these duties with limited diagnostic tools and support, and often have to work when fatigued, uncomfortable, or uncertain of their surroundings. These combined stressors can lead to an increased risk of medical error.3,4

Studies of human factors that can contribute to medical error have led to many theories of cognition and process models to explain clinical behavior; Situational Awareness (SA) is one such model. Although the theory originated in aviation safety, SA has been widely applied to the health care setting.2,5,6 As it encapsulates awareness of one’s immediate surroundings, their context, meaning, and the possible progression of events,5 SA seems well-suited to the study of paramedic practice. SA links cognition to environment, and it has been suggested that improved SA can lead to a reduction in diagnostic error and an improvement in patient care.6–9 However, because SA is a construct of cognitive processing, it is notoriously difficult to measure.10

Eye-tracking is a technique that provides an objective, quantifiable, and measurable link between an individual and the immediate environment;11,12 it uses technology to determine what a subject is looking at. Because links between the focus of vision and the focus of cognition have been established,13–15 eye-tracking provides useful information for studying SA.16

In this study, we explore the possibility that wearable eye-tracking technology can be used to record paramedic students’ attention focus, and that this may help to understand their level of SA. To the best of our knowledge, this question has not been theoretically or empirically tested in the paramedic profession. We consider the problems of measuring SA and review current eye-tracking systems to see whether they might offer solutions. Our findings could be used to develop systems and techniques that provide useful feedback to students and assist the further development of SA as well. Improved SA may ultimately lead to reduced rates of medical error and improved patient safety.

Situational awareness: the concept

The concept of SA was developed in the late 1980s for the military aviation sector,17 but began to be applied more broadly to “dynamic systems” by the mid-1990s. The seminal paper on SA, published by Endsley in 1995,11 applied the concept to aviation, manufacturing, military strategy, driving, and medical decision-making. His paper introduced a three-layer model of SA that incorporates perception and comprehension of the situation and projection of future events.11,16 Components of SA include observation, verification of observation, problem recognition, allocation of attention, and prioritization (Figure 1).

Figure 1

Elements of situation awareness – cognitive processes.

However, it is also important to note that various broader senses of “situational awareness” are also often referred to in studies relating to emergency health care.18–20 Some, for example, refer to the awareness of the many factors that may affect major incident management, such as the availability of resources and locations of casualties and hospitals.21,22 In the present study, however, when we refer to SA, we are employing Endsley’s definition, with a focus on individual perception, comprehension, and judgment.

Measuring situational awareness

Various attempts have been made to evaluate or measure SA, both in clinical and in simulated clinical settings.10 Observer-rating techniques employ expert observers to rate the performance of participants in a clinical scenario.23,24 Whereas these may be useful in evaluating the subject’s technical skill, they are insensitive to the “inner world” of cognition23 and do not measure perception, comprehension, or projection. In a study of the visual attention (and hence, perception) of anesthetists, covert observers reported that the subjects spent 5% of their time looking at the patient monitor. Objective eye-tracking data in a similar study suggested that the subjects actually spent 30% of their time looking at the monitor.25,26 This discrepancy suggests that it is inherently difficult for one person to observe the attention of another. Further, observation techniques require the time and expense of expert observers, who may bring with them their own experiences, ideas, attitudes, and biases.

More successful are techniques that combine observer-rating with participant self-report.27 During a clinical simulation, the scenario can be halted and participants immediately asked questions that explore the three levels of their SA. For example, a student in a clinical simulation might be asked about perceptions of things such as vital signs, what meaning those vital signs have in the current context, and their projections of what may happen as the scenario progresses. This general approach has been formalized by Endsley as the Situation Awareness Global Assessment Technique (SAGAT)28 and has been shown to have reliability and validity in clinical training scenarios.29,30 Variations on the SAGAT technique have been used to measure nurses’ SA in a scenario involving a simulated deteriorating patient,31 and student midwives’ SA in scenarios involving postpartum and antepartum hemorrhage.32

SAGAT requires that a scenario be halted so that questions can be asked. Not surprisingly, a technique that does not involve halting the scenario has also been developed. In the Situation Present Assessment Method (SPAM) of measuring SA, an audible warning indicates to subjects that a question is waiting.33,34 Subjects then reply to the request and answer the question when they are able to do so. The delay between the warning and the request are assumed to be a measure of workload, whereas the delay between the question and the provision of the correct answer are assumed to be an inverse measure of SA.

The SPAM measure arose from experiments that attempted to measure the SA of chess players of varying skill levels.34 It was found that the ratings of SA produced by SPAM correlated well with the already known expertise level of chess players. SPAM has recently been reevaluated in a simulated air traffic control environment and, in general, recent findings support those of the original experiment.33

The original SPAM experiment is also interesting because it evaluated eye-tracking as a measure of SA.34 Whereas eye movement was found to have little or no correlation to SA in 1995, nearly two decades of development in eye-tracking devices and the possibility of different experimental designs suggest gaps in the research field. With the exception of a recent study that used personal digital assistants to administer SPAM questions,9 there has been little or no other work that uses technology to provide an objective measure of SA. Eye-tracking, either used alone or in conjunction with other techniques, may hold promise; this again highlights a gap in the literature.

Eye-tracking

Eye-tracking is the process of recording movement of the eyeball to determine what a subject is looking at as they perform some task.12 Devices used to do this have evolved since 1898 when Edmund Delabarre, then Professor of Psychology at Brown University, anesthetized his eyeball with a cocaine solution and fused a plaster-of-Paris dome to its surface.35 A small wire ring was set in the plaster, a thread attached to it, and while Professor Delabarre peered through a small hole in his rudimentary “contact lens”, movements of his eye were recorded via a lever mechanism attached to the thread. In similar research, he attempted to attach a tiny mirror to his eyeball and record the beam of light that it reflected onto a photographic plate – a technique similar to that used today.

The full details of how modern eye-tracking devices achieve their aim is beyond the scope of this article, but most rely in one way or another on using an image of the eye or reflections from its multiple surfaces to estimate the direction of gaze. The eye creates at least four reflections that are visible to an external observer, referred to as the Purkinje images P1 through P4: P1 is the reflection from the external surface of the cornea; P2 the reflection from its inner surface; P3 is reflected from the anterior surface of the lens; and P4 from its posterior surface.36 Most eye-tracking algorithms work by measuring the movement and offset of P1 and P4,36–38 calculating the plane of the iris based on the fact that it is a circle that will be seen as an ellipse,39–42 measuring reflections from or images of the fovea,43–45 or deriving data from some other externally visible aspect of the eye.46

Recording the position of the eyeball, however, is only a part of the picture. For eye-tracking to be truly useful, the field of view must be either held constant or accurately recorded.12 This means that the subject’s head position must be fixed in relation to the eye-tracking device, or that its movements are constantly recorded, which became practical only relatively recently. Until 2011, eye-tracking was limited to studying fixed tasks, such as the way we read the text on a page or the order in which we look at the different elements on a Web page. Now, however, it can be used to study dynamic tasks. Eye-tracking devices can be combined with video cameras in wearable devices,47 essentially as a pair of glasses that watches your eyes while you watch the world and records both your field of vision and what you are looking at within it. The output from such systems is typically a video with a small moving dot or circle that moves to show the exact focus of vision.

Linking gaze and cognition

Of course, knowing what someone is looking at does not necessarily reveal what they are thinking about. We all know it is possible to look out the window while thinking about some abstract problem or simply daydreaming. And, whereas there is research that suggests a strong link between eye movement and cognition,13,48,49 there is no evidence that conclusively links the attention of gaze to the attention of the mind.50 This is hardly surprising, as we are yet to devise a way to see what is going on in a person’s mind. It may be intuitive and agreeable to assume that when we look at something, we think about it at least to some extent; but we can also imagine doing some well-practiced task without directing our gaze toward it. We can tie our shoelaces without looking at them, for example. Further research on the relationship of gaze and cognition may change the way in which eye-tracking studies are designed and interpreted. For the time being, however, eye-tracking does have some value, even if based on the premise that gaze and cognition are linked.

Eye-tracking in clinical education

Eye-tracking systems as previously described have been used to aid the design of Web pages,51 to study consumer behavior in a retail environment,52–55 and to investigate eye movements during complex tasks such as driving a car.56,57 These systems have also made valuable contributions to the study of cognition and neurological development.58–63

The use of eye-tracking in the study or development of clinical education, or indeed in any clinical setting, is limited. One study used eye-tracking to see whether clinicians looked at patient identity bands before administering medications.64 Another study used eye-tracking to determine the amount of time that anesthetists looked at a patient monitor.25 In both these studies, the eye-tracking data provided useful insights. The patient identity study revealed that clinicians did look at patient armbands and identity labels, but tended not to notice discrepancies. What may have been put down to the human error of not checking identification is revealed as a more complex problem of not noticing, or acting on, discrepancies. The study by Schulz et al25 showed that novices spend more time looking at the patient monitor than experts; whether similar findings are found in other areas of medicine and allied health, such as resuscitation and critical care, is yet to be formally examined.

We found no studies that link eye-tracking, SA, and clinical education; to the best of our knowledge, no formal research has been published that uses eye-tracking technology to examine the SA of paramedics. This, combined with the current wearable nature of eye-tracking technology, provides rich opportunities to researchers who wish to explore eye-tracking in paramedic studies. Valuable insights could be gained from the use of eye-tracking during paramedic education in simulated scenarios, during professional development sessions, and even during paramedics’ on-road shifts; though the latter is likely to be difficult, given the many ethical dilemmas paramedics confront.

Although validated approaches have yet to be developed (educationally) or empirically tested (scientifically) that explore eye-tracking technology in paramedics, the fact that a general lack of epidemiological data65 regarding paramedic-related patient safety exists suggests that it is worth some exploration. This would be particularly useful in paramedic internship training rotations and continuing professional development activities, such as role plays, high-fidelity simulation, and/or scenarios with simulated patients.

Future examination and empirical testing of eye-tracking technologies are important because little is known about their use in different aspects of health care – for example resuscitation, airway management, or defibrillation. How practical, feasible, and cost-effective eye-tracking technology will be for medical and health educators are key questions to be addressed. These questions are particularly important for tertiary institutions with large student cohorts where clinical teaching time and opportunities impose complex demands.

Conclusion

Given the importance of SA to the quality of health care and to the safety of patients, it is possible that eye-tracking technology may offer new and important insights into these areas. As new technology makes eye-tracking devices both more affordable and more usable, we look forward to the insights that this technology may reveal about the work of health care professionals in dynamic, uncontrolled, and emergency situations, and particularly in paramedic education.