Preliminary study of eye tracking to investigate the differences in gaze behaviors depending on the experience of neuroendovascular therapy.

BACKGROUND: Neuroendovascular therapy is now the choice for the management of many neurovascular pathologies, and physicians with endovascular skills are in high demand. In addition to the traditional method of practicing hand movements to learn skills, a new strategy of practicing eye movements to learn skills is also attracting attention. This preliminary study explored the differences in gaze behavior depending on experience with endovascular procedures to be facilitated in future skill training in neuroendovascular therapy.
METHODS: Four physicians with experience of 3-412 neuroendovascular procedures wore eye-tracking devices during coil embolization of swine cervical arteries. Gaze metrics with direct correlations to the expertise of endovascular procedures were explored.
RESULTS: Gaze metrics with a positive direct correlation to experience included the proportion of fixation durations (PFD) in the screen area and the native images. Those with a negative direct correlation included the PFD in the off-screen area and the roadmap images and the average fixation durations in the off-screen and coil areas. During the parent artery occlusion procedure with detachable coils, more experienced operators preferred to look at native images rather than roadmap images and that less experienced operators tended to look down at their hands more frequently.
CONCLUSION: This preliminary study demonstrated the feasibility of eye tracking to identify the differences in gaze behavior depending on the experience of endovascular procedures and may guide future eye-tracking studies in neuroendovascular therapy. Copyright:

INTRODUCTION

While neuroendovascular therapy (NET) was considered a second-line approach to treat neurovascular pathologies before 2000, it is now the approach of choice;[8,10] thus, physicians with endovascular skills are in high demand. Unfortunately, skill learning in NET is not easy. For example, in 2020, acceptance rates for the technical skills tests in the board certification program of the Japanese Society for Neuroendovascular Therapy were as low as 60%.[4] While it has generally been accepted that long term, on-site training programs where trainees watch and practice the hand movements of experts are necessary to master the art of NET, a new skill learning strategy is now attracting attention; practicing the eye movements of experts.[9,11]

Eye movement characteristics are thought to differ between experts and nonexperts.[1,6] Repetitive exposure to similar scenes is believed to help experts understand where they need to focus their visual attention to obtain relevant information from a scene more efficiently. Thus, experts’ visual attention tends to be more concentrated on relevant areas of interest and diminished in irrelevant areas.[1] These differences may have useful implications during skill training. For example, in a study using a virtual laparoscopic surgical simulator, eye movement training was reported to lead to the acquisition of relevant skills more rapidly than hand movement training.[11]

While expert eye movement characteristics likely have useful applications to skill learning in NET, eye movement characteristics cannot be generalized across different domains of expertise.[1] Therefore, it is necessary to independently investigate the distinctive eye movement characteristics of experts in various fields, including NET. In this study, physicians with varying levels of experience were recruited, and their eye movements during endovascular procedures were examined to explore the gaze behavior characteristics of experienced operators.

MATERIALS AND METHODS

Ethical considerations

The ethical aspects of this study were reviewed and approved by the ethical committee of the research institute. All participants provided written informed consent. All experimental procedures were conducted according to the Guidelines for the Care and Use of Laboratory Animals of the research institute.

Subjects

Study subjects were recruited from neurosurgeons who belonged to the affiliated institutions of the researchers. Among the 20 board-certified neurosurgeons informed of the study, four chose to participate. The participant experience level for endovascular procedures was quantified as the number of NET cases in which he or she was the operator.

Preparation of the animal

A 35 kg male swine was obtained from Tokyo Laboratory Animals Science Co., Ltd. (Tokyo, Japan). The habituated animal was inducted into anesthesia by intramuscular injections of ketamine and xylazine. After intubation, anesthesia was maintained with isoflurane inhalation. Under sterile conditions, both sides of the groin were incised to expose the femoral arteries. Six-French sheath introducers were cannulated into the bilateral femoral arteries under direct vision. After the 8 h experiment, euthanasia was performed using an intravenous injection of a pentobarbital overdose.

Task

Each study participant was assigned the task of interrupting antegrade blood flow at a bifurcation of a swine cervical artery by coil embolization under X-ray fluoroscopy. The X-ray fluoroscopic apparatus had a single flat detector and a 60” monitor (INFX-8000V, Canon Medical Systems Corporation, Tochigi, Japan).

The vessel diameters and appearances were nearly identical at the site where the maxillary artery gave off the facial artery and where the subclavian artery bifurcated into the axillary and subscapular arteries. Therefore, each subject took charge of one of these four bifurcations for the task. The subjects were instructed to completely interrupt the antegrade flow by putting three detachable coils compactly within the area designated by the researchers. Each subject could freely choose detachable coils from the lineup with varying sizes and lengths.

Recording of eye movements

For recording eye movements, each subject put on a head-mounted, eye-tracking device (Tobii Pro Glasses 2, Tobii AB, Stockholm, Sweden), as shown in [Figure 1]. Before recording each subject, the eye-tracking device was calibrated to ensure that the spatial error was within 1.15° of visual angle, that is, the error at the monitor at a distance of 1.5 m was within 3 cm.

Figure 1:

Eye-tracking device. (a) The eye-tracking device used in this study consisted of a glasses unit, a recording unit, and a mini-HDMI cable. (b) The operator (right) is wearing the eye-tracking device, and the assistant is wearing ordinary glasses. The recording device is attached to the operator’s back and is not seen in this figure. Informed consent was obtained from the subjects for the posting of this photo.

The coil insertion period, from when the coil arrived at the tip of the microcatheter to when the delivery wire was removed after the coil was detached, was determined for each coil. Only the data from that period were further analyzed. The time required for coiling was defined for analysis as the sum of the coil insertion time for each subject.

Using gaze data recorded at 100 Hz, gaze points that fell within a narrow area for a short period were grouped as fixations using the Tobii I-VT (attention) filter.[7] Fixations were then remapped to still fluoroscopy captured images by comparing the camera images from the eye-tracking device to the fluoroscopy images recorded during the procedure. The mapping procedure was performed in a manual frame-by-frame manner [Figure 2].

Figure 2:

Areas of interest for analysis (upper) and gaze plots (lower). The upper figure presents the area of interest set up for subject 1. The rectangular area with orange lines defines the screen area of interest. The rectangular areas with yellow continuous and dotted lines define the roadmap fluoroscopic image and the native fluoroscopic image. The circular areas painted in blue and red represent the coil area of interest and the microcatheter area of interest, respectively. The lower figure shows gaze plots of subject 1 during the coil insertion procedure. The yellow circle represents the fixation site, and the diameter of the circle is proportional to the fixation time.

Areas of interest

Areas of interest (AOI) were drawn on the still images, wherein the fixations had been mapped [Figure 2]. Within the screen area, two rectangular areas displaying native images and roadmap images were drawn. Circles, where the coils and proximal marker of the microcatheter, were placed were also drawn.

The diameters of the coil AOI and microcatheter AOI were 16 times and 8 times that of the 6-Fr guiding catheter, respectively. The area outside the screen was defined as off-screen AOI.

Gaze metrics

The duration of each fixation was summed in each AOI as the total fixation duration. For between-subject comparisons, the proportion of fixation duration (PFD) was calculated as the ratio of the total fixation duration to the time required for coiling for each area of interest. This metric has been reported to reflect the degrees of the attention of subjects; the importance, informativeness, and visibility of targets; and the difficulty of tasks.[3]

Each fixation had its duration, and the average fixation durations (AFDs) were calculated for each AOI. This metric has been reported to increase or decrease based on the efficiency of visual intake, informativeness of targets, cognitive workload to be processed, and visibility of targets.[3]

Data analyses

Spearman’s rank correlation coefficients (ρ) were calculated between experience level and the gaze metrics to identify specific gaze metrics related to experience level (i.e., the number of NET cases as an operator). An absolute value of ρ = 1.0 meant a direct correlation between experience level and a gaze metric.

RESULTS

The number of NET cases as an operator varied from 3 to 412 cases in the study participants. All subjects succeeded in completely interrupting antegrade blood flow by coil embolization. The average time required for coiling was 590.8 s.

The time required for coiling tended to decrease as the experience level increased, though the correlation with experience level was not direct (ρ = −0.8).

Relationships between the experience level of operators and gaze metrics are shown in [Table 1]. Gaze metrics directly correlated with experience level included PFD in the screen and off-screen area, PFD in the native and roadmap images, and AFD in the roadmap images and coil area [Figure 3].

Table 1:

Experience level and gaze metrics.

Figure 3:

Gaze metrics directly correlating to experience. Among the gaze metrics analyzed in this study, only the gaze metrics that demonstrated direct correlations with experience are shown in the line charts. The figures between parentheses along the horizontal axis are the number of experienced cases. AFD: Average fixation duration, PFD: Proportion of fixation duration.

On average, the PFD in the screen area was 89.5%, with the remaining 10.5% being devoted to the off-screen area. When subjects were watching the off-screen area, they were mainly looking down at their own hands. There were direct relationships between experience level and PFD in the screen area (ρ = 1.0) and PFD in the off-screen area (ρ = −1.0) [top of Figure 3].

PFD in the native images and PFD in the roadmap images correlated directly with the experience level (middle of Figure 3), indicating that more experienced operators spent more time looking at native images. In contrast, less experienced subjects preferred roadmap images.

There was no direct relationship between the PFD in the coil and microcatheter areas and the experience level; however, the PFD in the coil area tended to decrease slightly as the experience level increased (ρ = −0.8).

AFD tended to decrease as the experience level increased when compared between subjects. In particular, experience level correlated directly with AFD in the roadmap images and AFD in the coil area in a negative manner (bottom of Figure 3).

DISCUSSION

This preliminary study performed eye tracking of four neurosurgeons during coil embolization of swine cervical arteries and identified several gaze metrics directly correlated with experience levels of endovascular procedures.

It is generally expected that more experienced operators will make quicker decisions. More experienced operators would be more likely to distribute their visual attention over a broader area if so. Part of this expectation was demonstrated by the shorter AFD of the more experienced operators. However, the experienced subjects’ gaze behavior differed from these expectations in that their visual attention was biased toward the native image rather than being broadly distributed between the roadmap and native images.

The characteristic deflection of the gaze to the native fluoroscopy by experienced subjects may be attributed to the simplicity of the task employed in this study. The roadmap image provides a fluoroscopic image of the coil overlaid on the vessel image, but the digital imaging process leads to the coil’s slightly unnatural appearance. In the parent artery occlusion procedure, once the first coil is placed, an experienced operator can see the vessels’ approximate course by merely looking at the native image. Besides, almost all operator considerations are to ensure that the coils are densely inserted. For this reason, a more experienced operator may have concentrated on native fluoroscopy with more natural coils’ appearance. If the task was complicated, such as the aneurysm embolization procedure, experienced operators’ gaze behavior may have differed from that obtained in this study. Experienced operators may have frequently switched their gazes between the roadmap and native images because the positional relationship between the coils and the vessels is also essential to avoid the aneurysm rupture or the coil protrusion into the parent vessel. The gaze behavior that serves as a learning model may differ from procedure to procedure. To further this study, it seems necessary not only to generalize the findings by increasing the number of subjects but also to investigate variations in gaze behavior in various procedures.

Less experienced operators spent more time looking out of the screen than experienced operators, and they tended to look down at their hands during that time. The PFD in the off-screen area was 17.5% in subject 1, with the least amount of experience, and was only 5.6% in subject 4, who had the largest amount of experience. In a study by Law et al.,[5] experts tended to look at the target during manipulation of laparoscopic tools, while novices looked down at their devices to compensate for lack of proprioceptive feedback. This difference in gaze behavior may be used to distinguish between varying NET experience levels. Awareness of right and wrong gaze behavior may improve skill learning for NET beginners.

In all areas of interest, AFD was negatively correlated with experience levels. Particularly, AFD in the roadmap image and coil area had a direct negative correlation with the experience level. Whether the AFD grows longer or shorter with experience depends on the type of task. Experts in inspection work are supposed to have acquired effective templates for processing scenes fast and implicitly, and AFD is reported to get shorter with experience.[3] On the other hand, delicate manual procedures such as cutting and suturing under the operating microscope have been reported to result in longer AFDs for more experienced operators than for less experienced operators.[2] While neuroendovascular procedures require the ability to rapidly examine fluoroscopic images and the ability to manipulate therapeutic devices carefully, this preliminary study may indicate that shorter AFD is characteristic of experienced operators in neuroendovascular therapy. If the experience level of individual operators could be identified quickly based on gaze metrics such as AFD, the instructor may tailor the training to the trainee’s experience. A computer-aided surgery system may also provide personalized support depending on the experience level of operators.

This study had several limitations. First, due to the small number of subjects, the results of this preliminary study need to be reconfirmed in studies with increased numbers of subjects before applying to actual skill learning in neuroendovascular therapy. Second, the blood vessels assigned to each subject were different. However, the diameter and the shape of the bifurcation were almost identical in the three-dimensional images. Thus, the influence of this difference may have been not significant. Third, the task adopted in this study might have been too easy compared to the actual endovascular procedures. In future studies, it may be better to devise more complex tasks or perform eye tracking in clinical cases while giving due consideration to safety. Further, there are some inevitable errors when measuring gaze points with eye-tracking devices. In this study, the eye-tracking device was calibrated to keep the spatial errors less than a 1.15° visual angle; however, the manual mapping procedure for fixations could also have been a source of spatial errors of fixation. The estimation of errors during the manual mapping procedure was not available in this study.

In actual neuroendovascular procedures, the four screens of the biplane fluoroscopy are referenced, and operator-assistant pairs work together to distribute their visual attention to the four screens. Simultaneous eye tracking of operator-assistant pairs to investigate intraoperative cooperation in terms of visual attention sharing will be a future issue to be investigated.

CONCLUSION

This preliminary study demonstrated the feasibility of eye tracking to identify the differences in the gaze behavior depending on the experience of endovascular procedures and may guide future eye-tracking studies in neuroendovascular therapy.