Increasing Navigation Speed at Endoluminal CT Colonography Reduces Colonic Visualization and Polyp Identification.

Purpose To investigate the effect of increasing navigation speed on the visual search and decision making during polyp identification for computed tomography (CT) colonography Materials and Methods Institutional review board permission was obtained to use deidentified CT colonography data for this prospective reader study. After obtaining informed consent from the readers, 12 CT colonography fly-through examinations that depicted eight polyps were presented at four different fixed navigation speeds to 23 radiologists. Speeds ranged from 1 cm/sec to 4.5 cm/sec. Gaze position was tracked by using an infrared eye tracker, and readers indicated that they saw a polyp by clicking a mouse. Patterns of searching and decision making by speed were investigated graphically and by multilevel modeling. Results Readers identified polyps correctly in 56 of 77 (72.7%) of viewings at the slowest speed but in only 137 of 225 (60.9%) of viewings at the fastest speed (P = .004). They also identified fewer false-positive features at faster speeds (42 of 115; 36.5%) of videos at slowest speed, 89 of 345 (25.8%) at fastest, P = .02). Gaze location was highly concentrated toward the central quarter of the screen area at faster speeds (mean gaze points at slowest speed vs fastest speed, 86% vs 97%, respectively). Conclusion Faster navigation speed at endoluminal CT colonography led to progressive restriction of visual search patterns. Greater speed also reduced both true-positive and false-positive colorectal polyp identification. © RSNA, 2017 Online supplemental material is available for this article.

Introduction

Since its original description in 1994, computed tomographic (CT) colonography has been adopted internationally because it is sensitive for colorectal cancer and polyps, well tolerated, safe, and relatively inexpensive (1,2). Competent interpretation requires both two-dimensional and three-dimensional image display, although relative contribution varies by radiologist preference. Interpretation is time consuming, particularly for three-dimensional interpretation (3,4); prolonged interpretation time will reduce the number of studies interpreted per working day. Radiologists are increasingly under pressure to maximize reading speed to accommodate service demand, in both screening and symptomatic practice.

CT colonography interpretation software permits the reader to vary the speed at which the volume-rendered viewpoint (ie, the virtual colonoscope) progresses along the colonic centerline. Faster navigation lessens time taken to traverse the colon, but individual radiologists’ preferred speeds for colonic navigation are unknown, as is the effect of increasing speed on diagnostic accuracy, if any. When interpreting two-dimensional images such as chest (5) or bone (6) radiographs, radiologists are capable of high sensitivity even at considerably reduced viewing times—as low as 4 seconds for pulmonary nodule detection at chest radiography in one study (7). It is therefore plausible that radiologists could increase CT colonography navigation speed without compromising diagnostic performance. However, this is speculative; what is true for interpretation of static two-dimensional radiographs may not apply to moving three-dimensional images, where abnormality changes in shape, size, and position. To our knowledge, the effect of faster velocities on visual interrogation and detection of colorectal polyps is currently unknown. Technology that tracks eye gaze (referred to here as eye tracking), whereby gaze is monitored during interpretation, can document and quantify these parameters (8) because it permits the study of both diagnostic decision making and the nature of image visual interrogation simultaneously.

By tracking eye gaze of radiologists during interpretation, we aimed to investigate the effect of increasing navigation speed on the visual search and decision making during polyp identification for CT colonography.

Materials and Methods

Permissions

Ethical approval was granted by the University College London committee (project identification: 5967/001) for this prospective study. Anonymized CT colonography images were derived from institutional review board and research ethics committee–approved studies (9,10). Written informed consent was obtained from the readers.

CT Colonography Data Sets and Video Generation

CT colonography data from 112 patients (symptomatic, 11 patients; screening, 101 patients) were collated from three U.S. and two European centers between January 2002 and June 2005. A reference standard for the presence and location of polyps in these patients was established in consensus by three radiologists (including S.A.T. and S.H., each with experience of >1000 CT colonography cases and >10 years, and a radiologist nonauthor with experience of >500 CT colonography cases and >5 years), each of whom interpreted each dataset twice, assisted by colonoscopy reports (10). From this validated case set, following a suitable power calculation, a selection of 12 fly-through videos was generated by using a commercially available CT colonography workstation (Vitrea; Vital Images, Minnetonka, Minn). Images had been acquired on a 16 multi–detector row CT imager (Lightspeed Plus or Ultra; GE Medical Systems, Milwaukee, Wis) by using a section thickness of 1.5–3 mm, a reconstruction interval of 1 mm, variable mAs (50–100 mAs), and 120 kVp. Patients received oral contrast agent tagging but no intravenous contrast agent.

Eight true-positive cases (P1–P8, each depicting a single 5–8-mm polyp) and four true-negative cases (N1–N4) were selected. For each of these 12 cases, an image perception scientist (P.J.P.) produced four videos that were identical other than they had different navigation speeds along the colonic centerline. For each case, a source video was exported from the CT colonography workstation and fixed colonic landmarks (eg, diverticula or polyps) were subsequently identified. An experienced colonographer (A.A.P, >1000 cases and >8 years of experience with CT colonography) then measured the distance along the colonic centerline between these fixed landmarks. This distance was then used to establish navigation speed during standard playback in centimeters per second. This was then adjusted to achieve navigation speeds of 1 cm/sec, 1.5 cm/sec, 3 cm/sec, and 4.5 cm/sec (referred to as speeds 1–4, respectively). These speeds were selected after discussion with several CT colonography experts and after a pilot experiment that used five volunteers to confirm that the videos spanned a plausible range of CT colonography interpretation speeds.

Eye-tracking Procedure

Piloting showed that viewing longer than 25 minutes or more than 40 videos caused fatigue. We therefore randomly selected 40 videos for each reader by using block randomization (five blocks of eight videos). Within each block, one video was at speed 1, one was at speed 2, and there were three videos each for speeds 3 and 4. This structure imposed an adequate mix of differing speeds and maximized the total number of video viewings (because faster speeds require less time to view) while constraining the experiment within approximately 25 minutes. Thus, the 40 viewings undertaken by each reader were drawn from the same set of videos, but the mix of speeds per video differed between readers. Randomization was performed separately for each reader by using the sample command in R version 3.0.1 (R Foundation for Statistical Computing, Vienna, Austria) (11).

Viewing was conducted in a quiet area at the European Society of Gastrointestinal and Abdominal Radiology CT colonography training workshop (Leeds, 2014) by using the central 512 × 512 pixels of a 1280 × 1024 pixel thin-film transistor monitor (ProLite B1902S; Iiyama, Hoofddorp, the Netherlands) with a 4:3 aspect ratio and 19-inch diagonal display. Ambient light levels were not measured formally, but were held constant during the experiments and were subjectively similar to a radiology reporting room with no external natural daylight. During case viewing, an infrared eye tracker (X120; Tobii Technology, Stockholm, Sweden) was used to monitor pupil movements. The device is a small set of sensors that can be positioned beneath a normal monitor. Readers were asked to indicate when they had detected a polyp (not the polyp’s location) and received the following instructions before case viewing:

You are about to see some videos of CT colonography endoluminal “fly-through” examinations, some of which have polyps. These will be displayed at different speeds. Please click your mouse when you see a lesion that you think is highly likely to be a real polyp or cancer.

Before eye tracking, we recorded the preferred navigation speeds of the radiologists, both before and after the main experiment. A further CT colonography video was displayed at a known but user-controllable velocity, and participants were invited to set the video playback to their normal clinical interpretation speed.

Power and Sample Size

Sample size was estimated by simulation to allow for the cross-classified nature of the design by using variance estimates from a previous study (12). A sample of 25 readers, each viewing four videos at speed 1 and 10 videos at speed 4, had 80% power to detect a reduction in true-positive polyp identifications from 60% (speed 1) to 40% (speed 4) at a 5% level of significance and 90% power to detect a reduction from 60% (speed 1) to 35% (speed 4). To prevent under powering because of the risk of under recruitment, ultimately we raised the number of case viewings to five each of speeds 1 and 2 and 15 each of speeds 3 and 4 (which yielded 40 videos per reader).

Participants

All participants provided written informed consent. We recruited experienced (>2 years of experience with CT colonography and >300 total case experience, including >100 cases in the preceding 12 months; n = 13) and inexperienced (not meeting these criteria; all had <200 cases of experience; n = 10) radiologists. We recorded participant age, sex, clinical seniority or grade, previous experience with CT colonography (years and cases), previous radiologic experience in years, preferred CT colonography reading strategy (two-dimensional vs three-dimensional), and number of CT colonography examinations interpreted in the preceding 12 months.

Postprocessing and Analysis of Eye-tracking Data

To determine if readers’ gaze pursued a polyp, we followed a procedure similar to published experiments (8,12–14). In brief, a visual perception scientist (P.J.P., 8 years of experience handling eye-tracking data) assisted by a radiologist (A.A.P., 4 years of experience with eye tracking and 8 years of experience interpreting CT colonography examinations) in consensus outlined the approximate boundary of individual polyps with a spherical region of interest. We defined eye pursuit of a polyp as occurring when the reader’s gaze fell within 50 pixels of this polyp boundary for a continuous period of 100 msec or more. Eye pursuits were analyzed as follows: the presence of at least one pursuit during viewing of a video, the number of pursuits during viewing of a video, and the rate of pursuits per second of video display. Total pursuit time was defined as the proportion of the total time the polyp was on the screen during which eye pursuit of that polyp occurred. Time to first pursuit was defined as the proportion of this time that had elapsed before the first eye pursuit occurred.

We treated all mouse clicks while the polyp was displayed on screen (plus a 500-msec subsequent window for reaction time) as representative of true-positive polyp identifications. All mouse clicks that occurred before the polyp was displayed on screen were treated as false-positive identifications. Mouse clicks that occurred more than 500 msec after the polyp had disappeared from the screen were ignored because we could not distinguish with certainty whether these were false-positive polyp identifications or delayed but correct clicks after a longer period of reader consideration. We also conducted a sensitivity analysis in which these clicks were regarded as false-positive identifications, which inspection of the eye-tracking data showed was the more plausible of the two scenarios.

We also measured reader eye-gaze distribution during video interpretation by dividing the 512 × 512 pixel video size into two areas: a 256 × 256 pixel central square (25% total video area) and the peripheral 75% beyond this; the periphery was further subdivided into equal upper and lower halves.

Statistical Analysis

Data were collated by using Excel (Microsoft, Redmond, Wash) and analyzed by using statistical software (Stata version 14.0, Statacorp, College Station, Tex; and R version 3.2.4, R Foundation for Statistical Computing). The effect of navigation speed along the colonic centerline on polyp identification and eye-tracking metrics were tested by using multilevel regression modeling (a maximum likelihood method), with independent random terms for reader and case to allow for clustering, which included speed as a fixed factor variable and was adjusted for reader level of experience.

Short runs of missing eye-tracking data were imputed by using the immediate preceding and following eye coordinates, with the addition of random measurement error. Summary estimates were then pooled by using multiple imputation methods (15) with 10 imputations. A small number of cases (18 of 920 cases; 2.0%) with either more than 50% missing data values in total or more than 50 consecutive missing values (100 missing values for the screen coverage metric) was regarded as unreliable and excluded.

Results are expressed relative to speed 1 (1.0 cm/sec, slowest) as the reference category. A sensitivity analysis included an interaction term between speed and level of experience. Effect sizes were summarized by using odds ratios or rate ratios as appropriate, with 95% confidence intervals; P values less than .05 were indicative of statistical significance.

Results

Twenty-three readers participated (18 men; mean age, 40 years; age range, 30–63 years). Experienced readers (n = 13) had an average of 10 years of experience with CT colonography.

Preferred Navigation Speed

The average preferred navigation speed was 1.18 cm/sec ± 0.52 (standard deviation) at the start of the experiment, with no significant difference between experienced and inexperienced readers (experienced readers vs inexperienced readers, 1.22 cm/sec vs 1.12 cm/sec; mean difference, 0.10 cm/sec; 95% confidence interval: 0.53, −0.33; P = .62). After case viewing was completed, preferred speed was similar (1.18 ± cm/sec 0.39; experienced readers vs inexperienced readers, 1.13 cm/sec vs 1.24 cm/sec; mean difference, −0.11 cm/sec; 95% confidence interval: −0.45, 0.24; P = .53).

Neither level of experience nor viewing order were related to polyp identification or gaze tracking metrics; therefore, results are shown for all readers combined. For three metrics (proportion of gaze within central region, at least one polyp pursuit, and time to first pursuit of polyp) multilevel model fitting was unreliable because of large variation between cases, so the results for these variables are presented descriptively.

True-Positive Polyp Identification

The proportion of viewings during which the reader made a true-positive polyp identification decreased significantly at higher speed, from 56 of 77 (72.7%) of viewings at speed 1 to 137 of 225 (60.9%) of viewings at speed 4 (P = .004; Tables 1, 2). This effect of speed on true-positive polyp identification was highly case-dependent (Fig 1), which influences the estimate of the effect size. Some polyps (eg, P7) showed a gradual decline in detection as speed increased, while others (eg, P2 and P5) had a consistent rate of polyp identification regardless of speed. Figure 2 shows the delaying effect of video speed on decision time for polyp identification, with clicks made while the polyp was visible (or within the allowed 500 msec reaction time) tending to occur slightly later at faster speeds relative to the total duration of the video.

Table 1

Summary of Metrics by Speed

Note.—Data are numerator and denominator and data in parentheses are percentage unless otherwise indicated.

*Data are median; data in parentheses are range.

Table 2

Comparison of Metrics between Speeds Derived from Multilevel Models

Note.—Data in parentheses are 95% confidence intervals.

Figure 1:

Graphs show percentage of all viewings, at each speed, for which there was at least one pursuit, and for which there was at least one polyp identification. Separate figures are shown for each true-positive case (P1–P8). Percentages in parentheses are the proportion of the total video duration during which the polyp was on the screen. The total number of viewings (N ) is shown at the top of each graph.

Figure 2:

Plot shows timing of all polyp identifications for all video viewings. Each polyp identification is shown as a single point, separately for each speed. Times are shown as the proportion of the whole video duration; hence, the x-axis refers to a longer period in real time for videos at slower speeds. Periods when the polyp was on screen are indicated by the pink shaded rectangles, and the allowed reaction time is indicated by a red vertical dash. Cases N1–N4 are without polyps and cases P1–P8 are with polyps.

False-Positive Polyp Identification

The proportion of viewings with false-positive identifications also declined significantly with speed, from 42 of 115 (36.5%) at speed 1 to 89 of 345 (25.8%) at speed 4 (P = .02; Tables 1, 2). Figure 2 depicts clusters of false-positive identifications, which implied that the presence of specific endoluminal features provoked false-positive findings at all speeds (eg, one-third of the way through video P8). These corresponded to diverticula, areas of fecal residue, or motion artifacts. The rate of false-positive identifications (considering video duration) was higher at faster speeds (Tables 1, 2). In the sensitivity analysis (in which late clicks were treated as false-positive polyp identifications), the effect of speed was still evident (Tables 1, 2).

Reader Gaze Distribution

Aggregated across all viewings, the proportion of time spent viewing the central 256 × 256 pixels increased from 84% at speed 1 to 95% at speed 4 (Table 1), illustrated by the heat-map in Figure 3. For 90 of 902 viewings (10.0%), readers’ gaze did not move to the peripheral region of the screen throughout the entire video: 68 of these viewings were at speed 4 and 22 were at speed 3. There were no instances of this “tunnel vision” at the two slowest speeds. An example of the influence of speed on the pattern of eye gaze for a single reader is shown in the Movie and Figure E1 (online). Reader gaze at speed 1 showed comprehensive interrogation of the central field, with occasional visits to the periphery, whereas at speed 4 the reader’s gaze position became almost exclusively central.

Figure 3:

Heat map showing recorded eye locations when no polyp was visible, aggregated over all viewings at each speed. The outer solid 512 × 512 pixel square indicates the monitor’s display area, and the inner dashed 256 × 256 pixel square the central region. A, Speed 1; B, speed 2; C, speed 3; and D, speed 4.

Movie.

Video depicting reader gaze for two viewings of the same case (Polyp P4) at speed 1 and speed 4. Gaze is shown as a semitranslucent blue marker on the video and as a heat map in Figure E1 with warmer (redder) colors corresponding to longer gaze times.

Polyp Eye Gaze Pursuits

At speeds 1 and 2, a polyp pursuit was recorded in 69 of 77 (89.6%) and 75 of 77 (97.4%) of video viewings, respectively. Only a single polyp was pursued on fewer than 90% of occasions at these slower speeds (P3; Fig 1). This specific polyp was subjectively inconspicuous (Fig E2 [online]). Conversely, at higher navigation speeds, polyps were frequently not pursued; at speed 4, there were only 119 of 223 (53.4%) viewings with at least one polyp pursuit. Polyp pursuits decreased in line with increased viewing speeds, falling from an average of 3.2 per viewing at speed 1 to 0.9 at speed 4 (P < .001; Table 1). Trends in time to first pursuit with respect to speed were highly case dependent (Fig 4).

Figure 4:

Time to first pursuit of polyp, expressed as a percentage of the period the polyp was onscreen, at each speed. Each point represents one viewing, and the lines connect the median at each speed. Separate figures are shown for each true positive case (P1–P8). Immediate pursuits (time 0%) are excluded. For case P5, there were no pursuits of the polyp at Speed 4.

Despite several polyps exhibiting a particularly pronounced reduction in eye pursuits with increasing viewing speed (eg, P1, P4, and P5; Fig 1), this was not necessarily associated with a corresponding reduction in polyp identifications. For example, for case P4, the polyp was identified by almost all readers at all speeds, even though fewer than 20% readers pursued this polyp at speed 4. Identification was presumed because of peripheral vision.

Among viewings for which at least one polyp pursuit was achieved, the average proportion of time spent looking directly at the polyp did not vary across speeds (mean between 41% and 45% at each speed; Table 1).

Discussion

Endoluminal CT colonography is generally displayed from the perspective of a virtual colonoscope, and most workstations allow the reader to fly through the lumen at their preferred speed (which we found was approximately 1.2 cm/sec). In our study, we found that as navigation speed increased, readers’ gaze narrowed progressively to the central portion of the display; “tunnel vision.” At the fastest speed, readers spent less than 5% of their time looking directly at the peripheral 75% of the image, thereby becoming almost entirely reliant on low-acuity peripheral vision here. A major advantage of CT colonography over colonoscopy is the ability of the reader to readily view the entire colonic mucosal surface (16); our data suggest this can be negated by excessively hasty image navigation, where the surface is displayed but not interrogated by the reader. Accordingly, the percentage of true-positive polyp identifications was reduced by over 10% at higher speeds and offset by a similar reduction in false-positive identifications. The effect of speed on polyp detection was highly dependent on the specific characteristics of the polyp being viewed.

Previous research investigating the effect of enforcing rapid radiologic image interpretation is relatively scant. One study asked radiologists to interpret static two-dimensional radiographs performed after extremity trauma (6). When reporting at twice their normal rate, observer sensitivity was unchanged, but the rate of false-positive findings dropped from 7.4% to 1.4%. A second study, which used chest radiographs with lung nodules, varied interpretation time between 0.25 seconds and unrestricted. There was no statistically significant difference in sensitivity or specificity versus unlimited time once interpretation of at least 4 seconds was permitted; sensitivity only dropped when viewing times fell to 1 second or less (7). More recently, the effect of rapid image interpretation on error rate for abdominopelvic CT was investigated (17). Radiologists were asked to interpret a sample of 12 CT studies at their normal rate and a further 12 cases at double this; the false-negative rate for major pathologic changes was 10.0% at normal speeds versus 26.6% at the faster speed. Specificity was not reported. Taken together, these studies suggest that faster interpretation leads radiologists to overlook both true- and false-positive findings, meaning that both sensitivity and the false-positive rate drop at higher interpretation speeds.

We were able to demonstrate the underlying cause of this change in diagnostic test characteristics for endoluminal CT colonography: the progressive narrowing of reader gaze with increased viewing speeds. The effect was particularly marked for speeds above 1.5 cm/sec, and a CT colonography workstation feature that displays navigation speed might therefore be a useful addition in clinical practice. However, at very slow speeds (eg, 1.0 cm/sec), we found false-positive identifications were relatively high, at 37%. These data present a dilemma for clinical practice; slow colonic navigation will likely maximize sensitivity, but at the cost of increased false-positive findings. Notwithstanding this, in most clinical scenarios (including diagnosis of colorectal polyps by using CT colonography) diagnostic sensitivity is prioritized by both patients and doctors (18), which implies that slower navigation speeds would be preferable. It is notable that one performance indicator for colonoscopists is the negative withdrawal time (19) (ie, the length of time an endoscopist spends inspecting the colon during examinations in which they did not find a polyp or cancer). This recognizes the positive correlation between long withdrawal times and high adenoma detection rates. There is a clear analogy to be drawn with CT colonography interpretation, and therefore further work may be warranted to determine if either overall interpretation time or endoluminal navigation speed are associated with polyp detection at CT colonography.

This study has limitations. The endoluminal CT colonography videos we used do not reflect real-world CT colonography interpretation directly, and therefore we cannot be certain that the effects we observed are present in clinical practice. We portrayed videos at prespecified fixed navigation speeds, whereas in reality readers will vary their speed as needed (for example, slowing down to navigate flexures). Also, we were only able to investigate a relatively limited number of different polyps and speeds, because piloting showed that a larger number provoked reader fatigue. Our power calculation and study results indicated that the sample size was sufficient to demonstrate differences in outcomes between speed settings, but our findings would have been stronger if it had been possible to use a larger number of different videos and speeds. Although we selected these video speeds to encompass a plausible range of speeds that might be used clinically, no reader selected a preferred navigation speed beyond our faster velocities; therefore, the higher speeds we used (speeds 3 and 4) were unlikely scenarios in clinical practice. We asked readers to click a mouse to identify a polyp, whereas in reality a potential polyp causes readers to stop and perform detailed inspection, including correlation with two-dimensional images, before a diagnostic decision is made. Although we have described such incorrect identifications as false-positive findings, this does not reflect the true false-positive rate for CT colonography because many would be dismissed after subsequent two-dimensional inspection. Nonetheless, productivity and throughput would be reduced because of the additional time required to interrogate the increased number of polyp candidates, and ultimately might translate to a higher referral rate for unnecessary colonoscopy. Similarly, the fact that our simulated reading mode is not identical to that used in clinical practice may have also affected true-positive polyp identifications.

In conclusion, as navigation speed increased, gaze of the reader became more central and polyp identification rates fell. Radiologists should consider reducing their navigation speed during interpretation of CT colonography to maximize sensitivity. Navigation speeds of over 1.5 cm/sec should be discouraged, and this information could be displayed by CT colonography workstations.

■ As fly-through speed increases beyond 1 cm/sec, reader gaze progressively narrows, reaching less than 3% time spent looking at the periphery of the image at higher speeds; “tunnel vision.”

■ Increasing fly-though speed is accompanied by a reduction in true-positive polyp identifications, from 72.7% (56 of 77) at 1 cm/sec to 60.9% (137 of 225) at 4.5 cm/sec (P = .004).

■ False-positive polyp identifications also fall at higher speeds (36.5% [42 of 115] at 1 cm/sec and 25.8% [89 of 345] at 4.5 cm/sec; P = .02).

■ Radiologists should avoid pressures to interpret large numbers of CT colonography examinations rapidly because this is likely to provoke incomplete luminal views and reduce polyp detection.

■ CT colonography workstations may benefit from software to monitor and display endoluminal navigation speed.