Improvement in adenoma detection using a novel artificial intelligence-aided polyp detection device.

Background and study aims  Detecting colorectal neoplasia is the goal of high-quality screening and surveillance colonoscopy, as reflected by high adenoma detection rate (ADR) and adenomas per colonoscopy (APC). The aim of our study was to evaluate the performance of a novel artificial intelligence (AI)-aided polyp detection device, Skout, with the primary endpoints of ADR and APC in routine colonoscopy. Patients and methods  We compared ADR and APC in a cohort of outpatients undergoing routine high-resolution colonoscopy with and without the use of a real-time, AI-aided polyp detection device. Patients undergoing colonoscopy with Skout were enrolled in a single-arm, unblinded, prospective trial and the results were compared with a historical cohort. All resected polyps were examined histologically. Results  Eighty-three patients undergoing screening and surveillance colonoscopy at an outpatient endoscopy center were enrolled and outcomes compared with 283 historical control patients. Overall, ADR with and without Skout was 54.2 % and 40.6 % respectively ( P  = 0.028) and 53.6 % and 30.8 %, respectively, in screening exams ( P  = 0.024). Overall, APC rate with and without Skout was 1.46 and 1.01, respectively, ( P  = 0.104) and 1.18 and 0.50, respectively, in screening exams ( P  = 0.002). Overall, true histology rate (THR) with and without Skout was 73.8 % and 78.4 %, respectively, ( P  = 0.463) and 75.0 % and 71.0 %, respectively, in screening exams ( P  = 0.731). Conclusion  We have demonstrated that our novel AI-aided polyp detection device increased the ADR in a cohort of patients undergoing screening and surveillance colonoscopy without a significant concomitant increase in hyperplastic polyp resection. AI-aided colonoscopy has the potential for improving the outcomes of patients undergoing colonoscopy. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commecial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/).

Introduction

Colonoscopy has been shown to reduce the incidence of colorectal cancer through detection and removal of colorectal neoplasia 1 2 . Adenoma detection rate (ADR) is the recognized quality indicator for colonoscopy and has been associated with the risk of post-colonoscopy interval colorectal cancer 3 4 5 . However, there is substantial variability in ADR among endoscopists 6 . ADR alone may be insufficient to identify the quality of a colonoscopy, since an endoscopist may be less inclined to identify and remove polyps once a single adenoma has been detected 7 . Therefore, adenomas per colonoscopy (APC) has been proposed as an additional quality indicator to reduce this bias 7 . A recently published meta-analysis of 43 publications with 15,000 colonoscopies found that both ADR and APC were independently associated with the adenoma miss rate 8 . In addition, recent studies have highlighted the importance of sessile serrated lesions (SSLs) as precursors to colon cancer and the importance of improving their detection during colonoscopy 9 10 .

Improving ADR, APC, and detection of SSLs is important for colonoscopy quality programs, with the goal of reducing the disease burden of colorectal cancer. In recent years, various applications of artificial intelligence (AI) have been explored and several promising AI-aided polyp detection devices have been developed. These devices have been shown to assist endoscopists in the detection of polyps, and in doing so, improved ADR and APC 11 12 . Because AI-aided devices may inadvertently increase the detection and removal of histologically unimportant tissue (hyperplastic polyps or normal tissue), it is important to study the change in the detection of adenomas and SSLs in relation to all polyps resected, termed as the true histology rate (THR), as part of a broader analysis of downside risk to the use of an AI device 13 .

Our primary aim in this study was to evaluate the pilot performance and usability of a novel AI-aided polyp detection device in improving quality indicators such as ADR, APC, and detection rate of SSLs, assessing these outcomes with an eye on impact to THR during routine colonoscopies in a real-world setting.

Patients and methods

Study population and data description

Previous research on and development of Skout have included assessment of the AI-aided device in a non-clinical setting. This testing has included validated machine learning (ML) benchmarks, human factors evaluation, and performance assessment through retrospective review of endoscopic procedural videos. (ML is a subset of AI that studies computer algorithms that improve automatically through experience.) However, translation of the device into a real-world clinical setting had yet to be explored. This was the first clinical pilot of Skout.

We prospectively collected information on colonoscopies performed with the aid of Skout by three providers in one ambulatory endoscopy center in New Hampshire over a 6-week period, from February 2020 to March 2020. Participating providers had an average ADR of 32.6 %, 10 to 15 years of experience, and performed an average of 500 colonoscopies per year with a median withdrawal time of 7 minutes and 20 seconds.

We included 83 patients undergoing screening and surveillance colonoscopies in the all-comers study population. Patients with a history of inflammatory bowel disease or familial adenomatous polyposis were excluded from participation. Bowel preparation was reported by the endoscopist performing the procedure and was described as adequate for all individuals in the study. The PCF-Hi190/1 colonoscope was used for 66 of the procedures and the CF-HQ190/1 colonoscope was used for 17 of the procedures in combination with the Olympus CV-190 image processor, which was used for all procedures.

Withdrawal time (WT) was recorded, with WT being defined as time from reaching the cecum to removal of the colonoscope from anal verge, including time taken for polypectomy during withdrawal, if applicable. In line with the standard of care, endoscopists were encouraged to have a minimum WT of 6 to 8 minutes. Information around each polyp resected was collected, including size, location, and histological characteristics.

We compared the results to colonoscopies performed by the same three endoscopists at the same practice location 6 months prior to the study period (historical controls). The study was approved by an institutional review board and patient consent was received prior to the device being used. For a definition of terms, please see Table 1 .

Definition of terms.

Adenoma was defined as any polyp with tubular, villous, or tubulovillous histology as well as polyps with dysplasia.

Adenoma Detection Rate (ADR) was defined as the number of colonoscopies in which one or more adenomas were detected, divided by the total number of colonoscopies.

Adenoma Per Colonoscopy (APC) was defined as the number of detected adenomas divided by the total number of colonoscopies.

Sessile Serrated Lesion (SSL) was defined as any polyp with histology or serrated adenoma, traditional serrated adenoma or serrated lesion with cytological dysplasia. Hyperplastic polyps were excluded.

Sessile Serrated Lesion Detection Rate (SSLDR) was defined as the number of colonoscopies in which one or more SSLs were detected, divided by the total number of colonoscopies.

Sessile Serrated Lesion Per Colonoscopy (SSLPC) was defined as the number of detected sessile serrated lesions divided by the total number of colonoscopies.

True Histology Rate (THR) was defined as the number of polyps that were adenomatous or sessile serrated lesions divided by the total number of polyps resected.

Withdrawal Time – Total (WT) defined as the time spent withdrawing from the cecum including time for polypectomy or other interventions.

Withdrawal Time – Corrected (Corrected-WT) defined as the time spent withdrawing from the cecum for inspection only (excluding polypectomy or other interventions).

Intervention

Skout is an AI-aided device that uses a deep neural network to identify potential polyps in real-time. Skout was trained on prospectively acquired full-length colonoscopy procedure videos. Annotations were performed by expert medical image annotators and reviewed by experienced endoscopists. No imputation was made for missing input data. This is the first version of Skout deployed in a clinical setting, and there were no updates made to the device throughout the duration of this study.

Our algorithm detects various events including but not limited to the use of narrow band-imaging, the insertion of biopsy forceps and hot or cold snares, and the presence of polyps. The output is a graphical user interface featuring a bounding box over the region that most likely contains a polyp. Examples of the user interface can be seen in Fig. 1 . Skout’s specificity and sensitivity for polyp detection were determined at a frame level as 99.1 % and 74 %​​, respectively. The current standard video output for most colonoscopes is 60 frames per second. It was made clear to performing endoscopists that the decision to resect or biopsy a polyp remained at their clinical discretion. This reflects how we would expect an endoscopist to interact with Skout in a real-world setting.

Fig. 1

 Polyp detection with Skout, indicated by the blue bounding box around the perimeter of the lesion. a Detection of a 10-mm tubular adenoma in the sigmoid colon. b Detection of a 3-mm sessile serrated lesion in the rectum. c , d Detection of a 5-mm sessile serrated lesion in the ascending colon, taken upon first detection and upon closer look, respectively.

During the development of Skout, Iterative Scopes held numerous informal conversations with gastroenterologists, nurses, and technicians to gain an understanding of user behavior and preferences. Iterative Scopes held five rounds of formal usability studies with seventeen gastroenterologists, four endoscopy technicians, and one nurse. Feedback from these sessions led to the development of several key features including: integrating the software in one monitor versus multiple monitors; the design of the polyp localization signal (bounding box); stabilization and smoothing improvements of the bounding box signal; automated resizing of the bounding box signal depending on the size of the polyp shown on screen; and adding a margin between the bounding box signal and the detected boundary of the polyp to avoid obstructing the field of view.

In addition, feedback was collected during the study using two mechanisms: a Skout AI-aided Colonoscopy Tool (ACT) Evaluation Form and informal observation notes.

Statistical analysis

We compared ADR, APC, sessile serrated lesion detection rate (SSLDR), sessile serrated lesion per colonoscopy (SSLPC), THR, and WT for colonoscopies from historical controls to colonoscopies with use of Skout. Patient characteristics were described as percentages for categorical variables or means with standard deviations (SD) for continuous variables. We assumed the historical control and Skout to be two independent samples and normally distributed. Means were compared using a Student’s t -test. For the comparison of proportions, we used the “N-1” χ 2 test as recommended by Campbell 14 . All statistical tests were two-sided with P  < 0.05 regarded as statistically significant without Bonferroni correction. Usability questions were open ended and qualitative only.

Results

There were 283 historical control colonoscopies and 83 Skout colonoscopies performed during the study period (28 screening, 55 surveillance). The average age of the enrolled subjects during the study period was 62.2 years and 51.8 % were male. Demographics are summarized in Table 2 .

Demographics of patients undergoing Skout colonoscopy.

	N	Male	Female	Mean age
All	83	43	40	62.2 (10.2 SD)
Screening	28	14	14	59.6 (10.8 SD)
Surveillance	55	29	26	63.5 (9.7 SD)

We found that Skout had an impact on all exams; however, the impact was most significant on screening exam performance. Overall, ADR increased with Skout from 40.6 % to 54.2 % (33.5 % relative increase; P  = 0.028), and in screening exams from 30.7 % to 53.6 % (74.0 % relative increase; P  = 0.024). Overall, the APC rate increased with Skout from 1.01 to 1.46 (44.6 % relative increase; P  = 0.104) and in screening exams from 0.50 to 1.18 (136.0 % relative increase; P  = 0.002). Overall, SSLDR increased with Skout from 8.8 % to 12.0 % (36.4 % relative increase; P  > 0.05) and in screening exams from 6.8 % to 14.3 % (110.0 % relative increase; P  > 0.05). There was a significant increase in the SSLPC for screening exams from 0.09 to 0.43 (378.0 % relative increase; P  = 0.034). There was no statistically or clinically meaningful change in THR overall or for screening exams (78.4 % vs 73.8 % overall and 71.0 % vs 75.0 % for screening exams). The overall corrected WT increased from 7.9 minutes to 9.3 minutes (17.7 % relative change; P  = 0.039) and for screening exams from 7.7 minutes to 9.1 minutes (18.2 % relative change; P  = 0.100). The results are summarized in Table 3 .

Comparison of colonoscopy quality markers by all exams, screening, and surveillance colonoscopy.

	Historical control colonoscopy	Skout colonoscopy	Relative % change	P 1 value
All exams (n = 283 Historical Control, 83 Skout™)
APC	1.01	1.46	44.6	0.104
ADR %	40.6	54.2	33.5	0.028
SSLPC	0.14	0.24	71.4	0.178
SSLDR %	8.8	12.0	36.4	0.383
THR %	78.4	73.8	–5.9	0.463
Corrected-WT minutes	7.9	9.3	17.7	0.039
Total WT minutes	9.2	11.5	25.0	< 0.001
Screening exams (n = 117 Historical control, 28 Skout)
APC	0.50	1.18	136.0	0.002
ADR %	30.8	53.6	74.0	0.024
SSLPC	0.09	0.43	378.0	0.034
SSLDR %	6.8	14.3	110.0	0.197
THR %	71.0	75.0	5.6	0.731
Corrected-WT minutes	7.7	9.1	18.2	0.100
Total WT minutes	8.9	10.9	22.3	< 0.001
Surveillance exams (n = 166 Historical control; 55 Skout)
APC	1.37	1.60	16.8	0.582
ADR %	47.6	54.5	14.5	0.376
SSLPC	0.17	0.15	–11.7	0.793
SSLDR %	10.2	10.9	6.9	0.883
THR %	82.6	73.2	–11.4	0.201
Corrected-WT minutes	8.1	9.3	14.8	0.241
Total WT minutes	9.5	11.8	24.2	< 0.001

APC, adenomas per colonoscopy; ADR, adenoma detection rate; SSLPC, sessile serrated lesion per colonoscopy; SSLDR, sessile serrated lesion detection rate; THR, true histology rate; WT, withdrawal time.

Considered statistically significant at the 0.05 level

The size and histological distribution of polyps is presented in Fig. 2 . In the Skout colonoscopies, we observed an increase in ADR, APC, SSLDR, and SSLPC in both the proximal and distal colon. This finding was consistent in polyps of all histologies. The results are summarized in Table 4 .

Fig. 2 

Distribution of polyps by size and histology in Hhstorical control vs Skout colonoscopy.

Location and histology of polyps.

		Historical control colonoscopy(n = 283)			Skout colonoscopy(n = 83)
Histology	Location	Total number	Percent of total %	Number per patient	Total number	Percent of total %	Number per patient
Adenomas	Proximal Colon	238	82.9	0.81	92	77.3	1.11
	Distal Colon	49	17.1	0.17	27	22.7	0.33
SSLs	Proximal Colon	35	85.4	0.12	8	40.0	0.10
	Distal Colon	6	14.6	0.02	12	60.0	0.14
Hyperplastic polyps	Proximal Colon	29	34.5	0.10	17	34.7	0.20
	Distal Colon	55	65.5	0.19	32	65.3	0.39
Total polyps	Proximal Colon	307	72.6	1.09	118	62.1	1.42
	Distal Colon	116	27.4	0.41	72	37.9	0.87

SSL, sessile serrated lesion.

Usability

In this study, three gastroenterologists without prior Skout experience each completed a minimum of 15 colonoscopies with Skout prior to completing a Skout ACT Evaluation Form designed to assess their experience using the device. All of the gastroenterologists felt comfortable using Skout safely and effectively. The ease of use of Skout was reported by two endoscopists to be extremely easy and by one endoscopist as easy. None found any factors that would result in a negative impact to procedural efficiency.

Further qualitative feedback from providers revealed that Skout provided “an additional pair of eyes,” and that using Skout was “like having someone looking with you, not for you.” Providers expressed that they would rather have Skout present slightly more false positives than miss a true positive and that the type of events on which Skout was presenting false positives, such as mucosal bunching caused by suctioning as a common example, were reasonably discounted by providers and did not cause real concern. According to one provider, Skout “makes you take a closer look and [the signal] quickly goes away once it realizes it isn’t a polyp.”

Technical device adverse events were reported in four of 83 Skout procedures. The Skout system includes a video display switch that allows the endoscopist to switch between a standard feed of the endoscopic monitor and an enhanced video with the Skout display overlaid. As each of the technical complications arose, the provider or technician switched the display to standard feed and the procedure was not affected.

Discussion

In this study, we used two primary endpoints, ADR and APC, to detect differences in the results of screening colonoscopies when using a novel AI-aided polyp detection device, Skout. In contrast to previous studies, we performed this study in a largely White population with well-characterized high risk of adenomas 15 . The physicians performing the colonoscopy were experienced and had a high average ADR of 32.6 %. Previously conducted prospective randomized trials using an AI-aided device for polyp detection were performed in an Asian population with ADRs of 20 %, 28 %, and 8 % in the control groups 11 12 16 . In our population, we found that the overall ADR increased significantly from 40.6 % to 54.2 % and in screening exams, the ADR increased significantly from 30.8 % to 53.6 %. Our results are consistent with the Asian studies in that an AI-aided device increases the ADR of endoscopists. These results also demonstrate that our AI-aided device is effective in a population with a relatively high prevalence of adenomas. While past studies have shown an increase of 6 % to 9 %, the use of our device yielded an absolute increase of 22.8 % 11 12 16 .

Our study analyzed all removed polyps and tabulated the histologic subtypes of adenomas, SSLs, and hyperplastic polyps. Given the concern that AI technologies may increase the detection of non-neoplastic polyps, we examined THR. With the use of our device, we found an overall relative decrease of 5.9 % in THR in our study. In our population, we found that the overall THR decreased from 78.4 % to 73.8 % ( P  > 0.05) and in screening exams, the THR increased from 71.0 % to 75.0 % ( P  > 0.05). The change seen each of these cohorts was not statistically significant. This suggests that the use of Skout does not lead to an increase in biopsies or resections of hyperplastic polyps in a meaningful way.

A key consideration in the development of AI-aided devices is utility in the detection of SSLs. The detection of SSLs is of particular importance as they have been found to be associated with a heightened risk of development of advanced neoplasia. Prior studies have demonstrated the ability of AI-aided devices to detect SSLs but with inconclusive efficacy 12 16 17 . In our study, we found that overall SSLDR increased from 8.8 % to 12 % ( P  > 0.05), and overall SSLPC rate increased from 0.14 to 0.24 ( P  > 0.05). Finally, we observed a significant increase in SSLPC rate in screening exams from 0.09 to 0.42 ( P  = 0.03). While none of the SSLs eventually yielded a histology of advanced neoplasia, it is worth noting that two patients did have adenomas with high-grade dysplasia. These early results suggest that Skout is highly effective in detecting SSLs.

An examination of the sizes of adenomas detected by Skout found that 97 % of all adenomas were less than 10 mm. For adenomas, 50.4 % were in the range of 0 to 4 mm and 44.6 % were in the range of 5 to 9 mm. For SSLs, all lesions were found to be less than 10 mm, with 70 % being in the range of 0 to 4 mm and 30 % in the range of 5 to 9 mm. These data demonstrate the ability of Skout to effectively detect not only pedunculated lesions, but also flat SSLs. Furthermore, these findings suggest that increase in detection is not limited to diminutive lesions, but larger and clinically meaningful ones as well.

To our knowledge, this is the first study to report qualitative user feedback on the use of an AI-aided device during standard screening colonoscopies. The device and user experience described herein was informative for understanding the usability of Skout. The feedback was largely positive and indicated enthusiasm for widespread adoption by practicing endoscopists. Our AI method relied on a deep convolutional neural network. Our design carefully balanced accuracy and computational requirements to yield state-of-the-art results in real time, i. e., processing every image frame on the fly as they are ingested, with no perceptible lag. While slightly higher accuracy could potentially be obtained using a more complex ML model running without time constraints, the clinical utility of such a model would be limited as findings detected retrospectively would require follow-up colonoscopies with their corresponding difficulty in implementation in real clinical practice.

There are limitations of this study. The size of the patient population was relatively small and thus, the study may not detect all differences with using the Skout device. In addition, the study design was a non-randomized trial with an historical cohort. This may lead to bias by the endoscopists as they were aware of Skout use. However, this study accurately reflects real practice experience in the United States using a large, well-established, previously described outpatient colonoscopy practice. It is important to note that most screening and surveillance colonoscopies occur in ambulatory centers such as our site.

The endpoints used in this study, ADR and APC, are often used as surrogate measures of colonoscopy quality. An inverse linear relationship has been demonstrated between endoscopist ADR and the interval colon cancer rate 3 . Furthermore, for each 1 % increase in ADR, there was an associated 3 % decrease in the risk of colon cancer. One could hypothesize that significant increases in ADR with AI-aided colonoscopy could lead to proportional decreases in the rates of interval colon cancer. For example, in a recent quality improvement program, increases in ADR were associated with an observed 50 % decrease in interval colon cancer rate 18 . However long-term studies will be needed to assess the impact of AI-aided colonoscopy and its impact on interval colon cancer.

Video 1  Skout identifying a 4-mm sessile serrated lesion in the transverse colon.

Conclusion

In summary, we have developed and demonstrated that AI-aided polyp detection device using Skout increased ADR in a cohort of patients undergoing screening and surveillance colonoscopy. SSL detection was increased in screening colonoscopy, addressing a key clinical area of concern that has previously proven difficult to improve with AI-aided interventions. The improvement in ADR and SSLDR took place without significant increases in resection of hyperplastic polyps, hence indicating minimal false positives for non-neoplastic lesions with no associated increase in risk or cost for participating patients or payers, respectively. Should these results be shown to be replicable in larger studies involving a range of endoscopists, we believe that AI-aided colonoscopies will become the standard of care, particularly as we strive to achieving better outcomes in the prevention of colorectal cancer in our patients.