Application of artificial intelligence in gastrointestinal disease: a narrative review.

OBJECTIVE: We collected evidence on the application of artificial intelligence (AI) in gastroenterology field. The review was carried out from two aspects of endoscopic types and gastrointestinal diseases, and briefly summarized the challenges and future directions in this field.
BACKGROUND: Due to the advancement of computational power and a surge of available data, a solid foundation has been laid for the growth of AI. Specifically, varied machine learning (ML) techniques have been emerging in endoscopic image analysis. To improve the accuracy and efficiency of clinicians, AI has been widely applied to gastrointestinal endoscopy.
METHODS: PubMed electronic database was searched using the keywords containing "AI", "ML", "deep learning (DL)", "convolution neural network", "endoscopy (such as white light endoscopy (WLE), narrow band imaging (NBI) endoscopy, magnifying endoscopy with narrow band imaging (ME-NBI), chromoendoscopy, endocytoscopy (EC), and capsule endoscopy (CE))". Search results were assessed for relevance and then used for detailed discussion.
CONCLUSIONS: This review described the basic knowledge of AI, ML, and DL, and summarizes the application of AI in various endoscopes and gastrointestinal diseases. Finally, the challenges and directions of AI in clinical application were discussed. At present, the application of AI has solved some clinical problems, but more still needs to be done. 2021 Annals of Translational Medicine. All rights reserved.

Introduction

In the 1950s, the concept of artificial intelligence (AI) was first proposed at the Dartmouth Conference, with the aim to create complex machines that simulate cognitive traits of the working human brain (1). Namely refers to using artificial methods and technologies to imitate, extend and expand human intelligence, to achieve some “machine thinking”. With 70 years of effort, AI has come to be widely used in many fields, such as health care, finance, education, and others. It has made certain operations more convenient and rational, especially in the medical industry.

In gastroenterological services, reviewing a large number of endoscopic images will lead to physicians’ overwork and indirectly affect the accuracy of diagnosis and the efficiency of decision making. To offload tedious work but target more comprehensive tasks, the need for AI-assisted tools in clinical practice is on the rise. Researchers have developed AI methods to segment lesions of interest in endoscopic images automatically. These are of value for the diagnosis, treatment, and prognosis of gastrointestinal diseases. At present, the application of AI in gastrointestinal diseases is still in the early stage, and the acquisition, cleaning and standardization of data are huge problems that limit the development of AI. Moreover, whether AI can be quickly applied to gastrointestinal diseases depends on the performance of intelligent system in clinical application, and also depends on the understanding and acceptance of AI by clinical medical staff.

In this review, we introduce the classification of AI techniques, and AI are reviewed from two aspects in the application of gastroenterology, one is the application of AI in the different types of endoscopes, the second is the application of AI in various gastrointestinal diseases. Finally, we discuss the challenges and future developmental direction of AI applications in gastrointestinal diseases.

We present the following article in accordance with the Narrative Review reporting checklist (available at https://dx.doi.org/10.21037/atm-21-3001).

Methods

We searched the PubMed electronic database for English literature published between 2000 to 2020. The search keywords containing “AI”, “machine learning (ML)”, “deep learning (DL)”, “convolutional neural network (CNN)”, “endoscopy”, “white light endoscopy (WLE)”, “narrow band imaging (NBI) endoscopy”, “magnifying endoscopy with narrow band imaging (ME-NBI)”, “chromoendoscopy”, “endocytoscopy (EC)”, and “capsule endoscopy (CE)”. The search results were manually reviewed to confirm studies involving AI applications in the gastrointestinal field.

AI

With the improvement of computers and the contributions from other disciplines, the field of AI has advanced remarkably, recently emerging as its own field. ML, one of the core topics in AI, was first proposed in the 1980s as a way to implement AI. Through continuous exploration and improvement, a new subbranch DL has grown from ML. DL has a more complex feature extraction process than ML.

ML

Over the last 40 years, ML has developed into a multidisciplinary and interdisciplinary field of study, involving statistics, probability theory, and other disciplines. ML is a type of automatic analysis that learns from data. Using multiple iterations, it continuously improves on the gaps in the existing knowledge system to improve the performance of the task at hand. According to learning methods, ML can be roughly divided into three types: supervised learning, unsupervised learning, and reinforcement learning. Supervised learning uses labeled data to train algorithms, unsupervised learning uses unlabeled data to discover new patterns, and reinforcement learning uses continuous self-optimization through the autonomous learning of the machine to gradually complete the target task. Unlike supervised learning and unsupervised learning, reinforcement learning does not require any data to be given in advance, and there is a balance between exploration and exploitation (2).

Various ML algorithms, including decision trees, support vector machines, and regression, have been used in medical research. A decision tree is a flowchart-like structure that is usually built to aid in decision making. Based on the decision tree algorithm, a preventive measure guide was developed, and has been proven considerably valuable in the protection and safety of health care workers (3). The support vector machine algorithm is adept at binary classification. Mori et al. built a computer-aided system (CAD) for real-time identification of diminutive polyps through the support vector machine algorithm. It could identify diminutive polyps as either tumor polyps or non-tumor polyps (4). Regression is generally used to identify the state relationship between variables, which has been advantageous for constructing a prediction model of preoperative lymph node metastasis of colon cancer (5).

DL

DL outperforms previous conventional ML in big data fitting due to its automatic data-driven operation, which contrasts specific preprocessing procedures. In addition, the basic ideas and technologies of DL used in different fields are easy to convert and amenable to later application. However, for a small volume of data, traditional ML has a higher capacity to achieve excellent performance. DL works based on neural networks with an algorithmic architecture of multiple hidden layers, each of which further refines the conclusions of the previous layer (6). Neural networks are typically trained using supervised or unsupervised learning methods, whereas a CNN uses the former and a generative adversarial network uses the latter.

Types of gastrointestinal endoscopy

AI-based endoscopy image analysis is one of the most promising applications in the medical field. An endoscope is an illuminated optical instrument used to examine the inner structures of the human body through natural orifices or surgical incisions and can determine the necessity of local biopsy or treatment. It mainly consists of a light source, a lens, and a pipe. Because of its minimal invasiveness, endoscopy has become an important diagnostic tool for early gastrointestinal neoplasms. There are six types of commonly used endoscopes (): WLE, NBI endoscopy, ME-NBI, chromoendoscopy, EC, and CE.

Figure 1

Endoscopy images often used to develop artificial intelligence models. WLE, white light endoscopy; NBI, narrow band imaging; ME-NBI, magnifying endoscopy with narrow band imaging; EC, endocytoscopy; CE, capsule endoscopy. Reprinted, with permission, from (7).

WLE

WLE is the preferred endoscopic technique of screening for gastrointestinal diseases due to its low cost and rapidity of examination. However, it suffers from limited sensitivity to small precursor lesions.

Bossuyt et al. collected WLE images of 35 participants with ulcerative colitis and healthy controls to develop an AI system with a red density algorithm to reflect disease activity (8). This method automatically constructed a red density map of endoscopic images by extracting values of red-green-blue pixels through the red channel. It measured disease activity with the final disease activity score, which was closely related to the histological remission score (8).

Invasion depth is one of the important risk factors for lymph node metastasis of gastrointestinal tumors and affects therapy selection. A retrospective study, by Cho et al. established a supervised CNN model (combine Inception-ResNet-v2 and the DenseNet161 models) to categorize gastric neoplasms into a binary class using invasion depth (mucosa-confined versus submucosa-invaded), with the area under the curve (AUC) of 0.887 in both internal and external tests (9).

NBI endoscopy

Where WLE uses white light, NBI endoscopy reduces the range of the visible light spectrum through a wavelength filter, which retains the blue (415 nm) and green (540 nm) light only. As the kept wavelengths match with the hemoglobin absorption spectrum, NBI endoscopy enhances the clarity of microvascular morphology and mucosal surface structures. This assists in the diagnosis of the mucosal surface lesions, better defining the scope and boundaries of lesions (10,11).

Mori et al. prospectively developed a CAD-NBI model using ML algorithms to detect diminutive polyps and predict related pathologies (neoplastic polyps and nonneoplastic polyps) (4). The negative predictive values for the diminutive rectosigmoid adenomas in the worst and best cases were 95.20% and 96.50%, respectively. In terms of better performance, the CAD-NBI model proved more time efficient than those based on chromoendoscopy. The excellent performance of this model benefited from the observation scope of NBI for microstructures and capillaries of the mucosal epithelium, which is also a step towards realizing the automatic detection of pathology during endoscopy.

Adenomas is the precursors of most colorectal malignancies. Endoscopic resection of adenomas, contributes to the reduction of the incidence and mortality of colorectal cancer (12). Therefore, the detection and classification of polyps is crucial for treatment and prognosis. A recent study reported that diagnosis of NBI by DNN-CAD model was satisfactory (13). The authors analyzed 2,441 images and achieved an accuracy of 90.10%, a sensitivity of 96.30%, and a specificity of 78.10% in identifying neoplastic or proliferative polyps less than 5 mm in size.

ME-NBI

ME-NBI is a hybrid technique combining NBI and magnifying endoscopy, which enables one to observe the various details of the mucosal capillaries. However, there is still an appreciable rate of missed diagnoses.

One of the endoscopic characteristics of early squamous cell tumors is the presence of intrapapillary capillary loops, which is related to invasion depth (14). A supervised CNN system was developed to classify intrapapillary capillary loops into either normal or abnormal patterns by training 7,046 ME-NBI images of 17 patients, yielding an accuracy of 93.30% (15). Another CNN system with a GoogLeNet algorithm using 2,828 ME-NBI images was used to identify early gastric cancer and gastritis (16).

Chromoendoscopy

Chromoendoscopy introduces pigment dye into the mucosa under endoscopy to enhance color contrast between lesions and normal mucosa. The positive screening rate of chromoendoscopy is significantly higher than that of conventional endoscopy. In particular, some flat and concave lesions that are easily missed in conventional endoscopy (17).

To automatically detect the gastric cancer, Hirasawa et al. trained a CNN model with 13,584 images of gastric cancer and validated it in an independent testing set (2,296 stomach images) (18), yielding a sensitivity of 92.20% in diagnosing gastric cancer. Ikenoyama et al. compared the performance of a CNN model with that of endoscopists in detecting gastric cancer (19). The detection speed and performance of the CNN model proved superior to those of endoscopists.

EC

EC is a type of optical microscopic endoscopy, which can rapidly magnify objects 100 to 1,000 times. Combined with in vivo staining agents which increase cell contrast of the mucosa, the cell structure of the superficial cross-section of the digestive tract mucosa are observed in real time. EC is beneficial for diagnosis of the nature of lesions, improving its accuracy, and reducing the number of biopsies.

To distinguish between nonmalignant lesions and esophageal squamous cell carcinoma, Kumagai et al. mapped an AI model based on a GoogLeNet algorithm using 6,235 EC images (20) and achieved 90.90% accuracy, 92.60% sensitivity, and 89.30% specificity. However, EC images with optical magnification of ×400 and ×500 times were used in this study, which might have reduced the diagnostic performance of the AI model.

CE

CE involves a small capsule mainly consisting of a video camera, flash lamp, radio transmitter, and a battery. As the capsule endoscope is swallowed into the stomach and transported by gastrointestinal motility, the condition of the digestive tract is recorded. CE allows one to directly view the inner surface of the bowels if intestinal preparation is effective.

An AI model based on the Single Shot Multibox Detector algorithm was developed to detect small-bowel angioectasia using 12,725 CE images (21). This model had an AUC of 0.998, a sensitivity of 98.80%, a specificity of 98.40%, a positive predictive value of 75.40%, and a negative-positive value of 99.90%. CE images have also been used for the automatic identification of colon cancers and polyps with a CNN algorithm (22,23).

Application of AI in gastrointestinal diseases

According to common sites of gastrointestinal diseases, AI applications in gastroenterological endoscopy relate to three aspects: upper gastrointestinal diseases, small intestinal diseases, and large intestinal diseases.

Upper gastrointestinal diseases

AI applications in endoscopy of upper digestive tract diseases are shown in and include detection of esophageal and gastric cancer, prediction of the invasion depth of cancer, distinction of cancers from other diseases, and detection of Helicobacter pylori infection.

Table 1

Application of artificial intelligence in upper gastrointestinal diseases

Ref.	Study aim	Study type	Diagnostic modality	AI classifier	Training data set	Test data set	AI performance (Acc/Sen/Spe)	Physician performance (Acc/Sen/Spe)
Cho et al. (9), 2020	Identify the depth of mucosal invasion of gastric cancer	Retrospective	WLI	DenseNet161 + Inception-ResNet-v2	2,590 images	Data set A: 309 images; Data set B: 206 images	77.30/80.40/80.70	–
Everson et al. (15), 2019	Classification of ESCN on the basis of capillary loop in the nipple	Retrospective	ME-NBI	CNN	7,046 images	–	93.30/89.70/96.90	–
Horiuchi et al. (16), 2020	Distinguish gastric cancer from gastritis	Retrospective	ME-NBI	GoogLeNet	2,570 images	258 images	85.30/95.40/71	–
Hirasawa et al. (18), 2018	Diagnosis of gastric cancer	Retrospective	WLI, NBI, and chromoendoscopy	SSD	13,584 images	2,296 images	NA/92.20/NA	–
Ikenoyama et al. (19), 2020	Comparison of the ability of CNN system and physicians in detecting gastric cancer	Retrospective	WLI	SSD	13,584 images	2,940 images	NA/58.40/87.30	NA/31.90/97.20
Kumagai et al. (20), 2019	Diagnosis of ESCC	Retrospective	EC	GoogLeNet	4,715 images	1,520 images	90.90/92.60/89.30	100/89.30/90
Guo et al. (24), 2020	Diagnosis of early esophageal cancer	Retrospective	NBI	SegNet	6,473 images	Data set A: 59 patients, Data set B: 2004 patients, Data set C: 47 videos, Data set D: 33 videos	NA/98.04/95.03	–
Nakagawa et al. (25), 2019	Assessment of depth of invasion in superficial ESCC	Retrospective	NBI, WLI and chromoendoscopy	SSD + VGG	14,338 images	914 images	91/90.10/95.80	89.60/89.80/88.30
Tokai et al. (26), 2020	Identify the depth of mucosal invasion of ESCC	Retrospective	WLI and NBI	SSD and GoogLeNet	8,428 images	293 images	80.90/84.10/73.30	73.50/78.80/61.70
Zhao et al. (27), 2019	Detection of early ESCC	Retrospective	NBI and ME-NBI	VGG16	219 cases	–	89.20/87/84.10	Junior: 73.30/67.70/76.40
Ueyama et al. (28), 2020	Diagnosis of EGC	Retrospective	ME-NBI	ResNet50	4,460 images	Data set A: 1,114 images; Data set B: 2,300 images	98.70/98/100	–
Horiuchi et al. (29), 2020	Diagnosis of EGC	Retrospective	ME-NBI, WLI and chromoendoscopy	GoogLeNet	2,570 images	174 videos	85.10/87.40/82.80	85.10/94.20/75.90
Sakai et al. (30), 2018	Diagnosis of EGC	Retrospective	WLI	GoogLeNet	19,387 images	9,650 images	87.60/80/94.80	–
Wu et al. (31), 2019	Diagnosis of EGC	Retrospective	endoscopy	DCNN	9,151 images	200 images	92.50/94/91	81.16/75.33/88.83
Yoon et al. (32), 2019	Diagnosis of EGC	Retrospective	WLI	VGG16 and Grad-CAM	11,539 images	660 images	NA/80.70/92.50	–
Zhu et al. (33), 2019	Detection of invasion depth of gastric cancer	Retrospective	endoscopy	ResNet50	790 images	203 images	89.16/76.47/95.56	71.49/87.80/63.31
Luo et al. (34), 2019	Detection of upper gastrointestinal cancers	Case-control	endoscopy	DeepLab's V3+	125,898 images	Data set A: 15,672 images, Data set B: 812,539 images, Data set C: 66,750 images; Data set D: 15,637 images	92.80/94.20/92.30	Junior: 88.60/72.20/94.50
Cho et al. (35), 2019	Classification of gastric neoplasms	Prospective	WLI	Inception-v4, ResNet152 and Inception-ResNet-v2	4,180 images	Dateset A: 812 images; prospective cohort: 200 images	93/60.70/98.30	99.50/96.40/100
Namikawa et al. (36), 2020	Discrimination gastric cancers from gastric ulcers	Retrospective	WLI and NBI	SSD	4,453 images	1,459 images	NA/99/93.30	–
Shichijo et al. (37), 2017	Detection of H. pylori infection	Retrospective	EGD	GoogLeNet	32,208 images	11,481 images	81.90/83.40/NA	82.40/79/83.20
Itoh et al. (38), 2018	Detection of H. pylori infection	Retrospective	endoscopy	GoogLeNet	149 images	30 images	NA/86.70/86.70	–
Nakashima et al. (39), 2018	Detection of H. pylori infection	Prospective	BLI, LCI and WLI	GoogLeNet	162 cases	60 cases	NA/66.70/60	–
Zheng et al. (40), 2019	Evaluation of H. pylori infection	Retrospective	WLI	ResNet50	1,507 images	452 images	84.50/81.40/90.10	–
Nakashima et al. (41), 2020	Evaluation of H. pylori infection	Prospective	WLI and LCI	DCNN	395 patients	120 patients	75.00/95.00/65.00	91.20/NA/NA
Shichijo et al. (42), 2019	Evaluation of H. pylori infection	Retrospective	EGD	GoogLeNet	98,564 images	23,699 images	80/NA/NA	–
Li et al. (43), 2018	Detection of nasopharyngeal cancer	Prospective	WLI	Fully convolutional network	19,275 images	9,691 images	88.70/91.30/83.10	Interns: 66.50/92.20/38.90
Ebigbo et al. (44), 2019	Diagnosis of early esophageal adenocarcinoma	Retrospective	HD-WLI and NBI	ResNet	148 images	–	NA/92/100	–
Iwagami et al. (45), 2020	Detection of early esophageal and esophagogastric junction adenocarcinoma	Retrospective	NBI, BLI, and WLI	SSD	3,443 images	232 images	66/94/42	63/88/43
Cai et al. (46), 2019	Diagnosis of esophageal cancer	Retrospective	WLI	DNN	2,428 images	187 images	91.40/97.80/85.40	Senior: 88.80/86.30/91.20
Guimarães et al. (47), 2020	Diagnosis of atrophic gastritis	Retrospective	WLI	VGG16	200 images	70 images	92.90/100/87.50	80/80/80
Zhang et al. (48), 2020	Improvement of diagnostic rate of chronic atrophic gastritis	Retrospective	endoscopy	DenseNet121	5,470 images	–	94.20/94.50/94	–

Acc, accuracy; Sen, sensitivity; Spe, specificity; ESCN, early squamous cell neoplasia; EGC, early gastric cancer; ESCC, esophageal squamous cell carcinoma; Helicobacter pylori, H. pylori. CNN, convolutional neural network; DCNN, deep convolutional neural network;DNN, deep neural network; WLI, white light image; ME-NBI, magnifying endoscopy with narrow band imaging; NBI, narrow band imaging; EGD, esophagogastroduodenoscopy; BLI, blue laser imaging; LCI, linked color imaging; HD-WLI, high-definition white light endoscopy; SSD, single-shot multibox detector; EGD, esophagogastroduodenoscopy.

To enable early detection of esophageal squamous cell carcinoma, Guo et al. established a CAD system using SegNet architecture that was trained on 6,473 NBI images and validated with image and video data sets (24). The system showed an AUC of 0.989, a sensitivity of 98%, and a specificity of 95%. Comparatively, other CAD-based detection systems of esophageal squamous cell carcinoma had slightly inferior sensitivity due to the small data volume (15,20,25-27). Those studies carried out comparative experiments on the performance between physicians and intelligent systems, suggesting that the detection capacity of CAD systems can reach the level of a junior physician (26,27). In gastric cancer detection, a CNN model used 7,874 ME-NBI images from a single center for training and had an accuracy of 98.70%, a sensitivity of 98%, and a specificity of 100% (28). In a comparative study of CAD systems and physicians in detection of early gastric cancer, the CAD system with a GoogLeNet algorithm obtained an AUC of 0.868, an accuracy of 85.10%, a sensitivity of 87.40%, and a specificity of 82.80% (29). Sakai et al. used 29,037 images to detect early gastric cancer with an accuracy of 87.60% (30). Meanwhile, Wu et al. collected 9,151 images to train the deep CNN model for the detection of early gastric cancer, achieving an accuracy of 92.50% (31).

The invasion depth of cancer is crucial for selecting patients with gastric cancer for endoscopic resection. Many studies have detected the invasion depth of gastric cancer based on ML (9,18,25,26,32). Zhu et al. published a CNN-CAD system based on the ResNet50 algorithm to determine the invasion depth of gastric cancer. The AUC for the CNN-CAD system was 0.940, and the accuracy, sensitivity, and specificity were 89.16%, 76.47%, and 95.56%, respectively (33). The CNN-CAD system appears to be capable of outperforming endoscopists. Yoon et al. constructed a novel CNN diagnostic system based on the VGG16 algorithm, which had the highest performance (AUC =0.851) in determining the invasion depth of gastric cancer (32). Hirasawa et al. used the CNN system to identify the invasion depth and tumor size of gastric cancer (18). In addition, Luo et al. created the gastrointestinal AI diagnosis system (GRAIDS) based on DeepLab’s V3+ algorithm, a binary classification model for real-time detection of upper gastrointestinal tumors that was trained on 1,036,496 endoscopy images from six centers (34). The diagnostic accuracy of GRAIDS was 97.70% in the five external validation sets. Cho et al. established a five-category classification CNN model to identify neoplasm, early gastric cancer, low-grade dysplasia, high-grade dysplasia, and advanced gastric cancer (35). The CNN model was developed and validated using 5,017 WLE images based on the 5-fold-cross validation method. Two other aforementioned studies focused on distinguishing gastric cancer from gastritis (16) and gastric ulcers (36).

Helicobacter pylori infection is associated with the incidence of gastric cancer. Therefore, many studies have used ML algorithms to build models for the diagnosis of Helicobacter pylori infection, with the early models mostly using binary classification (37-40). A retrospective study used 179 images to create a model to detect Helicobacter pylori infection, which yielded an AUC of 0.956, a sensitivity of 86.70%, and a specificity of 86.70% (38). Other studies examined the ability of three-category methods to discriminate between uninfected, infected, and post-eradication (41,42).

Other diseases, including nasopharyngeal cancer (one study) (43), esophageal cancer (three studies) (44-46), and atrophic gastritis (two studies) (47,48) have been diagnosed using ML algorithms.

Small intestinal diseases

AI applications in small intestinal diseases are based on CE images or videos (). For ulcer detection, Klang et al. created a CNN model that could detect small-bowel ulcers in Crohn’s disease patients based on 17,640 images (49). The CNN model obtained an AUC of 0.990 in the randomly split images. To develop an easily transformable diagnostic model for ulcers, a retrospective study used 1,416 videos to develop and validate the model, which had favorable performance (AUC =0.973) (50). A CAD system was proposed to recognize polyps based on a stacked sparse autoencoder with the image manifold constraint method and yielded an accuracy of 98% (51). He et al. developed an AI system that could identify hookworm infection using 440K CE images (52,53); meanwhile, another study that used a CNN algorithm to detect angioectasia achieved a sensitivity of 100% and a specificity of 96% (54). AI has also been used for the detection of bleeding (55,56) and Crohn’s disease (49).

Table 2

Application of artificial intelligence in small intestinal diseases

Ref.	Study aim	Study type	Diagnostic modality	AI classifier	Training data set	Test data set	AI performance (Acc/Sen/Spe)
Tsuboi et al. (21), 2020	Detection of small intestinal blood vessels	Retrospective	CE	SSD	2,237 images	10,488 images	NA/98.80/98.40
Klang et al. (49), 2020	Detection of Crohn's disease ulcers	Retrospective	CE	Xception	17,640 images	–	96.40/97.10/96
Wang et al. (50), 2019	Detection of ulcers	Retrospective	CE	ResNet34	990 videos	Data set A: 141 videos; Data set B: 283 videos	92.05/91.64/92.42
Yuan et al. (51), 2017	Detection of ploys	Retrospective	CE	Softmax	–	4,000 images	98/NA/NA
He et al. (52), 2018	Detection of hookworm	Retrospective	CE	DHDF	440,000 images	–	88.50/84.60/88.60
Wu et al. (53), 2016	Detection of hookworm	Retrospective	CE	PPRD, UTR and HAI	440,000 images	–	78.20/77.20/77.90
Leenhardt et al. (54), 2019	Detection of blood content	Retrospective	CE	CNN	600 images	600 images	NA/100/96
Aoki et al. (55), 2020	Detection of blood content	Retrospective	CE	ResNet50	27,847 images	10,208 images	99.89/96.63/99.96
Xiao et al. (56), 2016	Detection of intestinal bleeding	Retrospective	CE	SVM	8,200 images	1,800 images	–

Acc, accuracy; Sen, sensitivity; Spe, specificity; SSD, single-shot multibox detector; DHDF, deep hookworm detection framework; PPRD, piecewise parallel region detection; UTR, uncurled tubular region; HAI, histogram of average intensity; CNN, convolutional neural network; SVM, support vector machine.

Large intestinal diseases

summarizes the studies that have leveraged AI to assist in the diagnosis of large intestinal diseases, most of which focus on polyp detection, and related to identification, localization, and segmentation. Three studies of polyp segmentation showed high accuracy (57-59), while among the four studies of polyp localization (23,60-62), there has been great heterogeneity concerning data between training and test sets, subsequently leading to the variable performance of these models. Nevertheless, the accuracy of most models has been greater than 85% (13,63-66). A randomized controlled study constructed a system to improve the detection rate of adenoma (67). Furthermore, Zhou et al. developed a DL model for diagnosing colorectal cancer based on colonoscopy images of 14,442 patients (68), achieving an AUC of 0.990, 0.991 and 0.997 in three test sets at the image level. Finally, AI has been used extensively to assess disease activity in ulcerative colitis (8,69,70).

Table 3

Application of artificial intelligence in large intestinal diseases

Ref.	Study aim	Study type	Diagnostic modality	AI classifier	Training data set	Test data set	AI performance (Acc/Sen/Spe)	Physician performance (Acc/Sen/Spe)
Mori et al. (4), 2018	Identification polyps smaller than 5 mm	Prospective	NBI and chromoendoscopy	SVM	325 cases	–	NA/93.30/70	NA/77.70/66.70
Bossuyt et al. (8), 2020	Identification UC disease activity	Prospective	WLI	Red density	35 cases	–	–	–
Chen et al. (13), 2018	Accurate classification of tiny polyps	Retrospective	NBI	CNN	2,157 images	284 images	90.10/96.30/78.10	84.20/93.60/65.60
Yamada et al. (22), 2020	Detection of colorectal neoplasms	Retrospective	CE	SSD	15,933 images	4,784 images	83.90/79/87	–
Blanes-Vidal et al. (23), 2019	Detection of colorectal polyps	Retrospective	CE	AlexNet, GoogLeNet, ResNet50, VGG16 and VGG19	7,910 images	1,695 images	96.40/97.10/93.30	–
Guo et al. (57), 2019	Automatic segmentation of polyps	Retrospective	Colonoscopy	Unet-VGG + PSPNet + SegNet-VGG	943 images	cvc300: 45 images; CVC-ClinicDB: 91 images; ETIS-LaribPolypDB: 29 images	98.04/NA/NA	–
Akbari et al. (58), 2018	Segmentation of polyps	Retrospective	Colonoscopy	FCN-8S	200 images	300 images	97.77/74.80/99.30	–
Bagheri et al. (59), 2019	Segmentation of polyps	Retrospective	Endoscopy	LinkNet	284 frames	71 frames	97.70/82.90/99.10	–
Urban et al. (60), 2018	Detection of polyps	Retrospective	WLI and NBI	VGG16, VGG19 and ResNet50	8,641 images	20 videos	96.40/96.90/NA	NA/93/93
Poon et al. (61), 2020	Detection of colon polyps	Retrospective	Colonoscopy	ResNet50, YOLOv2 and temporal tracking	119,703 images	34,469 images	92/72.60/93.30	–
Zheng et al. (62), 2018	Detection of colorectal ploys	Retrospective	WLI and NBI	YOLO	12,592 images	196 images	NA/71.60/NA	–
Byrne et al. (63), 2019	Distinguish adenomas from polyps	Retrospective	NBI	DCNN	223 videos	40 videos	94/98//83	–
Wang et al. (64), 2018	Detection of polyps	Retrospective	Colonoscopy	SegNet	5,545 images	Data set A: 27,113 images; CVC-ClinicDB: 29 videos	NA/94.38/95.92	–
Yu et al. (65), 2017	Automatic detection of polyps in colonoscopy video	Retrospective	Endoscopy	CNN	20 videos	18 videos	–	–
Billah et al. (66), 2017	Detection of polyps	Retrospective	Endoscopy	SVM	14,000 images	–	98.65/98.79/98.52	–
Gong et al. (67), 2020	Detection of colorectal adenomas	Randomizedcontrolled	WLI	VGG16	21,427 images	3,600 images + 84 videos	–	–
Zhou et al. (68), 2020	Detection of colorectal cancer	Retrospective	Colonoscopy	CRCNet	464,105 images	2,263 cases	87.30/NA/85.30	82.40/NA/91.20
Ozawa et al. (69), 2019	Assessment of endoscopic disease activity in patients with UC	Retrospective	WLI	GoogLeNet	26,304 images	3,981 images	–	–
Takenaka et al. (70), 2020	Prediction of histological remission in UC	Retrospective	Colonoscopy	DNN	40,758 images of colonoscopies and 6,885 biopsies from 2,012 patients with UC	4,187 endoscopic images from 875 patients with UC and 4,104 biopsy specimens	90.10/93.30/87.80

Acc, accuracy; Sen, sensitivity; Spe, specificity; UC, ulcerative colitis; WLI, white light image; NBI, narrow band imaging; CE, capsule endoscopy; SVM, support vector machine; SSD, single-shot multibox detector; DCNN, deep convolutional neural network; CNN, convolutional neural network; DNN, deep neural network.

Challenges and future directions

Some factors may limit the development of AI systems in the diagnosis of gastrointestinal diseases. Due to the small sample size of current studies, the current models are prone to overfitting. The number of amplified samples can alleviate this phenomenon. Also, it is crucial to validate the accuracy of model in multiple external data sets. Specifically, multicenter, diagnostic studies are needed, while video data are critical for expediting model verification by simulating the clinical settings (34). Moreover, the previous studies have been limited in disease diversity, which weakens the ability to generalize the findings of the research. The included training data should thus have greater fidelity to real application scenarios, so that the AI models could be made more suited to the clinical transformation. Training with offset data has a considerable impact on the generalization and application of the model. In addition, prospective studies are needed to compare the differences across AI systems, physicians, and physicians aided by AI, which may clarify the clinical application of AI systems. Currently, model development relies largely on manual preprocessing and labeling, which is extremely time-consuming and hinders technique advancement.

AI has been applied to most gastrointestinal diseases, but esophageal polyps, esophageal lipoma, gastric cyst, and a few other diseases remain conspicuous exceptions. In addition, due to the difficulty of long-term follow-up, there are relatively few AI studies that have focused on the prognosis of disease. From the current research, AI models are regularly based on one type of image. However, with the improvement of technology, it is possible to create a cross-platform AI system that overcomes differences in image quality, manufacturer, and color. This will reduce the training burden and platform construction cost.

Conclusions

This brief overview of the status of AI’s application in gastrointestinal diseases provides potential value to solving clinical problems and to further utilizing AI in the future. AI is widely used in endoscopy, including in procedures involving the upper gastrointestinal tract, large intestine, and small bowel, and has been able to resolving several issues of missed and challenging diagnoses in clinical settings. Although AI may offer benefit to patients in the process of diagnosis and treatment, its use increases the complexity of operation to a certain extent. Hence, medical staff should work and be patient with AI during the early stages of AI utilization.

The article’s supplementary files as