KID Project: an internet-based digital video atlas of capsule endoscopy for research purposes.

BACKGROUND AND AIMS: Capsule endoscopy (CE) has revolutionized small-bowel (SB) investigation. Computational methods can enhance diagnostic yield (DY); however, incorporating machine learning algorithms (MLAs) into CE reading is difficult as large amounts of image annotations are required for training. Current databases lack graphic annotations of pathologies and cannot be used. A novel database, KID, aims to provide a reference for research and development of medical decision support systems (MDSS) for CE.
METHODS: Open-source software was used for the KID database. Clinicians contribute anonymized, annotated CE images and videos. Graphic annotations are supported by an open-access annotation tool (Ratsnake). We detail an experiment based on the KID database, examining differences in SB lesion measurement between human readers and a MLA. The Jaccard Index (JI) was used to evaluate similarity between annotations by the MLA and human readers.
RESULTS: The MLA performed best in measuring lymphangiectasias with a JI of 81 ± 6 %. The other lesion types were: angioectasias (JI 64 ± 11 %), aphthae (JI 64 ± 8 %), chylous cysts (JI 70 ± 14 %), polypoid lesions (JI 75 ± 21 %), and ulcers (JI 56 ± 9 %).
CONCLUSION: MLA can perform as well as human readers in the measurement of SB angioectasias in white light (WL). Automated lesion measurement is therefore feasible. KID is currently the only open-source CE database developed specifically to aid development of MDSS. Our experiment demonstrates this potential.

Introduction

Capsule endoscopy (CE) has changed the field of small-bowel (SB) investigation 1 with the potential to become a panenteric diagnostic tool 2 . Computational methods incorporated into CE reading software can enhance diagnostic yield (DY) 3 . Several information technology (IT) groups have proposed software for detection of SB lesions/bleeding, reducing reading time, lesion localization, motility assessment, video enhancement and/or data management 1 3 . Reducing reading time is beneficial, especially in high volume centers. Previous work has shown that readers’ experience does not improve detection of lesions in CE 4 . Therefore, computer-aided detection/diagnosis (CAD) can improve DY.

Despite prolific IT research, incorporating artificial intelligence (AI) systems into CE reading remains difficult 3 . The backbone of AI system development is based on machine learning algorithms (MLAs) for automatic detection, localization, and recognition of pathology in CE images and videos. A large amount of data, in the form of annotations, is required to train MLAs. Semantic annotations describe the content of CE videos and images, whereas graphic annotations are pixel-level labels indicating regions of interest (ROIs) (  Fig.1 ). Although there are some online databases 5 , these usually include the necessary semantic annotations, but lack graphic annotations of ROIs. Therefore, such material cannot be directly used by IT scientists for intelligent systems’ training or as a reference for their evaluation.

Fig. 1

 Dataset of angioectasia images and their corresponding graphic annotations, seen within the KID website interface.

A limited number of datasets composed of images with graphic annotations have become available in the context of IT studies 3 6 . A novel database, KID (κάψουλα interactive database; based on Greek for “capsule”) ( http://is-innovation.eu/kid/ ) was developed to fill this gap. It is available online, upon free registration, aiming to provide a reference for research on the development of medical decision support systems (MDSS) for CE, including the study of the performance of human observers in comparison to others and CAD.

Methods

Database

Open-source database (Oracle MySQL; https://www.mysql.com/ ) and web-gallery development software (Coppermine; http://coppermine-gallery.net/ ) were used. Software tools for video manipulation and image annotation were added to the KID website. To date, six centers (the KID working group) have contributed anonymized, annotated CE images/videos from various CE models; more than 2500 annotated CE images and 47 videos have been uploaded. These include images of (a) normal CE; (b) vascular lesions including angioectasias and/or bleeding; (c) inflammatory lesions, including mucosal aphthae and ulcers, erythema, cobblestoning, and luminal stenosis; (d) lymphangiectasias; and (e) polypoid lesions ( Fig. 2 ).

Fig. 2

 Top row, from left: P1 and P2 angioectasias, aphthae and ulcer, with corresponding graphic annotations made using Ratsnake beneath each image, showing the position, size and shape of the lesions in the images. Bottom row, from left: two images of nodular lymphangiectasias and two images of polypoid lesions, with graphic annotations below each image.

Image and video standards

Lesion categorization is based on the CE Structured Terminology (CEST) 7 . Contributions are of high quality (original resolution), not distorted by additional compression. For images, the recommended standard is ISO/IEC 15948 PNG (Portable Network Graphics), a popular platform-independent format with lossless compression. Other acceptable standards include: ISO/IEC, 14496-10, MPEG-4, AVC (Advanced Video Coding) and H.264. Supported formats for videos include F4V & FLV (Flash video).

Image annotation

The usefulness of KID relies on image annotations. Semantic and graphic annotations are supported by an open access, platform-independent annotation tool (Ratsnake) 8 . The graphic annotation process is shown in Fig. 3 and Video 1 . Semantic annotation is done through textual labels, and using standard web ontology language description logics (OWL DL) 9 . The quality of data and annotations submitted to KID are scrutinized by an international scientific committee ( http://is-innovation.eu/kid/committee.php ); contributions not meeting the aforementioned standards are rejected.

Fig. 3

 Use of the Ratsnake annotation tool to perform graphical annotation of an angioectasia on capsule endoscopy (CE). Left: original image. Right: graphic annotation of the angioectasia.

An experiment using the KID database: Computer-aided lesion size measurements based on color image segmentation

A total of 64 images of gastrointestinal lesions taken with MiroCam ® (IntroMedic Co., Seoul, Korea) were used. The lesions were: angioectasias (n = 27), lymphangiectasias (n = 9), ulcers (n = 9), chylous cysts (n = 8), polypoid lesions (n = 6), and small-bowel aphthae (n = 5). Graphic annotations made by expert readers (AK, ER, ET; > 2000 CE readings each) were used as lesion surface size reference standards. The images were automatically segmented into two regions: a ROI, i. e. the lesion in question, and the rest of the image. This was performed using the Localized Region-based Active Contour (LRAC) 10 algorithm, which is capable of segmenting regions characterized by heterogeneity in grayscale images; see Fig. 4 for a stepwise graphic presentation. The reader initializes the LRAC by defining a circular contour roughly on or around the lesion, starting at a random point in the image. The lesion did not need to be fully included in the initial contour. The algorithm calculates contours based on intensity histogram information (i. e. information on image brightness and intensity) from the regions inside and outside the contour. The calculations are performed locally, around each point along the contour. The algorithm continues to run until the overall similarity of the histograms inside and outside the contour is minimized. In this experiment, we extended the algorithm to the three components of the Commission internationale de l’éclairage-Lab (CIE-Lab) color space representation (instead of the standard RGB) 11 . Components of this space represent lightness ( L ), which is approximately equivalent to the respective grayscale image, quantity of red ( a  > 0) or quantity of green (–  a  > 0), quantity of yellow ( b  > 0) or quantity of blue (–  b  > 0) of a pixel ( Fig. 5 ). Fig. 6 shows the results of image segmentation using this algorithm applied to the a component of CIE-Lab, compared to in RGB. The Jaccard Index (JI) 12 was used to assess the similarity of the ROI obtained with the aid of LRAC compared to the graphically annotated ROI obtained by the expert readers (gold standard) per image, i. e. the agreement between the expert human readers and the algorithm. The JI is considered to be the most suitable and popular measure for the assessment of image segmentation algorithms 12 . It quantifies the overlap between two ROIs as the ratio of their intersection to their union with respect to the human readers. Therefore, it is independent from the measurement unit, e. g. pixels 2 or mm 2 , used to quantify the measured area. An illustrative example is provided in Fig. 7 .

Fig. 4

 Segmentation of image using the Localized Region-based Active Contour (LRAC) algorithm. a User-defined initial contour. b Contour deformation/morphing based on local histogram information on brightness and intensity in the various circular neighborhoods at each point on the contour. c Segmented image obtained.

Fig. 5

 CIE-Lab color wheel (left) compared to the RGB color wheel (right).

Fig. 6

 Image segmentation by Localized Region-based Active Contour (LRAC) algorithm. Top row, from left: original image of mucosal break with surrounding erythema; image segmentation using the a component of CIE-Lab; the final result of image segmentation where the contours have been defined and marked. Bottom row: the image when broken down into red (R), green (G) and blue (B) channels under the traditional RGB system.

Fig. 7 

Agreement between a human reader and the algorithm as quantified by the Jaccard Index (JI). Given a region annotated by a human expert (left) and a region annotated by the algorithm (right), the intersection of the two regions corresponds to the True Positive (TP) pixels, i. e. those actually belonging to the abnormality. The union of the two regions corresponds to the sum of the False Negative (FN), the False Positive (FP) and the TP. Thus, if the two regions perfectly coincide, FN = 0, FP = 0 and their intersection (TP) becomes equal to their union, resulting in JI = 100 %. If there is no match between the two regions, then TP = 0 and JI = 0.

Results

The algorithm was evaluated for the measurement of six different types of small-bowel lesions, for each channel of CIE-Lab color space. The lesion areas were measured in pixel units, which, in the context of CE, is a more feasible and accurate approach. The average surface measurements closest to those performed by expert human readers were obtained by application of LRAC on the red-green scale of the CIE-Lab color space, with a JI of 67 ± 13 %. This result complements the findings in our previous study, indicating component a as an informative source of saliency for automated lesion detection 11 . The agreement between human readers and the algorithm per lesion type is summarized in   Table 1 . The most accurate measurements were obtained for lymphangiectasias, whereas this algorithm is less suitable for the measurement of ulcers.

Agreement between reviewers and software in measuring lesion size for various types of lesion seen on capsule endoscopy (CE).

Lesion	JI, mean ± SD, %
Angioectasias	64 ± 11
Aphthae	64 ± 8
Chylous cysts	70 ± 14
Lymphangiectasias	81 ± 6
Polypoid lesions	75 ± 21
Ulcers	56 ± 9

Abbreviations: JI, Jaccard Index; SD, standard deviation.

Discussion

Human factors remain a barrier to timely and accurate CE diagnosis 4 . AI systems can improve clinical performance, patient safety, and resource utilization 1 3 . Open interdisciplinary exchange of information is key to technological advancement and therefore improved clinical outcomes 3 . New technological developments may not always meet pertinent healthcare needs due to little communication between software engineers and clinicians; furthermore, open access databases of endoscopic images are scarce, especially those specifically related to small-bowel CE 5 . This is despite growing clinical demand and use of CE as an investigative modality. However, such interactive formats are vital for engaging a new generation of clinicians; this is currently hindered by inadequately developed software 13 . Therefore, KID aims to be a comprehensive and all-encompassing resource for continuous development of CAD in CE, and to encourage two-way dialog between technological developers and end-users. For example, KID compiles images from all commercial CE models and is international, thus increasing its scope.

The experiment detailed above shows that generally good agreement was achieved between expert human readers and the MLA in measuring the size of common small-bowel lesions. This implies automated lesion measurement is feasible, and MLAs could eventually replace or drastically reduce the workload of valuable human resources. In a recent study, van der Sommen et al. 14 detailed collaboration between IT engineers and clinicians to develop a CAD algorithm for diagnosis of early neoplasia in Barrett’s esophagus, with good results. An advantage of the method presented in this study over previous automated measurement approaches is its suitability for a variety of lesion types. In a recent study 15 using images of angiectasias available in KID, we showed that the interobserver agreement between CE reviewers, in terms of JI, in lesion annotation ranges between 65 ± 15 % and 67 ± 13 %, and the respective intraobserver agreement, between 69 ± 17 % and 71 ± 13 %. This dataset was similar in terms of the morphological characteristics of the displayed angiectasias, indicating that our MLA has a performance comparable to that of human readers. However, a limitation shown by the experiment is that it does not perform as well with all mucosal lesions. Further algorithm development is therefore required, showing the need for platforms such as KID.

In conclusion, KID is, to our knowledge, the only database of CE images and videos with both graphic and semantic annotations, developed specifically for MDSS research. It provides a platform for data sharing and CAD software development. The experiments detailed are proof-of-principle studies demonstrating the potential for KID to fulfill this role.