| Literature DB >> 35743748 |
Eun Jeong Gong1,2,3, Chang Seok Bang1,2,3,4, Jae Jun Lee3,4,5, Seung In Seo1,2, Young Joo Yang1,2, Gwang Ho Baik1,2, Jong Wook Kim6.
Abstract
BACKGROUND: The authors previously developed deep-learning models for the prediction of colorectal polyp histology (advanced colorectal cancer, early cancer/high-grade dysplasia, tubular adenoma with or without low-grade dysplasia, or non-neoplasm) from endoscopic images. While the model achieved 67.3% internal-test accuracy and 79.2% external-test accuracy, model development was labour-intensive and required specialised programming expertise. Moreover, the 240-image external-test dataset included only three advanced and eight early cancers, so it was difficult to generalise model performance. These limitations may be mitigated by deep-learning models developed using no-code platforms.Entities:
Keywords: colonic neoplasms; colonoscopy; convolutional neural network; deep learning; endoscopy; no code; polyps
Year: 2022 PMID: 35743748 PMCID: PMC9225479 DOI: 10.3390/jpm12060963
Source DB: PubMed Journal: J Pers Med ISSN: 2075-4426
Distribution of histological classes within datasets used for the establishment and testing of no-code tool-based deep-learning models.
| Whole Dataset | Training Dataset for No-Code Tools 1 and 3 | Internal-Test Dataset for No-Code Tools 1 and 3 | Training Dataset for No-Code Tool 2 | Internal-Test Dataset for No-Code Tool 2 | External-Test Dataset 1 | External-Test Dataset 2 | External-Test Dataset 3 | External-Test Dataset 4 | |
|---|---|---|---|---|---|---|---|---|---|
| Overall | 3828 | 3444 | 384 | 3638 | 190 | 575 | 752 | 603 | 1888 |
| Advanced colorectal cancer | 810 | 729 | 81 | 760 | 50 | 184 | 53 | 65 | 328 |
| Early colorectal cancer/high-grade dysplasia | 806 | 725 | 81 | 768 | 38 | 79 | 212 | 178 | 776 |
| Tubular adenoma with or without low-grade dysplasia | 1316 | 1184 | 132 | 1254 | 62 | 144 | 254 | 232 | 512 |
| Non-neoplasm | 896 | 806 | 90 | 856 | 40 | 168 | 233 | 128 | 272 |
No-code deep-learning tool 1: Neuro-T; tool 2: Create-ML image classifier; tool 3: Vision Learning for Advanced Detection OX. External-test dataset 1 was collected from Chuncheon Sacred Heart Hospital, dataset 2 was from Kangdong Sacred Heart Hospital, dataset 3 was from Inje University Ilsan Paik Hospital, and dataset 4 was from Gangneung Asan Hospital.
Figure 1Representative lesions in each histological category used for deep-learning model construction.
Summary of internal-test performance metrics.
| Accuracy (%) | Precision (%) | Recall (%) | F1 Score (%) | AUC (%) | |
|---|---|---|---|---|---|
| Model established by no-code deep-learning establishment tool 1 | |||||
| Internal test ( | 75.3 (71.0–79.6) | 77.9 (73.8–82.0) | 78.1 (74.0–82.2) | 78.0 (73.9–82.1) | |
| Per class performance for advanced colorectal cancers | 97.3 (93.6–99.9) | 88.9 (82.1–95.7) | 92.6 (90.7–94.5) | ||
| Per class performance for early colorectal cancers/high-grade dysplasias | 75.6 (66.5–84.7) | 80.2 (71.5–88.9) | 83.6 (80.9–86.3) | ||
| Per class performance for tubular adenomas | 78.5 (70.1–86.9) | 55.3 (46.8–63.8) | 74.0 (71.5–76.5) | ||
| Per class performance for non-neoplasms | 56.8 (48.3–65.3) | 87.8 (81.0–94.6) | 77.2 (74.3–80.1) | ||
| Model established by no-code deep-learning establishment tool 2 | |||||
| Internal test ( | 66.8 (60.1–73.5) | 70.0 (63.5–76.5) | 63.5 (56.7–70.3) | 66.6 (59.9–73.3) | |
| Per class performance for advanced colorectal cancers | 87.0 (77.7–96.3) | 80.0 (68.9–91.1) | |||
| Per class performance for early colorectal cancers/high-grade dysplasias | 73.1 (59.0–87.2) | 50.0 (34.1–65.9) | |||
| Per class performance for tubular adenomas | 55.9 (43.5–68.3) | 83.9 (74.7–93.1) | |||
| Per class performance for non-neoplasms | 64.0 (52.1–75.9) | 40.0 (27.8–52.2) | |||
| Model established by no-code deep-learning establishment tool 3 | |||||
| Internal test ( | 64.6 (59.8–69.4) | 68.2 (63.5–72.9) | 63.0 (58.2–67.8) | 65.5 (60.7–70.3) | |
| Per class performance for advanced colorectal cancers | 88.9 (82.1–95.7) | 88.9 (82.1–95.7) | |||
| Per class performance for early colorectal cancers/high-grade dysplasias | 69.6 (58.7–80.5) | 59.3 (48.6–70.0) | |||
| Per class performance for tubular adenomas | 53.7 (46.8–60.6) | 81.8 (75.2–88.4) | |||
| Per class performance for non-neoplasms | 60.6 (43.9–77.3) | 22.2 (13.6–30.8) |
No-code deep-learning establishment tool 1: Neuro-T; tool 2: Create-ML image classifier; tool 3: Vision Learning for Advanced Detection OX. Values with 95% confidence intervals are described.
Figure 2Confusion matrix for the no-code tool-1-based deep-learning model with the best performance.
Summary of external-test performance metrics.
| Accuracy (%) | Precision (%) | Recall (%) | F1 Score (%) | |
|---|---|---|---|---|
| Model established by no-code deep-learning establishment tool 1 | ||||
| External test 1 ( | 80.2 (76.9–83.5) | 78.5 (75.1–81.9) | 78.8 (75.5–82.1) | 78.6 (75.3–81.9) |
| External test 2 ( | 73.0 (69.8–76.2) | 76.4 (73.4–79.4) | 74.2 (71.1–77.3) | 75.3 (72.2–78.4) |
| External test 3 ( | 75.1 (71.6–78.6) | 75.3 (71.9–78.7) | 78.8 (75.5–82.1) | 77.0 (73.6–80.4) |
| External test 4 ( | 76.2 (74.3–78.1) | 74.5 (72.5–76.5) | 78.9 (77.1–80.7) | 76.7 (74.8–78.6) |
| Model established by no-code deep-learning establishment tool 2 | ||||
| External test 1 ( | 72.7 (70.8–74.6) | 76.5 (73.0–80.0) | 66.0 (62.1–69.9) | 70.9 (67.2–74.6) |
| External test 2 ( | 63.8 (60.4–67.2) | 66.4 (63.0–69.8) | 69.8 (66.5–73.1) | 68.0 (64.7–71.3) |
| External test 3 ( | 57.0 (53.0–61.0) | 59.0 (55.1–62.9) | 62.0 (58.1–65.9) | 60.5 (56.6–64.4) |
| External test 4 ( | 49.9 (47.6–52.2) | 57.8 (43.5–68.3) | 57.0 (55.6–60.0) | 57.4 (55.2–59.6) |
| Model established by no-code deep-learning establishment tool 3 | ||||
| External test 1 ( | 73.6 (70.0–77.2) | 74.1 (70.5–77.7) | 72.4 (68.7–76.1) | 73.2 (69.6–76.8) |
| External test 2 ( | 68.2 (64.9–71.5) | 71.3 (68.1–74.5) | 71.3 (68.1–74.5) | 71.3 (68.1–74.5) |
| External test 3 ( | 68.2 (64.5–71.9) | 69.1 (65.4–72.8) | 69.6 (65.9–73.3) | 69.3 (65.6–73.0) |
| External test 4 ( | 65.3 (63.2–67.4) | 64.7 (62.5–66.9) | 81.8 (75.2–88.4) | 68.3 (66.2–70.4) |
No-code deep-learning establishment tool 1: Neuro-T; tool 2: Create-ML image classifier; tool 3: Vision Learning for Advanced Detection OX. External-test dataset 1: from Chuncheon Sacred Heart hospital; 2: from Kangdong Sacred Heart hospital; 3: from Inje University Ilsan Paik Hospital; 4: from Gangneung Asan Hospital. Values with 95% confidence intervals are described.
Figure 3Representative cases of correctly determined classes in the external-test datasets using no-code tool 1. Left: gradient-weighted class activation mapping image. Right: white-light endoscopic image.
Figure 4Representative cases of incorrectly determined classes in the external-test datasets using no-code tool 1. Left: gradient-weighted class activation mapping image. Right: white-light imaging endoscopic image.
Potential reasons for incorrect classification of external-test dataset 2 images by the established no-code tool-based deep-learning models.
| Unknown (Difficult Cases Even for Endoscopists) | Multiple Attention or Partial Attention Even Though the Image Was Appropriate | Normal Mucosal Folds or Blood Vessels Recognised as Lesions | Inappropriate Images (Only a Part of the Lesion Can Be Observed) | Inappropriate Images (Multiple Lesions Were Observed in One Image) | Inappropriate Images (Residual Food or a Bubble Was Recognised as a Lesion) | |
|---|---|---|---|---|---|---|
| Advanced colorectal cancers | ||||||
| Incorrectly diagnosed as early colorectal cancers/high-grade dysplasias ( | 4 | 5 | 1 | |||
| Incorrectly diagnosed as non-neoplasm ( | 1 | |||||
| Early colorectal cancers/high-grade dysplasias | ||||||
| Incorrectly diagnosed as tubular adenoma ( | 47 | 9 | ||||
| Incorrectly diagnosed as non-neoplasm ( | 1 | 14 | ||||
| Incorrectly diagnosed as advanced colorectal cancers ( | 3 | 4 | ||||
| Tubular adenomas | ||||||
| Incorrectly diagnosed as non-neoplasm ( | 27 | 35 | 6 | 2 | ||
| Incorrectly diagnosed as early colorectal cancers/high-grade dysplasias ( | 12 | 5 | 1 | 1 | 1 | |
| Non-neoplasms | ||||||
| Incorrectly diagnosed as tubular adenoma ( | 3 | 20 | 1 | |||
| Total | 94 (46.3%) | 76 (37.4%) | 27 (13.3%) | 1 (0.5%) | 3 (1.5%) | 2 (1%) |
External-test dataset 2: from Kangdong Sacred Heart Hospital.