| Literature DB >> 33282741 |
Shuo Wang1, Jiajun Xu2, Aylin Tahmasebi1, Kelly Daniels3,4, Ji-Bin Liu1, Joseph Curry3, Elizabeth Cottrill3, Andrej Lyshchik1, John R Eisenbrey1.
Abstract
BACKGROUND: The role of next generation sequencing (NGS) for identifying high risk mutations in thyroid nodules following fine needle aspiration (FNA) biopsy continues to grow. However, ultrasound diagnosis even using the American College of Radiology's Thyroid Imaging Reporting and Data System (TI-RADS) has limited ability to stratify genetic risk. The purpose of this study was to incorporate an artificial intelligence (AI) algorithm of thyroid ultrasound with object detection within the TI-RADS scoring system to improve prediction of genetic risk in these nodules.Entities:
Keywords: Thyroid Imaging Reporting and Data System classification; machine learning; next generation sequencing; object detection; thyroid nodules
Year: 2020 PMID: 33282741 PMCID: PMC7689011 DOI: 10.3389/fonc.2020.591846
Source DB: PubMed Journal: Front Oncol ISSN: 2234-943X Impact factor: 6.244
High risk genes on NGS used as a reference standard.
| Gene | Human Genome Region |
|---|---|
| AKT1 | aa 17-18 |
| APC | aa 178-291 and 312-1594 with splice sites |
| AXIN1 | aa 1-688 and 731-865 with splice sites |
| BRAF | aa 594-606, 439-478 |
| CDKN2A | Full with splice sites |
| CTNNB1 | aa 6-60 |
| DNMT3A | aa 881-883 |
| EGFR | Exons 18,19,20,21 |
| EIF1AX | aa 1-6, 35-86, and 115-147 |
| GNAS | aa 201-203 and 226-227 |
| HRAS | aa 10-14 and 60-62 and 146 |
| IDH1 | aa 67-71, 123-134 |
| KRAS | aa 10-14 and 60-62 and 146 |
| NDUFA13 | Full with splice sites |
| NRAS | aa 10-14 and 60-62 and 146 |
| PIK3CA | aa 520-554 and 980-1069 |
| PTEN | Full with splice sites |
| RET | aa 883, 918, 588-636 |
| SMAD4 | aa 36-552 with splice sites |
| TERT | promoter chr5: 1295228 and 1295250 |
| TP53 | aa 26-393 with splice sites |
| TSHR | Full with splice sites |
| VHL | Full with splice sites |
*aa denotes amino acid residue numbers.
Summary of training and prediction dataset composition.
| Dataset/# of images | Low-risk | High-risk | Total |
|---|---|---|---|
| Training dataset | 488 | 228 | 716 |
| Prediction dataset | 25 | 26 | 51 |
Figure 1Example of a de-identified image being input into the AutoML object detection model. In this example, radiologists used bounding boxes to mark the location of the lesion (Yellow bounding box) and overall image (Green bounding box). The label was assigned with bounding boxes where the yellow bounding box indicated a high-risk lesion and the green bounding box indicated an ultrasound image that contained a high-risk lesion.
Figure 2(A) The model performance report generated by the platform. The model had an AUC of 0.889. (B) The model split the 716 training images into new training (644 images) and new testing (72) datasets. At a confidence score level of 0.44, the model had 68.31% precision and 86.81% recall for the new testing (72 images) dataset.
Figure 3(A) Example prediction from the Object Detection Model that correctly detected a nodule and correctly assigned a high-risk label with 98% certainty. The position of the high-risk nodule was marked by the orange color bounding box drawn by the model. (B) Example prediction from the Object Detection Model that correctly detected the nodule and correctly designated the lesion as low risk with 100% confidence. The position of the low-risk nodule was marked by the yellow bounding box drawn by the model. (C) Example prediction from the Object Detection Model that detected a high-risk lesion, a high-risk area, and a low-risk area with confidence scores of 0.96, 0.76, and 0.56, respectively. The position of the high-risk nodule was marked with the orange bounding box drawn by the model. (D) Example prediction from the Object Detection Model that detected the nodule but assigned both high-risk and low-risk labels. The model provided a confidence score of 0.74 for the nodule to be high risk and a confidence score of 0.68 for the nodule to be low risk. The model also indicated two low risk areas with a confidence score of 0.93 and 0.8.
Confusion matrix for the prediction images with AI diagnosis.
| High-Risk | Low-Risk | |
|---|---|---|
| Prediction High-Risk | 17 | 7 |
| Prediction Low-Risk | 6 | 17 |
Sensitivity, specificity, PPV, and NPV, and accuracy from the AI alone, TI-RADS alone, and AI + TI-RADS.
| Sensitivity (95% CI) | Specificity (95% CI) | PPV (95% CI) | NPV (95% CI) | Accuracy (95% CI) | |
|---|---|---|---|---|---|
| Object Detection Model | 73.9% | 70.8% | 70.8% | 73.9% | 72.4% |
| Radiologist Alone | 52.1 ± 4.4% | 65.2 ± 6.4% | 59.1 ± 3.5% | 58.7 ± 1.8% | 58.8 ± 2.5% |
| Radiologist Post-AI | 53.6 ± 17.6% | 83.3 ± 7.2% | 75.7 ± 8.5% | 66.0 ± 8.8% | 68.7 ± 7.4% |
| p = 0.87 | p = 0.06 | p = 0.13 | p = 0.31 | p = 0.21 |
95% Confidence interval: 95% CI.
Figure 4Radiologist performance using TI-RADS criteria alone (white) and TI-RADS with AI assistance (checkered) for predicting the risk of thyroid nodules on B-mode ultrasound.