David Benjamin Ellebrecht1,2, Nicole Heßler3, Alexander Schlaefer4, Nils Gessert4. 1. Department of Thoracic Surgery, LungenClinic Großhansdorf, Großhansdorf, Germany. 2. Department of Surgery, Campus Lübeck, University Medical Centre Schleswig-Holstein, Lübeck, Germany. 3. Institute of Medical Biometry and Statistics, University of Lübeck, University Medical Center Schleswig-Holstein, Campus Lübeck, Lübeck, Germany. 4. Institute of Medical Technology and Intelligent Systems, Hamburg University of Technology, Hamburg, Germany.
Abstract
BACKGROUND: Confocal laser microscopy (CLM) is one of the optical techniques that are promising methods of intraoperative in vivo real-time tissue examination based on tissue fluorescence. However, surgeons might struggle interpreting CLM images intraoperatively due to different tissue characteristics of different tissue pathologies in clinical reality. Deep learning techniques enable fast and consistent image analysis and might support intraoperative image interpretation. The objective of this study was to analyze the diagnostic accuracy of newly trained observers in the evaluation of normal colon and peritoneal tissue and colon cancer and metastasis, respectively, and to compare it with that of convolutional neural networks (CNNs). METHODS: Two hundred representative CLM images of the normal and malignant colon and peritoneal tissue were evaluated by newly trained observers (surgeons and pathologists) and CNNs (VGG-16 and Densenet121), respectively, based on tissue dignity. The primary endpoint was the correct detection of the normal and cancer/metastasis tissue measured by sensitivity and specificity of both groups. Additionally, positive predictive values (PPVs) and negative predictive values (NPVs) were calculated for the newly trained observer group. The interobserver variability of dignity evaluation was calculated using kappa statistic. The F1-score and area under the curve (AUC) were used to evaluate the performance of image recognition of the CNNs' training scenarios. RESULTS: Sensitivity and specificity ranged between 0.55 and 1.0 (pathologists: 0.66-0.97; surgeons: 0.55-1.0) and between 0.65 and 0.96 (pathologists: 0.68-0.93; surgeons: 0.65-0.96), respectively. PPVs were 0.75 and 0.90 in the pathologists' group and 0.73-0.96 in the surgeons' group, respectively. NPVs were 0.73 and 0.96 for pathologists' and between 0.66 and 1.00 for surgeons' tissue analysis. The overall interobserver variability was 0.54. Depending on the training scenario, cancer/metastasis tissue was classified with an AUC of 0.77-0.88 by VGG-16 and 0.85-0.89 by Densenet121. Transfer learning improved performance over training from scratch. CONCLUSIONS: Newly trained investigators are able to learn CLM images features and interpretation rapidly, regardless of their clinical experience. Heterogeneity in tissue diagnosis and a moderate interobserver variability reflect the clinical reality more realistic. CNNs provide comparable diagnostic results as clinical observers and could improve surgeons' intraoperative tissue assessment.
BACKGROUND: Confocal laser microscopy (CLM) is one of the optical techniques that are promising methods of intraoperative in vivo real-time tissue examination based on tissue fluorescence. However, surgeons might struggle interpreting CLM images intraoperatively due to different tissue characteristics of different tissue pathologies in clinical reality. Deep learning techniques enable fast and consistent image analysis and might support intraoperative image interpretation. The objective of this study was to analyze the diagnostic accuracy of newly trained observers in the evaluation of normal colon and peritoneal tissue and colon cancer and metastasis, respectively, and to compare it with that of convolutional neural networks (CNNs). METHODS: Two hundred representative CLM images of the normal and malignant colon and peritoneal tissue were evaluated by newly trained observers (surgeons and pathologists) and CNNs (VGG-16 and Densenet121), respectively, based on tissue dignity. The primary endpoint was the correct detection of the normal and cancer/metastasis tissue measured by sensitivity and specificity of both groups. Additionally, positive predictive values (PPVs) and negative predictive values (NPVs) were calculated for the newly trained observer group. The interobserver variability of dignity evaluation was calculated using kappa statistic. The F1-score and area under the curve (AUC) were used to evaluate the performance of image recognition of the CNNs' training scenarios. RESULTS: Sensitivity and specificity ranged between 0.55 and 1.0 (pathologists: 0.66-0.97; surgeons: 0.55-1.0) and between 0.65 and 0.96 (pathologists: 0.68-0.93; surgeons: 0.65-0.96), respectively. PPVs were 0.75 and 0.90 in the pathologists' group and 0.73-0.96 in the surgeons' group, respectively. NPVs were 0.73 and 0.96 for pathologists' and between 0.66 and 1.00 for surgeons' tissue analysis. The overall interobserver variability was 0.54. Depending on the training scenario, cancer/metastasis tissue was classified with an AUC of 0.77-0.88 by VGG-16 and 0.85-0.89 by Densenet121. Transfer learning improved performance over training from scratch. CONCLUSIONS: Newly trained investigators are able to learn CLM images features and interpretation rapidly, regardless of their clinical experience. Heterogeneity in tissue diagnosis and a moderate interobserver variability reflect the clinical reality more realistic. CNNs provide comparable diagnostic results as clinical observers and could improve surgeons' intraoperative tissue assessment.
Authors: Andre Esteva; Brett Kuprel; Roberto A Novoa; Justin Ko; Susan M Swetter; Helen M Blau; Sebastian Thrun Journal: Nature Date: 2017-01-25 Impact factor: 49.962
Authors: David B Ellebrecht; Maximilian P E Gebhard; Marco Horn; Tobias Keck; Markus Kleemann Journal: Surg Innov Date: 2016-03-13 Impact factor: 2.058
Authors: M Goetz; R Kiesslich; H-P Dienes; U Drebber; E Murr; A Hoffman; S Kanzler; P R Galle; P Delaney; M F Neurath Journal: Endoscopy Date: 2008-07 Impact factor: 10.093
Authors: Gilles R Dagenais; Darryl P Leong; Sumathy Rangarajan; Fernando Lanas; Patricio Lopez-Jaramillo; Rajeev Gupta; Rafael Diaz; Alvaro Avezum; Gustavo B F Oliveira; Andreas Wielgosz; Shameena R Parambath; Prem Mony; Khalid F Alhabib; Ahmet Temizhan; Noorhassim Ismail; Jephat Chifamba; Karen Yeates; Rasha Khatib; Omar Rahman; Katarzyna Zatonska; Khawar Kazmi; Li Wei; Jun Zhu; Annika Rosengren; K Vijayakumar; Manmeet Kaur; Viswanathan Mohan; AfzalHussein Yusufali; Roya Kelishadi; Koon K Teo; Philip Joseph; Salim Yusuf Journal: Lancet Date: 2019-09-03 Impact factor: 79.321
Authors: David Benjamin Ellebrecht; Christiane Kuempers; Marco Horn; Tobias Keck; Markus Kleemann Journal: Surg Endosc Date: 2018-09-21 Impact factor: 4.584