Isabella F Churchill1, Anthony A Gatti2, Danielle A Hylton1, Kerrie A Sullivan1, Yogita S Patel3, Grigorious I Leontiadis4, Forough Farrokhyar1, Waël C Hanna5. 1. Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada; Division of Thoracic Surgery, Department of Surgery, St Joseph's Healthcare Hamilton, Hamilton, Ontario, Canada. 2. NeuralSeg Ltd, Hamilton, Ontario, Canada. 3. Division of Thoracic Surgery, Department of Surgery, St Joseph's Healthcare Hamilton, Hamilton, Ontario, Canada. 4. Division of Gastroenterology and Farncombe Family Digestive Health Research Institute, Department of Medicine, McMaster University, Hamilton, Ontario, Canada. 5. Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada; Division of Thoracic Surgery, Department of Surgery, St Joseph's Healthcare Hamilton, Hamilton, Ontario, Canada. Electronic address: hannaw@mcmaster.ca.
Abstract
BACKGROUND: Endobronchial ultrasound (EBUS) has features that allow a high accuracy for predicting lymph node (LN) malignancy. However their clinical application remains limited because of high operator dependency. We hypothesized that an artificial intelligence algorithm (NeuralSeg; NeuralSeg Ltd, Hamilton, Ontario, Canada) is capable of accurately identifying and predicting LN malignancy based on EBUS images. METHODS: In the derivation phase EBUS images were segmented twice by an endosonographer and used as controls in 5-fold cross-validation training of NeuralSeg. In the validation phase the algorithm was tested on new images it had not seen before. Logistic regression and receiver operator characteristic curves were used to determine NeuralSeg's capability of discrimination between benign and malignant LNs, using pathologic specimens as the gold standard. RESULTS: Two hundred ninety-eight LNs from 140 patients were used for derivation and 108 LNs from 47 patients for validation. In the derivation cohort NeuralSeg was able to predict malignant LNs with an accuracy of 73.8% (95% confidence interval [CI], 68.4%-78.7%). In the validation cohort NeuralSeg had an accuracy of 72.9% (95% CI, 63.5%-81.0%), specificity of 90.8% (95% CI, 81.9%-96.2%), and negative predictive value of 75.9% (95% CI, 71.5%-79.9%). NeuralSeg showed higher diagnostic discrimination during validation compared with derivation (c-statistic = 0.75 [95% CI, 0.65-0.85] vs 0.63 [95% CI, 0.54-0.72], respectively). CONCLUSIONS: NeuralSeg is able to accurately rule out nodal metastasis and can possibly be used as an adjunct to EBUS when nodal biopsy is not possible or inconclusive. Future work to evaluate the algorithm in a clinical trial is required.
BACKGROUND: Endobronchial ultrasound (EBUS) has features that allow a high accuracy for predicting lymph node (LN) malignancy. However their clinical application remains limited because of high operator dependency. We hypothesized that an artificial intelligence algorithm (NeuralSeg; NeuralSeg Ltd, Hamilton, Ontario, Canada) is capable of accurately identifying and predicting LN malignancy based on EBUS images. METHODS: In the derivation phase EBUS images were segmented twice by an endosonographer and used as controls in 5-fold cross-validation training of NeuralSeg. In the validation phase the algorithm was tested on new images it had not seen before. Logistic regression and receiver operator characteristic curves were used to determine NeuralSeg's capability of discrimination between benign and malignant LNs, using pathologic specimens as the gold standard. RESULTS: Two hundred ninety-eight LNs from 140 patients were used for derivation and 108 LNs from 47 patients for validation. In the derivation cohort NeuralSeg was able to predict malignant LNs with an accuracy of 73.8% (95% confidence interval [CI], 68.4%-78.7%). In the validation cohort NeuralSeg had an accuracy of 72.9% (95% CI, 63.5%-81.0%), specificity of 90.8% (95% CI, 81.9%-96.2%), and negative predictive value of 75.9% (95% CI, 71.5%-79.9%). NeuralSeg showed higher diagnostic discrimination during validation compared with derivation (c-statistic = 0.75 [95% CI, 0.65-0.85] vs 0.63 [95% CI, 0.54-0.72], respectively). CONCLUSIONS: NeuralSeg is able to accurately rule out nodal metastasis and can possibly be used as an adjunct to EBUS when nodal biopsy is not possible or inconclusive. Future work to evaluate the algorithm in a clinical trial is required.