Heming Yao1, Kayvan Najarian2, Jonathan Gryak3, Shrinivas Bishu4, Michael D Rice4, Akbar K Waljee5, H Jeffrey Wilkins6, Ryan W Stidham7. 1. Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, Michigan, USA. 2. Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, Michigan, USA; Department of Emergency Medicine, University of Michigan, Ann Arbor, Michigan, USA; Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, Michigan, USA; Michigan Center for Integrative Research in Critical Care (MCIRCC), University of Michigan, Ann Arbor, Michigan, USA; Michigan Institute for Data Science (MIDAS), University of Michigan, Ann Arbor, Michigan, USA. 3. Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, Michigan, USA; Michigan Institute for Data Science (MIDAS), University of Michigan, Ann Arbor, Michigan, USA. 4. Division of Gastroenterology and Hepatology, Department of Internal Medicine, University of Michigan, Ann Arbor, Michigan, USA. 5. Michigan Integrated Center for Health Analytics and Medical Prediction (MiCHAMP), University of Michigan, Ann Arbor, Michigan, USA; Division of Gastroenterology and Hepatology, Department of Internal Medicine, University of Michigan, Ann Arbor, Michigan, USA; Division of Gastroenterology and Hepatology, Department of Internal Medicine, Veteran Affairs Ann Arbor Health Care System, Ann Arbor, Michigan, USA; Veterans Affairs Center for Clinical Management Research, Ann Arbor, Michigan, USA. 6. Lycera Corporation, Plymouth Meeting, Pennsylvania, USA. 7. Department of Computational Medicine and Bioinformatics, University of Michigan, Ann Arbor, Michigan, USA; Michigan Integrated Center for Health Analytics and Medical Prediction (MiCHAMP), University of Michigan, Ann Arbor, Michigan, USA; Division of Gastroenterology and Hepatology, Department of Internal Medicine, University of Michigan, Ann Arbor, Michigan, USA.
Abstract
BACKGROUND AND AIMS: Endoscopy is essential for disease assessment in ulcerative colitis (UC), but subjectivity threatens accuracy and precision. We aimed to pilot a fully automated video analysis system for grading endoscopic disease in UC. METHODS: A developmental set of high-resolution UC endoscopic videos were assigned Mayo endoscopic scores (MESs) provided by 2 experienced reviewers. Video still-image stacks were annotated for image quality (informativeness) and MES. Models to predict still-image informativeness and disease severity were trained using convolutional neural networks. A template-matching grid search was used to estimate whole-video MESs provided by human reviewers using predicted still-image MES proportions. The automated whole-video MES workflow was tested using unaltered endoscopic videos from a multicenter UC clinical trial. RESULTS: The developmental high-resolution and testing multicenter clinical trial sets contained 51 and 264 videos, respectively. The still-image informative classifier had excellent performance with a sensitivity of 0.902 and specificity of 0.870. In high-resolution videos, fully automated methods correctly predicted MESs in 78% (41 of 50, κ = 0.84) of videos. In external clinical trial videos, reviewers agreed on MESs in 82.8% (140 of 169) of videos (κ = 0.78). Automated and central reviewer scoring agreement occurred in 57.1% of videos (κ = 0.59), but improved to 69.5% (107 of 169) when accounting for reviewer disagreement. Automated MES grading of clinical trial videos (often low resolution) correctly distinguished remission (MES 0,1) versus active disease (MES 2,3) in 83.7% (221 of 264) of videos. CONCLUSIONS: These early results support the potential for artificial intelligence to provide endoscopic disease grading in UC that approximates the scoring of experienced reviewers.
BACKGROUND AND AIMS: Endoscopy is essential for disease assessment in ulcerative colitis (UC), but subjectivity threatens accuracy and precision. We aimed to pilot a fully automated video analysis system for grading endoscopic disease in UC. METHODS: A developmental set of high-resolution UC endoscopic videos were assigned Mayo endoscopic scores (MESs) provided by 2 experienced reviewers. Video still-image stacks were annotated for image quality (informativeness) and MES. Models to predict still-image informativeness and disease severity were trained using convolutional neural networks. A template-matching grid search was used to estimate whole-video MESs provided by human reviewers using predicted still-image MES proportions. The automated whole-video MES workflow was tested using unaltered endoscopic videos from a multicenter UC clinical trial. RESULTS: The developmental high-resolution and testing multicenter clinical trial sets contained 51 and 264 videos, respectively. The still-image informative classifier had excellent performance with a sensitivity of 0.902 and specificity of 0.870. In high-resolution videos, fully automated methods correctly predicted MESs in 78% (41 of 50, κ = 0.84) of videos. In external clinical trial videos, reviewers agreed on MESs in 82.8% (140 of 169) of videos (κ = 0.78). Automated and central reviewer scoring agreement occurred in 57.1% of videos (κ = 0.59), but improved to 69.5% (107 of 169) when accounting for reviewer disagreement. Automated MES grading of clinical trial videos (often low resolution) correctly distinguished remission (MES 0,1) versus active disease (MES 2,3) in 83.7% (221 of 264) of videos. CONCLUSIONS: These early results support the potential for artificial intelligence to provide endoscopic disease grading in UC that approximates the scoring of experienced reviewers.
Authors: David Chen; Clifton Fulmer; Ilyssa O Gordon; Sana Syed; Ryan W Stidham; Niels Vande Casteele; Yi Qin; Katherine Falloon; Benjamin L Cohen; Robert Wyllie; Florian Rieder Journal: J Crohns Colitis Date: 2022-03-14 Impact factor: 10.020
Authors: Johanne Brooks-Warburton; James Ashton; Anjan Dhar; Tony Tham; Patrick B Allen; Sami Hoque; Laurence B Lovat; Shaji Sebastian Journal: Frontline Gastroenterol Date: 2021-12-10
Authors: Shirley Cohen-Mekelburg; Sameer Berry; Ryan W Stidham; Ji Zhu; Akbar K Waljee Journal: J Gastroenterol Hepatol Date: 2021-02 Impact factor: 4.029