Timothy D Imler1, Justin Morea2, Charles Kahi3, Eric A Sherer, Jon Cardwell4, Cynthia S Johnson5, Huiping Xu5, Dennis Ahnen6, Fadi Antaki7, Christopher Ashley8, Gyorgy Baffy9, Ilseung Cho10, Jason Dominitz11, Jason Hou12, Mark Korsten13, Anil Nagar14, Kittichai Promrat15, Douglas Robertson16, Sameer Saini17, Amandeep Shergill18, Walter Smalley19, Thomas F Imperiale20. 1. 1] Division of Gastroenterology and Hepatology, Indiana University School of Medicine, Indianapolis, Indiana, USA [2] Department of Medicine, Indiana University School of Medicine, Indianapolis, Indiana, USA [3] Department of Biomedical Informatics, Regenstrief Institute, LLC, Indianapolis, Indiana, USA. 2. 1] Department of Medicine, Indiana University School of Medicine, Indianapolis, Indiana, USA [2] Department of Biomedical Informatics, Regenstrief Institute, LLC, Indianapolis, Indiana, USA. 3. 1] Division of Gastroenterology and Hepatology, Indiana University School of Medicine, Indianapolis, Indiana, USA [2] Department of Medicine, Indiana University School of Medicine, Indianapolis, Indiana, USA [3] Center of Innovation, Health Services Research and Development, Richard L, Roudebush VA Medical Center, Indianapolis, Indiana, USA. 4. Center of Innovation, Health Services Research and Development, Richard L, Roudebush VA Medical Center, Indianapolis, Indiana, USA. 5. Department of Biostatistics, Indiana University School of Medicine, Indianapolis, Indiana, USA. 6. Division of Gastroenterology, University of Colorado, Denver, Colorado, USA. 7. Division of Gastroenterology, Wayne State University, Detroit, Michigan, USA. 8. Division of Gastroenterology, Albany Medical College, Albany, New York, USA. 9. Department of Medicine, VA Boston Healthcare System, Boston, Massachusetts, USA. 10. Division of Gastroenterology, New York University School of Medicine, New York, New York, USA. 11. Division of Gastroenterology, University of Washington School of Medicine, Seattle, Washington, USA. 12. Division of Gastroenterology and Hepatology, Baylor College of Medicine, Houston, Texas, USA. 13. Division of Gastroenterology, Icahn School of Medicine at Mount Sinai, Bronx, New York, USA. 14. Division of Digestive Diseases, Yale School of Medicine, New Haven, Connecticut, USA. 15. Division of Gastroenterology, Brown Medical School, Providence, Rhode Island, USA. 16. Division of Gastroenterology, The Dartmouth Institute, Lebanon, New Hampshire, USA. 17. Division of Gastroenterology, University of Michigan, Ann Arbor, Michigan, USA. 18. Division of Gastroenterology, University of California at San Francisco, San Francisco, California, USA. 19. Division of Gastroenterology, Vanderbilt University, Nashville, Tennessee, USA. 20. 1] Division of Gastroenterology and Hepatology, Indiana University School of Medicine, Indianapolis, Indiana, USA [2] Department of Medicine, Indiana University School of Medicine, Indianapolis, Indiana, USA [3] Center of Innovation, Health Services Research and Development, Richard L, Roudebush VA Medical Center, Indianapolis, Indiana, USA [4] Health Services Research, Regenstrief Institute, Indianapolis, Indiana, USA.
Abstract
BACKGROUND: An accurate system for tracking of colonoscopy quality and surveillance intervals could improve the effectiveness and cost-effectiveness of colorectal cancer (CRC) screening and surveillance. The purpose of this study was to create and test such a system across multiple institutions utilizing natural language processing (NLP). METHODS: From 42,569 colonoscopies with pathology records from 13 centers, we randomly sampled 750 paired reports. We trained (n=250) and tested (n=500) an NLP-based program with 19 measurements that encompass colonoscopy quality measures and surveillance interval determination, using blinded, paired, annotated expert manual review as the reference standard. The remaining 41,819 nonannotated documents were processed through the NLP system without manual review to assess performance consistency. The primary outcome was system accuracy across the 19 measures. RESULTS: A total of 176 (23.5%) documents with 252 (1.8%) discrepant content points resulted from paired annotation. Error rate within the 500 test documents was 31.2% for NLP and 25.4% for the paired annotators (P=0.001). At the content point level within the test set, the error rate was 3.5% for NLP and 1.9% for the paired annotators (P=0.04). When eight vaguely worded documents were removed, 125 of 492 (25.4%) were incorrect by NLP and 104 of 492 (21.1%) by the initial annotator (P=0.07). Rates of pathologic findings calculated from NLP were similar to those calculated by annotation for the majority of measurements. Test set accuracy was 99.6% for CRC, 95% for advanced adenoma, 94.6% for nonadvanced adenoma, 99.8% for advanced sessile serrated polyps, 99.2% for nonadvanced sessile serrated polyps, 96.8% for large hyperplastic polyps, and 96.0% for small hyperplastic polyps. Lesion location showed high accuracy (87.0-99.8%). Accuracy for number of adenomas was 92%. CONCLUSIONS: NLP can accurately report adenoma detection rate and the components for determining guideline-adherent colonoscopy surveillance intervals across multiple sites that utilize different methods for reporting colonoscopy findings.
BACKGROUND: An accurate system for tracking of colonoscopy quality and surveillance intervals could improve the effectiveness and cost-effectiveness of colorectal cancer (CRC) screening and surveillance. The purpose of this study was to create and test such a system across multiple institutions utilizing natural language processing (NLP). METHODS: From 42,569 colonoscopies with pathology records from 13 centers, we randomly sampled 750 paired reports. We trained (n=250) and tested (n=500) an NLP-based program with 19 measurements that encompass colonoscopy quality measures and surveillance interval determination, using blinded, paired, annotated expert manual review as the reference standard. The remaining 41,819 nonannotated documents were processed through the NLP system without manual review to assess performance consistency. The primary outcome was system accuracy across the 19 measures. RESULTS: A total of 176 (23.5%) documents with 252 (1.8%) discrepant content points resulted from paired annotation. Error rate within the 500 test documents was 31.2% for NLP and 25.4% for the paired annotators (P=0.001). At the content point level within the test set, the error rate was 3.5% for NLP and 1.9% for the paired annotators (P=0.04). When eight vaguely worded documents were removed, 125 of 492 (25.4%) were incorrect by NLP and 104 of 492 (21.1%) by the initial annotator (P=0.07). Rates of pathologic findings calculated from NLP were similar to those calculated by annotation for the majority of measurements. Test set accuracy was 99.6% for CRC, 95% for advanced adenoma, 94.6% for nonadvanced adenoma, 99.8% for advanced sessile serrated polyps, 99.2% for nonadvanced sessile serrated polyps, 96.8% for large hyperplastic polyps, and 96.0% for small hyperplastic polyps. Lesion location showed high accuracy (87.0-99.8%). Accuracy for number of adenomas was 92%. CONCLUSIONS: NLP can accurately report adenoma detection rate and the components for determining guideline-adherent colonoscopy surveillance intervals across multiple sites that utilize different methods for reporting colonoscopy findings.
Authors: Henk Harkema; Wendy W Chapman; Melissa Saul; Evan S Dellon; Robert E Schoen; Ateev Mehrotra Journal: J Am Med Inform Assoc Date: 2011-09-21 Impact factor: 4.497
Authors: David A Lieberman; Douglas K Rex; Sidney J Winawer; Francis M Giardiello; David A Johnson; Theodore R Levin Journal: Gastroenterology Date: 2012-07-03 Impact factor: 22.682
Authors: Mei Liu; Anushi Shah; Min Jiang; Neeraja B Peterson; Qi Dai; Melinda C Aldrich; Qingxia Chen; Erica A Bowton; Hongfang Liu; Joshua C Denny; Hua Xu Journal: AMIA Annu Symp Proc Date: 2012-11-03
Authors: Douglas K Rex; Dennis J Ahnen; John A Baron; Kenneth P Batts; Carol A Burke; Randall W Burt; John R Goldblum; José G Guillem; Charles J Kahi; Matthew F Kalady; Michael J O'Brien; Robert D Odze; Shuji Ogino; Susan Parry; Dale C Snover; Emina Emilia Torlakovic; Paul E Wise; Joanne Young; James Church Journal: Am J Gastroenterol Date: 2012-06-19 Impact factor: 10.864
Authors: Rodrigo Jover; Pedro Zapater; Eduardo Polanía; Luis Bujanda; Angel Lanas; José A Hermo; Joaquín Cubiella; Akiko Ono; Yanira González-Méndez; Antonio Peris; María Pellisé; Agustín Seoane; Alberto Herreros-de-Tejada; Marta Ponce; José C Marín-Gabriel; María Chaparro; Guillermo Cacho; Servando Fernández-Díez; Juan Arenas; Federico Sopeña; Luisa de-Castro; Pablo Vega-Villaamil; María Rodríguez-Soler; Fernando Carballo; Dolores Salas; Juan D Morillas; Montserrat Andreu; Enrique Quintero; Antoni Castells Journal: Gastrointest Endosc Date: 2012-12-04 Impact factor: 9.427
Authors: Timothy D Imler; Stuart Sherman; Thomas F Imperiale; Huiping Xu; Fangqian Ouyang; Christopher Beesley; Charity Hilton; Gregory A Coté Journal: Gastrointest Endosc Date: 2017-05-03 Impact factor: 9.427
Authors: Ateev Mehrotra; Michele Morris; Rebecca A Gourevitch; David S Carrell; Daniel A Leffler; Sherri Rose; Julia B Greer; Seth D Crockett; Andrew Baer; Robert E Schoen Journal: Gastrointest Endosc Date: 2017-09-01 Impact factor: 9.427
Authors: Rebecca A Gourevitch; Sherri Rose; Seth D Crockett; Michele Morris; David S Carrell; Julia B Greer; Reetesh K Pai; Robert E Schoen; Ateev Mehrotra Journal: Am J Gastroenterol Date: 2018-01-30 Impact factor: 10.864