Naveed Afzal1, Sunghwan Sohn1, Sara Abram2, Christopher G Scott1, Rajeev Chaudhry3, Hongfang Liu1, Iftikhar J Kullo2, Adelaide M Arruda-Olson4. 1. Department of Health Sciences Research, Mayo Clinic, Rochester, Minn. 2. Department of Cardiovascular Diseases, Mayo Clinic, Rochester, Minn. 3. Division of Primary Care Medicine, Knowledge Delivery Center and Center for Innovation, Mayo Clinic, Rochester, Minn. 4. Department of Cardiovascular Diseases, Mayo Clinic, Rochester, Minn. Electronic address: olson.adelaide@mayo.edu.
Abstract
OBJECTIVE: Lower extremity peripheral arterial disease (PAD) is highly prevalent and affects millions of individuals worldwide. We developed a natural language processing (NLP) system for automated ascertainment of PAD cases from clinical narrative notes and compared the performance of the NLP algorithm with billing code algorithms, using ankle-brachial index test results as the gold standard. METHODS: We compared the performance of the NLP algorithm to (1) results of gold standard ankle-brachial index; (2) previously validated algorithms based on relevant International Classification of Diseases, Ninth Revision diagnostic codes (simple model); and (3) a combination of International Classification of Diseases, Ninth Revision codes with procedural codes (full model). A dataset of 1569 patients with PAD and controls was randomly divided into training (n = 935) and testing (n = 634) subsets. RESULTS: We iteratively refined the NLP algorithm in the training set including narrative note sections, note types, and service types, to maximize its accuracy. In the testing dataset, when compared with both simple and full models, the NLP algorithm had better accuracy (NLP, 91.8%; full model, 81.8%; simple model, 83%; P < .001), positive predictive value (NLP, 92.9%; full model, 74.3%; simple model, 79.9%; P < .001), and specificity (NLP, 92.5%; full model, 64.2%; simple model, 75.9%; P < .001). CONCLUSIONS: A knowledge-driven NLP algorithm for automatic ascertainment of PAD cases from clinical notes had greater accuracy than billing code algorithms. Our findings highlight the potential of NLP tools for rapid and efficient ascertainment of PAD cases from electronic health records to facilitate clinical investigation and eventually improve care by clinical decision support.
OBJECTIVE: Lower extremity peripheral arterial disease (PAD) is highly prevalent and affects millions of individuals worldwide. We developed a natural language processing (NLP) system for automated ascertainment of PAD cases from clinical narrative notes and compared the performance of the NLP algorithm with billing code algorithms, using ankle-brachial index test results as the gold standard. METHODS: We compared the performance of the NLP algorithm to (1) results of gold standard ankle-brachial index; (2) previously validated algorithms based on relevant International Classification of Diseases, Ninth Revision diagnostic codes (simple model); and (3) a combination of International Classification of Diseases, Ninth Revision codes with procedural codes (full model). A dataset of 1569 patients with PAD and controls was randomly divided into training (n = 935) and testing (n = 634) subsets. RESULTS: We iteratively refined the NLP algorithm in the training set including narrative note sections, note types, and service types, to maximize its accuracy. In the testing dataset, when compared with both simple and full models, the NLP algorithm had better accuracy (NLP, 91.8%; full model, 81.8%; simple model, 83%; P < .001), positive predictive value (NLP, 92.9%; full model, 74.3%; simple model, 79.9%; P < .001), and specificity (NLP, 92.5%; full model, 64.2%; simple model, 75.9%; P < .001). CONCLUSIONS: A knowledge-driven NLP algorithm for automatic ascertainment of PAD cases from clinical notes had greater accuracy than billing code algorithms. Our findings highlight the potential of NLP tools for rapid and efficient ascertainment of PAD cases from electronic health records to facilitate clinical investigation and eventually improve care by clinical decision support.
Authors: Jeffrey W Olin; David E Allie; Michael Belkin; Robert O Bonow; Donald E Casey; Mark A Creager; Thomas C Gerber; Alan T Hirsch; Michael R Jaff; John A Kaufman; Curtis A Lewis; Edward T Martin; Louis G Martin; Peter Sheehan; Kerry J Stewart; Diane Treat-Jacobson; Christopher J White; Zhi-Jie Zheng Journal: J Vasc Surg Date: 2010-12 Impact factor: 4.268
Authors: Wendy W Chapman; Lee M Christensen; Michael M Wagner; Peter J Haug; Oleg Ivanov; John N Dowling; Robert T Olszewski Journal: Artif Intell Med Date: 2005-01 Impact factor: 5.326
Authors: Jacqueline Saw; Deepak L Bhatt; David J Moliterno; Sorin J Brener; Steven R Steinhubl; A Michael Lincoff; James E Tcheng; Robert A Harrington; Maarten Simoons; TingFei Hu; Mobeen A Sheikh; Dean J Kereiakes; Eric J Topol Journal: J Am Coll Cardiol Date: 2006-09-26 Impact factor: 24.094
Authors: Alan T Hirsch; Ziv J Haskal; Norman R Hertzer; Curtis W Bakal; Mark A Creager; Jonathan L Halperin; Loren F Hiratzka; William R C Murphy; Jeffrey W Olin; Jules B Puschett; Kenneth A Rosenfield; David Sacks; James C Stanley; Lloyd M Taylor; Christopher J White; John White; Rodney A White; Elliott M Antman; Sidney C Smith; Cynthia D Adams; Jeffrey L Anderson; David P Faxon; Valentin Fuster; Raymond J Gibbons; Jonathan L Halperin; Loren F Hiratzka; Sharon A Hunt; Alice K Jacobs; Rick Nishimura; Joseph P Ornato; Richard L Page; Barbara Riegel Journal: J Am Coll Cardiol Date: 2006-03-21 Impact factor: 24.094
Authors: Jin Fan; Adelaide M Arruda-Olson; Cynthia L Leibson; Carin Smith; Guanghui Liu; Kent R Bailey; Iftikhar J Kullo Journal: J Am Med Inform Assoc Date: 2013-10-28 Impact factor: 4.497
Authors: Sungrim Moon; Sijia Liu; Christopher G Scott; Sujith Samudrala; Mohamed M Abidian; Jeffrey B Geske; Peter A Noseworthy; Jane L Shellum; Rajeev Chaudhry; Steve R Ommen; Rick A Nishimura; Hongfang Liu; Adelaide M Arruda-Olson Journal: Int J Med Inform Date: 2019-05-13 Impact factor: 4.046
Authors: Nakeya Dewaswala; David Chen; Huzefa Bhopalwala; Vinod C Kaggal; Sean P Murphy; J Martijn Bos; Jeffrey B Geske; Bernard J Gersh; Steve R Ommen; Philip A Araoz; Michael J Ackerman; Adelaide M Arruda-Olson Journal: BMC Med Inform Decis Mak Date: 2022-10-18 Impact factor: 3.298
Authors: Kelly D Myers; Joshua W Knowles; David Staszak; Michael D Shapiro; William Howard; Mrinal Yadava; David Zuzick; Latoya Williamson; Nigam H Shah; Juan M Banda; Joe Leader; William C Cromwell; Ed Trautman; Michael F Murray; Seth J Baum; Seth Myers; Samuel S Gidding; Katherine Wilemon; Daniel J Rader Journal: Lancet Digit Health Date: 2019-10-21