Mindy K Ross1, Henry Zheng2, Bing Zhu2, Ailina Lao3, Hyejin Hong3, Alamelu Natesan1, Melina Radparvar1, Alex A T Bui2. 1. Department of Pediatrics, University of California Los Angeles, Los Angeles, California, United States. 2. Department of Radiological Sciences, University of California Los Angeles, Los Angeles, California, United States. 3. University of California Los Angeles, Los Angeles, California, United States.
Abstract
OBJECTIVES: Asthma is a heterogenous condition with significant diagnostic complexity, including variations in symptoms and temporal criteria. The disease can be difficult for clinicians to diagnose accurately. Properly identifying asthma patients from the electronic health record is consequently challenging as current algorithms (computable phenotypes) rely on diagnostic codes (e.g., International Classification of Disease, ICD) in addition to other criteria (e.g., inhaler medications)-but presume an accurate diagnosis. As such, there is no universally accepted or rigorously tested computable phenotype for asthma. METHODS: We compared two established asthma computable phenotypes: the Chicago Area Patient-Outcomes Research Network (CAPriCORN) and Phenotype KnowledgeBase (PheKB). We established a large-scale, consensus gold standard (n = 1,365) from the University of California, Los Angeles Health System's clinical data warehouse for patients 5 to 17 years old. Results were manually reviewed and predictive performance (positive predictive value [PPV], sensitivity/specificity, F1-score) determined. We then examined the classification errors to gain insight for future algorithm optimizations. RESULTS: As applied to our final cohort of 1,365 expert-defined gold standard patients, the CAPriCORN algorithms performed with a balanced PPV = 95.8% (95% CI: 94.4-97.2%), sensitivity = 85.7% (95% CI: 83.9-87.5%), and harmonized F1 = 90.4% (95% CI: 89.2-91.7%). The PheKB algorithm was performed with a balanced PPV = 83.1% (95% CI: 80.5-85.7%), sensitivity = 69.4% (95% CI: 66.3-72.5%), and F1 = 75.4% (95% CI: 73.1-77.8%). Four categories of errors were identified related to method limitations, disease definition, human error, and design implementation. CONCLUSION: The performance of the CAPriCORN and PheKB algorithms was lower than previously reported as applied to pediatric data (PPV = 97.7 and 96%, respectively). There is room to improve the performance of current methods, including targeted use of natural language processing and clinical feature engineering. Thieme. All rights reserved.
OBJECTIVES: Asthma is a heterogenous condition with significant diagnostic complexity, including variations in symptoms and temporal criteria. The disease can be difficult for clinicians to diagnose accurately. Properly identifying asthma patients from the electronic health record is consequently challenging as current algorithms (computable phenotypes) rely on diagnostic codes (e.g., International Classification of Disease, ICD) in addition to other criteria (e.g., inhaler medications)-but presume an accurate diagnosis. As such, there is no universally accepted or rigorously tested computable phenotype for asthma. METHODS: We compared two established asthma computable phenotypes: the Chicago Area Patient-Outcomes Research Network (CAPriCORN) and Phenotype KnowledgeBase (PheKB). We established a large-scale, consensus gold standard (n = 1,365) from the University of California, Los Angeles Health System's clinical data warehouse for patients 5 to 17 years old. Results were manually reviewed and predictive performance (positive predictive value [PPV], sensitivity/specificity, F1-score) determined. We then examined the classification errors to gain insight for future algorithm optimizations. RESULTS: As applied to our final cohort of 1,365 expert-defined gold standard patients, the CAPriCORN algorithms performed with a balanced PPV = 95.8% (95% CI: 94.4-97.2%), sensitivity = 85.7% (95% CI: 83.9-87.5%), and harmonized F1 = 90.4% (95% CI: 89.2-91.7%). The PheKB algorithm was performed with a balanced PPV = 83.1% (95% CI: 80.5-85.7%), sensitivity = 69.4% (95% CI: 66.3-72.5%), and F1 = 75.4% (95% CI: 73.1-77.8%). Four categories of errors were identified related to method limitations, disease definition, human error, and design implementation. CONCLUSION: The performance of the CAPriCORN and PheKB algorithms was lower than previously reported as applied to pediatric data (PPV = 97.7 and 96%, respectively). There is room to improve the performance of current methods, including targeted use of natural language processing and clinical feature engineering. Thieme. All rights reserved.
Authors: Jennifer A Pacheco; Pedro C Avila; Jason A Thompson; May Law; Jihan A Quraishi; Alyssa K Greiman; Eric M Just; Abel Kho Journal: AMIA Annu Symp Proc Date: 2009-11-14
Authors: Guoqian Jiang; Richard C Kiefer; Luke V Rasmussen; Harold R Solbrig; Huan Mo; Jennifer A Pacheco; Jie Xu; Enid Montague; William K Thompson; Joshua C Denny; Christopher G Chute; Jyotishman Pathak Journal: J Biomed Inform Date: 2016-07-05 Impact factor: 6.317
Authors: Jacqueline C Kirby; Peter Speltz; Luke V Rasmussen; Melissa Basford; Omri Gottesman; Peggy L Peissig; Jennifer A Pacheco; Gerard Tromp; Jyotishman Pathak; David S Carrell; Stephen B Ellis; Todd Lingren; Will K Thompson; Guergana Savova; Jonathan Haines; Dan M Roden; Paul A Harris; Joshua C Denny Journal: J Am Med Inform Assoc Date: 2016-03-28 Impact factor: 4.497
Authors: Alon Geva; Jessica L Gronsbell; Tianxi Cai; Tianrun Cai; Shawn N Murphy; Jessica C Lyons; Michelle M Heinz; Marc D Natter; Nandan Patibandla; Jonathan Bickel; Mary P Mullen; Kenneth D Mandl Journal: J Pediatr Date: 2017-06-16 Impact factor: 4.406
Authors: Zubair Afzal; Marjolein Engelkes; Katia M C Verhamme; Hettie M Janssens; Miriam C J M Sturkenboom; Jan A Kors; Martijn J Schuemie Journal: Pharmacoepidemiol Drug Saf Date: 2013-04-17 Impact factor: 2.890
Authors: Majid Afshar; Valerie G Press; Rachel G Robison; Abel N Kho; Sindhura Bandi; Ashvini Biswas; Pedro C Avila; Harsha Vardhan Madan Kumar; Byung Yu; Edward T Naureckas; Sharmilee M Nyenhuis; Christopher D Codispoti Journal: J Asthma Date: 2017-11-10 Impact factor: 2.515