Chung-Il Wi1, Sunghwan Sohn2, Mir Ali3, Elizabeth Krusemark1, Euijung Ryu2, Hongfang Liu4, Young J Juhn5. 1. Department of Pediatric and Adolescent Medicine, Mayo Clinic, Rochester, Minn; Asthma Epidemiology Research Unit, Mayo Clinic, Rochester, Minn. 2. Department of Health Sciences Research, Mayo Clinic, Rochester, Minn. 3. Department of Pediatrics, Sanford Children's Hospital, Sioux Falls, SD. 4. Department of Health Sciences Research, Mayo Clinic, Rochester, Minn. Electronic address: Liu.hongfang@mayo.edu. 5. Department of Pediatric and Adolescent Medicine, Mayo Clinic, Rochester, Minn; Asthma Epidemiology Research Unit, Mayo Clinic, Rochester, Minn. Electronic address: Juhn.young@mayo.edu.
Abstract
BACKGROUND: We developed and validated NLP-PAC, a natural language processing (NLP) algorithm based on predetermined asthma criteria (PAC) for asthma ascertainment using electronic health records at Mayo Clinic. OBJECTIVE: To adapt NLP-PAC in a different health care setting, Sanford Children Hospital, by assessing its external validity. METHODS: The study was designed as a retrospective cohort study that used a random sample of 2011-2012 Sanford Birth cohort (n = 595). Manual chart review was performed on the cohort for asthma ascertainment on the basis of the PAC. We then used half of the cohort as a training cohort (n = 298) and the other half as a blind test cohort to evaluate the adapted NLP-PAC algorithm. Association of known asthma-related risk factors with the Sanford-NLP algorithm-driven asthma ascertainment was tested. RESULTS: Among the eligible test cohort (n = 297), 160 (53%) were males, 268 (90%) white, and the median age was 2.3 years (range, 1.5-3.1 years). NLP-PAC, after adaptation, and the human abstractor identified 74 (25%) and 72 (24%) subjects, respectively, with 66 subjects identified by both approaches. Sensitivity, specificity, positive predictive value, and negative predictive value for the NLP algorithm in predicting asthma status were 92%, 96%, 89%, and 97%, respectively. The known risk factors for asthma identified by NLP (eg, smoking history) were similar to the ones identified by manual chart review. CONCLUSIONS: Successful implementation of NLP-PAC for asthma ascertainment in 2 different practice settings demonstrates the feasibility of automated asthma ascertainment leveraging electronic health record data with a potential to enable large-scale, multisite asthma studies to improve asthma care and research.
BACKGROUND: We developed and validated NLP-PAC, a natural language processing (NLP) algorithm based on predetermined asthma criteria (PAC) for asthma ascertainment using electronic health records at Mayo Clinic. OBJECTIVE: To adapt NLP-PAC in a different health care setting, Sanford Children Hospital, by assessing its external validity. METHODS: The study was designed as a retrospective cohort study that used a random sample of 2011-2012 Sanford Birth cohort (n = 595). Manual chart review was performed on the cohort for asthma ascertainment on the basis of the PAC. We then used half of the cohort as a training cohort (n = 298) and the other half as a blind test cohort to evaluate the adapted NLP-PAC algorithm. Association of known asthma-related risk factors with the Sanford-NLP algorithm-driven asthma ascertainment was tested. RESULTS: Among the eligible test cohort (n = 297), 160 (53%) were males, 268 (90%) white, and the median age was 2.3 years (range, 1.5-3.1 years). NLP-PAC, after adaptation, and the human abstractor identified 74 (25%) and 72 (24%) subjects, respectively, with 66 subjects identified by both approaches. Sensitivity, specificity, positive predictive value, and negative predictive value for the NLP algorithm in predicting asthma status were 92%, 96%, 89%, and 97%, respectively. The known risk factors for asthma identified by NLP (eg, smoking history) were similar to the ones identified by manual chart review. CONCLUSIONS: Successful implementation of NLP-PAC for asthma ascertainment in 2 different practice settings demonstrates the feasibility of automated asthma ascertainment leveraging electronic health record data with a potential to enable large-scale, multisite asthma studies to improve asthma care and research.
Authors: Young J Juhn; Hirohito Kita; Barbara P Yawn; Thomas G Boyce; Kwang H Yoo; Michaela E McGree; Amy L Weaver; Peter Wollan; Robert M Jacobson Journal: J Allergy Clin Immunol Date: 2008-09-13 Impact factor: 10.793
Authors: Pranab Haldar; Ian D Pavord; Ruth H Green; Dominic E Shaw; Michael A Berry; Michael Thomas; Christopher E Brightling; Andrew J Wardlaw Journal: Am J Respir Crit Care Med Date: 2008-05-14 Impact factor: 21.405
Authors: Seema Jain; Laurie Kamimoto; Anna M Bramley; Ann M Schmitz; Stephen R Benoit; Janice Louie; David E Sugerman; Jean K Druckenmiller; Kathleen A Ritger; Rashmi Chugh; Supriya Jasuja; Meredith Deutscher; Sanny Chen; John D Walker; Jeffrey S Duchin; Susan Lett; Susan Soliva; Eden V Wells; David Swerdlow; Timothy M Uyeki; Anthony E Fiore; Sonja J Olsen; Alicia M Fry; Carolyn B Bridges; Lyn Finelli Journal: N Engl J Med Date: 2009-10-08 Impact factor: 91.245
Authors: Jean Bousquet; James E Gern; Fernando D Martinez; Josep M Anto; Christine C Johnson; Patrick G Holt; Robert F Lemanske; Peter N Le Souëf; Robert S Tepper; Erika R M von Mutius; S Hasan Arshad; Leonard B Bacharier; Allan Becker; Kathleen Belanger; Anna Bergström; David I Bernstein; Michael D Cabana; Kecia N Carroll; Mario Castro; Philip J Cooper; Matthew W Gillman; Diane R Gold; John Henderson; Joachim Heinrich; Soo-Jong Hong; Daniel J Jackson; Thomas Keil; Anita L Kozyrskyj; Karin C Lødrup Carlsen; Rachel L Miller; Isabelle Momas; Wayne J Morgan; Patricia Noel; Dennis R Ownby; Mariona Pinart; Patrick H Ryan; Julie M Schwaninger; Malcolm R Sears; Angela Simpson; Henriette A Smit; Debra A Stern; Padmaja Subbarao; Rudolf Valenta; Xiaobin Wang; Scott T Weiss; Robert Wood; Anne L Wright; Rosalind J Wright; Alkis Togias; Peter J Gergen Journal: J Allergy Clin Immunol Date: 2014-03-15 Impact factor: 10.793
Authors: N Lazic; G Roberts; A Custovic; D Belgrave; C M Bishop; J Winn; J A Curtin; S Hasan Arshad; A Simpson Journal: Allergy Date: 2013-04-29 Impact factor: 13.146
Authors: Brian C Sauer; Barbara E Jones; Gary Globe; Jianwei Leng; Chao-Chin Lu; Tao He; Chia-Chen Teng; Patrick Sullivan; Qing Zeng Journal: EGEMS (Wash DC) Date: 2016-06-01
Authors: Tielman T Van Vleck; Lili Chan; Steven G Coca; Catherine K Craven; Ron Do; Stephen B Ellis; Joseph L Kannry; Ruth J F Loos; Peter A Bonis; Judy Cho; Girish N Nadkarni Journal: Int J Med Inform Date: 2019-07-06 Impact factor: 4.046
Authors: Monica Tang; Benjamin A Goldstein; Jingye He; Jillian H Hurst; Jason E Lang Journal: Ann Allergy Asthma Immunol Date: 2020-07-17 Impact factor: 6.347
Authors: Cody C Wyles; Meagan E Tibbo; Sunyang Fu; Yanshan Wang; Sunghwan Sohn; Walter K Kremers; Daniel J Berry; David G Lewallen; Hilal Maradit-Kremers Journal: J Bone Joint Surg Am Date: 2019-11-06 Impact factor: 5.284
Authors: Sunyang Fu; Guilherme S Lopes; Sandeep R Pagali; Bjoerg Thorsteinsdottir; Nathan K LeBrasseur; Andrew Wen; Hongfang Liu; Walter A Rocca; Janet E Olson; Jennifer St Sauver; Sunghwan Sohn Journal: J Gerontol A Biol Sci Med Sci Date: 2022-03-03 Impact factor: 6.053
Authors: Sunyang Fu; Guilherme S Lopes; Sandeep R Pagali; Bjoerg Thorsteinsdottir; Nathan K LeBrasseur; Andrew Wen; Hongfang Liu; Walter A Rocca; Janet E Olson; Jennifer St Sauver; Sunghwan Sohn Journal: J Gerontol A Biol Sci Med Sci Date: 2022-03-03 Impact factor: 6.053