Jacob A Martin1,2,3, Andrew Crane-Droesch2, Folasade C Lapite4, Joseph C Puhl2, Tyler E Kmiec2, Jasmine A Silvestri2, Lyle H Ungar5, Bruce P Kinosian3,6,7, Blanca E Himes8, Rebecca A Hubbard8, Joshua M Diamond9, Vivek Ahya9, Michael W Sims9, Scott D Halpern2,3,8,9, Gary E Weissman2,3,9. 1. Division of Cardiology, Department of Medicine, New York University Grossman School of Medicine, New York, New York, USA. 2. Palliative and Advanced Illness Research (PAIR) Center, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA. 3. Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA. 4. Tulane University School of Medicine, New Orleans, Louisiana, USA. 5. Department of Computer and Information Science, University of Pennsylvania School of Engineering and Applied Science, Philadelphia, Pennsylvania, USA. 6. Division of Geriatrics, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA. 7. Geriatrics and Extended Care Data Analysis Center, Corporal Michael J Crescenz VA Medical Center, Philadelphia, Pennsylvania, USA. 8. Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA. 9. Pulmonary, Allergy, and Critical Care Division, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA.
Abstract
OBJECTIVE: Frailty is a prevalent risk factor for adverse outcomes among patients with chronic lung disease. However, identifying frail patients who may benefit from interventions is challenging using standard data sources. We therefore sought to identify phrases in clinical notes in the electronic health record (EHR) that describe actionable frailty syndromes. MATERIALS AND METHODS: We used an active learning strategy to select notes from the EHR and annotated each sentence for 4 actionable aspects of frailty: respiratory impairment, musculoskeletal problems, fall risk, and nutritional deficiencies. We compared the performance of regression, tree-based, and neural network models to predict the labels for each sentence. We evaluated performance with the scaled Brier score (SBS), where 1 is perfect and 0 is uninformative, and the positive predictive value (PPV). RESULTS: We manually annotated 155 952 sentences from 326 patients. Elastic net regression had the best performance across all 4 frailty aspects (SBS 0.52, 95% confidence interval [CI] 0.49-0.54) followed by random forests (SBS 0.49, 95% CI 0.47-0.51), and multi-task neural networks (SBS 0.39, 95% CI 0.37-0.42). For the elastic net model, the PPV for identifying the presence of respiratory impairment was 54.8% (95% CI 53.3%-56.6%) at a sensitivity of 80%. DISCUSSION: Classification models using EHR notes can effectively identify actionable aspects of frailty among patients living with chronic lung disease. Regression performed better than random forest and neural network models. CONCLUSIONS: NLP-based models offer promising support to population health management programs that seek to identify and refer community-dwelling patients with frailty for evidence-based interventions.
OBJECTIVE: Frailty is a prevalent risk factor for adverse outcomes among patients with chronic lung disease. However, identifying frail patients who may benefit from interventions is challenging using standard data sources. We therefore sought to identify phrases in clinical notes in the electronic health record (EHR) that describe actionable frailty syndromes. MATERIALS AND METHODS: We used an active learning strategy to select notes from the EHR and annotated each sentence for 4 actionable aspects of frailty: respiratory impairment, musculoskeletal problems, fall risk, and nutritional deficiencies. We compared the performance of regression, tree-based, and neural network models to predict the labels for each sentence. We evaluated performance with the scaled Brier score (SBS), where 1 is perfect and 0 is uninformative, and the positive predictive value (PPV). RESULTS: We manually annotated 155 952 sentences from 326 patients. Elastic net regression had the best performance across all 4 frailty aspects (SBS 0.52, 95% confidence interval [CI] 0.49-0.54) followed by random forests (SBS 0.49, 95% CI 0.47-0.51), and multi-task neural networks (SBS 0.39, 95% CI 0.37-0.42). For the elastic net model, the PPV for identifying the presence of respiratory impairment was 54.8% (95% CI 53.3%-56.6%) at a sensitivity of 80%. DISCUSSION: Classification models using EHR notes can effectively identify actionable aspects of frailty among patients living with chronic lung disease. Regression performed better than random forest and neural network models. CONCLUSIONS: NLP-based models offer promising support to population health management programs that seek to identify and refer community-dwelling patients with frailty for evidence-based interventions.
Authors: Kenneth Rockwood; Xiaowei Song; Chris MacKnight; Howard Bergman; David B Hogan; Ian McDowell; Arnold Mitnitski Journal: CMAJ Date: 2005-08-30 Impact factor: 8.262
Authors: Frits M E Franssen; Roelinka Broekhuizen; Paul P Janssen; Emiel F M Wouters; Annemie M W J Schols Journal: Chest Date: 2004-06 Impact factor: 9.410
Authors: Thomas M Gill; Dorothy I Baker; Margaret Gottschalk; Peter N Peduzzi; Heather Allore; Amy Byers Journal: N Engl J Med Date: 2002-10-03 Impact factor: 91.245
Authors: Martijn A Spruit; Sally J Singh; Chris Garvey; Richard ZuWallack; Linda Nici; Carolyn Rochester; Kylie Hill; Anne E Holland; Suzanne C Lareau; William D-C Man; Fabio Pitta; Louise Sewell; Jonathan Raskin; Jean Bourbeau; Rebecca Crouch; Frits M E Franssen; Richard Casaburi; Jan H Vercoulen; Ioannis Vogiatzis; Rik Gosselink; Enrico M Clini; Tanja W Effing; François Maltais; Job van der Palen; Thierry Troosters; Daisy J A Janssen; Eileen Collins; Judith Garcia-Aymerich; Dina Brooks; Bonnie F Fahy; Milo A Puhan; Martine Hoogendoorn; Rachel Garrod; Annemie M W J Schols; Brian Carlin; Roberto Benzo; Paula Meek; Mike Morgan; Maureen P M H Rutten-van Mölken; Andrew L Ries; Barry Make; Roger S Goldstein; Claire A Dowson; Jan L Brozek; Claudio F Donner; Emiel F M Wouters Journal: Am J Respir Crit Care Med Date: 2013-10-15 Impact factor: 21.405
Authors: Kristina Doing-Harris; Bruce E Bray; Anne Thackeray; Rashmee U Shah; Yijun Shao; Yan Cheng; Qing Zeng-Treitler; Jennifer H Garvin; Charlene Weir Journal: J Biomed Semantics Date: 2019-01-18