Simon R Yadgir1, Collin Engstrom1,2, Gwen Costa Jacobsohn1, Rebecca K Green1, Courtney M C Jones3,4, Jeremy T Cushman3,4,5, Thomas V Caprio6, Amy J H Kind7,8,9, Michael Lohmeier1, Manish N Shah1,7,10, Brian W Patterson1,11,12. 1. BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison School of Medicine and Public Health, Madison, Wisconsin, USA. 2. Department of Computer Science, Winona State University, Rochester, MN, USA. 3. Department of Emergency Medicine, University of Rochester Medical Center, Rochester, New York, USA. 4. Department of Public Health Sciences, University of Rochester Medical Center, Rochester, New York, USA. 5. Department of Health Policy and Management, School of Public Health, University of Maryland, College Park, Maryland, USA. 6. Division of Geriatrics, Department of Medicine, University of Rochester Medical Center, Rochester, New York, USA. 7. Division of Geriatrics and Gerontology, Department of Medicine, School of Medicine and Public Health, University of Wisconsin, Madison, Wisconsin, USA. 8. William S. Middleton Veterans Affairs Geriatrics Research, Education, and Clinical Center, Madison, Wisconsin, USA. 9. UW Center for Health Disparities Research, School of Medicine and Public Health, University of Wisconsin-Madison, Madison, Wisconsin, USA. 10. Department of Population Health Sciences, School of Medicine and Public Health, University of Wisconsin-Madison, Madison, Wisconsin, USA. 11. Health Innovation Program, University of Wisconsin-Madison, Madison, Wisconsin, USA. 12. Department of Industrial and Systems Engineering, Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, Madison, Wisconsin, USA.
Abstract
BACKGROUND/ OBJECTIVES: Despite a high prevalence and association with poor outcomes, screening to identify cognitive impairment (CI) in the emergency department (ED) is uncommon. Identification of high-risk subsets of older adults is a critical challenge to expanding screening programs. We developed and evaluated an automated screening tool to identify a subset of patients at high risk for CI. METHODS: In this secondary analysis of existing data collected for a randomized control trial, we developed machine-learning models to identify patients at higher risk of CI using only variables available in electronic health record (EHR). We used records from 1736 community-dwelling adults age > 59 being discharged from three EDs. Potential CI was determined based on the Blessed Orientation Memory Concentration (BOMC) test, administered in the ED. A nested cross-validation framework was used to evaluate machine-learning algorithms, comparing area under the receiver-operator curve (AUC) as the primary metric of performance. RESULTS: Based on BOMC scores, 121 of 1736 (7%) participants screened positive for potential CI at the time of their ED visit. The best performing algorithm, an XGBoost model, predicted BOMC positivity with an AUC of 0.72. With a classification threshold of 0.4, this model had a sensitivity of 0.73, a specificity of 0.64, a negative predictive value of 0.97, and a positive predictive value of 0.13. In a hypothetical ED with 200 older adult visits per week, the use of this model would lead to a decrease in the in-person screening burden from 200 to 77 individuals in order to detect 10 of 14 patients who would fail a BOMC. CONCLUSION: This study demonstrates that an algorithm based on EHR data can define a subset of patients at higher risk for CI. Incorporating such an algorithm into a screening workflow could allow screening efforts and resources to be focused where they have the most impact.
BACKGROUND/ OBJECTIVES: Despite a high prevalence and association with poor outcomes, screening to identify cognitive impairment (CI) in the emergency department (ED) is uncommon. Identification of high-risk subsets of older adults is a critical challenge to expanding screening programs. We developed and evaluated an automated screening tool to identify a subset of patients at high risk for CI. METHODS: In this secondary analysis of existing data collected for a randomized control trial, we developed machine-learning models to identify patients at higher risk of CI using only variables available in electronic health record (EHR). We used records from 1736 community-dwelling adults age > 59 being discharged from three EDs. Potential CI was determined based on the Blessed Orientation Memory Concentration (BOMC) test, administered in the ED. A nested cross-validation framework was used to evaluate machine-learning algorithms, comparing area under the receiver-operator curve (AUC) as the primary metric of performance. RESULTS: Based on BOMC scores, 121 of 1736 (7%) participants screened positive for potential CI at the time of their ED visit. The best performing algorithm, an XGBoost model, predicted BOMC positivity with an AUC of 0.72. With a classification threshold of 0.4, this model had a sensitivity of 0.73, a specificity of 0.64, a negative predictive value of 0.97, and a positive predictive value of 0.13. In a hypothetical ED with 200 older adult visits per week, the use of this model would lead to a decrease in the in-person screening burden from 200 to 77 individuals in order to detect 10 of 14 patients who would fail a BOMC. CONCLUSION: This study demonstrates that an algorithm based on EHR data can define a subset of patients at higher risk for CI. Incorporating such an algorithm into a screening workflow could allow screening efforts and resources to be focused where they have the most impact.
Authors: Jin H Han; Suzanne N Bryce; E Wesley Ely; Sunil Kripalani; Alessandro Morandi; Ayumi Shintani; James C Jackson; Alan B Storrow; Robert S Dittus; John Schnelle Journal: Ann Emerg Med Date: 2011-01-26 Impact factor: 5.721
Authors: Christopher R Carpenter; Elizabeth R Bassett; Grant M Fischer; Jonathan Shirshekan; James E Galvin; John C Morris Journal: Acad Emerg Med Date: 2011-04 Impact factor: 3.451
Authors: Deborah E Barnes; Jing Zhou; Rod L Walker; Eric B Larson; Sei J Lee; W John Boscardin; Zachary A Marcum; Sascha Dublin Journal: J Am Geriatr Soc Date: 2019-10-14 Impact factor: 5.562
Authors: David B Reuben; Andrew S Hackbarth; Neil S Wenger; Zaldy S Tan; Lee A Jennings Journal: J Am Geriatr Soc Date: 2017-02-02 Impact factor: 5.562
Authors: Lauren T Southerland; Alexander X Lo; Kevin Biese; Glenn Arendts; Jay Banerjee; Ula Hwang; Scott Dresden; Vivian Argento; Maura Kennedy; Christina L Shenvi; Christopher R Carpenter Journal: Ann Emerg Med Date: 2019-11-13 Impact factor: 5.721
Authors: Michael A LaMantia; Frank C Messina; Shola Jhanji; Arif Nazir; Mungai Maina; Siobhan McGuire; Cherri D Hobgood; Douglas K Miller Journal: Dementia (London) Date: 2016-07-26
Authors: Michael A LaMantia; Timothy E Stump; Frank C Messina; Douglas K Miller; Christopher M Callahan Journal: Alzheimer Dis Assoc Disord Date: 2016 Jan-Mar Impact factor: 2.703
Authors: Ritsuko Kakuma; Guillaume Galbaud du Fort; Louise Arsenault; Anne Perrault; Robert W Platt; Johanne Monette; Yola Moride; Christina Wolfson Journal: J Am Geriatr Soc Date: 2003-04 Impact factor: 5.562
Authors: Christopher R Carpenter; Nada Hammouda; Elizabeth A Linton; Michelle Doering; Ugochi K Ohuabunwa; Kelly J Ko; William W Hung; Manish N Shah; Lee A Lindquist; Kevin Biese; Daniel Wei; Libby Hoy; Lori Nerbonne; Ula Hwang; Scott M Dresden Journal: Acad Emerg Med Date: 2020-12-12 Impact factor: 5.221