Jiyoun Song1, Mollie Hobensack2, Kathryn H Bowles3, Margaret V McDonald4, Kenrick Cato5, Sarah Collins Rossetti6, Sena Chae7, Erin Kennedy8, Yolanda Barrón9, Sridevi Sridharan10, Maxim Topaz11. 1. Columbia University School of Nursing, New York City, NY, USA; Center for Home Care Policy & Research, Visiting Nurse Service of New York, New York City, NY, USA. Electronic address: js4753@cumc.columbia.edu. 2. Columbia University School of Nursing, New York City, NY, USA. Electronic address: mxh2000@cumc.columbia.edu. 3. Center for Home Care Policy & Research, Visiting Nurse Service of New York, New York City, NY, USA; University of Pennsylvania School of Nursing, Department of Biobehavioral Health Sciences, Philadelphia, PA, USA. Electronic address: bowles@nursing.upenn.edu. 4. Center for Home Care Policy & Research, Visiting Nurse Service of New York, New York City, NY, USA. Electronic address: margaret.mcdonald@vnsny.org. 5. Columbia University School of Nursing, New York City, NY, USA; Emergency Medicine, Columbia University Irving Medical Center, New York, NY, USA. Electronic address: kdc2110@cumc.columbia.edu. 6. Columbia University School of Nursing, New York City, NY, USA; Columbia University, Department of Biomedical Informatics, New York City, NY, USA. Electronic address: sac2125@cumc.columbia.edu. 7. College of Nursing, University of Iowa, Iowa City, IA, USA. Electronic address: sena-chae@uiowa.edu. 8. University of Pennsylvania School of Nursing, Department of Biobehavioral Health Sciences, Philadelphia, PA, USA. Electronic address: erinken@nursing.upenn.edu. 9. Center for Home Care Policy & Research, Visiting Nurse Service of New York, New York City, NY, USA. Electronic address: yolanda.barron@vnsny.org. 10. Center for Home Care Policy & Research, Visiting Nurse Service of New York, New York City, NY, USA. Electronic address: sridevi.sridharan@vnsny.org. 11. Columbia University School of Nursing, New York City, NY, USA; Center for Home Care Policy & Research, Visiting Nurse Service of New York, New York City, NY, USA; Data Science Institute, Columbia University, New York City, NY, USA. Electronic address: mt3315@cumc.columbia.edu.
Abstract
BACKGROUND/ OBJECTIVE: Between 10 and 25% patients are hospitalized or visit emergency department (ED) during home healthcare (HHC). Given that up to 40% of these negative clinical outcomes are preventable, early and accurate prediction of hospitalization risk can be one strategy to prevent them. In recent years, machine learning-based predictive modeling has become widely used for building risk models. This study aimed to compare the predictive performance of four risk models built with various data sources for hospitalization and ED visits in HHC. METHODS: Four risk models were built using different variables from two data sources: structured data (i.e., Outcome and Assessment Information Set (OASIS) and other assessment items from the electronic health record (EHR)) and unstructured narrative-free text clinical notes for patients who received HHC services from the largest non-profit HHC organization in New York between 2015 and 2017. Then, five machine learning algorithms (logistic regression, Random Forest, Bayesian network, support vector machine (SVM), and Naïve Bayes) were used on each risk model. Risk model performance was evaluated using the F-score and Precision-Recall Curve (PRC) area metrics. RESULTS: During the study period, 8373/86,823 (9.6%) HHC episodes resulted in hospitalization or ED visits. Among five machine learning algorithms on each model, the SVM showed the highest F-score (0.82), while the Random Forest showed the highest PRC area (0.864). Adding information extracted from clinical notes significantly improved the risk prediction ability by up to 16.6% in F-score and 17.8% in PRC. CONCLUSION: All models showed relatively good hospitalization or ED visit risk predictive performance in HHC. Information from clinical notes integrated with the structured data improved the ability to identify patients at risk for these emergent care events.
BACKGROUND/ OBJECTIVE: Between 10 and 25% patients are hospitalized or visit emergency department (ED) during home healthcare (HHC). Given that up to 40% of these negative clinical outcomes are preventable, early and accurate prediction of hospitalization risk can be one strategy to prevent them. In recent years, machine learning-based predictive modeling has become widely used for building risk models. This study aimed to compare the predictive performance of four risk models built with various data sources for hospitalization and ED visits in HHC. METHODS: Four risk models were built using different variables from two data sources: structured data (i.e., Outcome and Assessment Information Set (OASIS) and other assessment items from the electronic health record (EHR)) and unstructured narrative-free text clinical notes for patients who received HHC services from the largest non-profit HHC organization in New York between 2015 and 2017. Then, five machine learning algorithms (logistic regression, Random Forest, Bayesian network, support vector machine (SVM), and Naïve Bayes) were used on each risk model. Risk model performance was evaluated using the F-score and Precision-Recall Curve (PRC) area metrics. RESULTS: During the study period, 8373/86,823 (9.6%) HHC episodes resulted in hospitalization or ED visits. Among five machine learning algorithms on each model, the SVM showed the highest F-score (0.82), while the Random Forest showed the highest PRC area (0.864). Adding information extracted from clinical notes significantly improved the risk prediction ability by up to 16.6% in F-score and 17.8% in PRC. CONCLUSION: All models showed relatively good hospitalization or ED visit risk predictive performance in HHC. Information from clinical notes integrated with the structured data improved the ability to identify patients at risk for these emergent care events.
Authors: Nicholas K Schiltz; Megan A Foradori; Andrew P Reimer; Matthew Plow; Mary A Dolansky Journal: J Am Geriatr Soc Date: 2022-04-05 Impact factor: 7.538