Sunny S Lou1, Hanyang Liu2, Benjamin C Warner2, Derek Harford1, Chenyang Lu2, Thomas Kannampallil3. 1. Department of Anesthesiology, School of Medicine, Washington University in St Louis, St Louis, MO, United States. 2. Department of Computer Science, McKelvey School of Engineering, Washington University in St Louis, St Louis, MO, United States. 3. Department of Anesthesiology, School of Medicine, Washington University in St Louis, St Louis, MO, United States; Institute for Informatics, School of Medicine, Washington University in St Louis, St Louis, MO, United States. Electronic address: thomas.k@wustl.edu.
Abstract
BACKGROUND: Burnout is a significant public health concern affecting more than half of the healthcare workforce; however, passive screening tools to detect burnout are lacking. We investigated the ability of machine learning (ML) techniques to identify burnout using passively collected electronic health record (EHR)-based audit log data. METHOD: Physician trainees participated in a longitudinal study where they completed monthly burnout surveys and provided access to their EHR-based audit logs. Using the monthly burnout scores as the target outcome, we trained ML models using combinations of features derived from audit log data-aggregate measures of clinical workload, time series-based temporal measures of EHR use, and the baseline burnout score. Five ML models were constructed to predict burnout as a continuous score: penalized linear regression, support vector machine, neural network, random forest, and gradient boosting machine. RESULTS: 88 trainee physicians participated and completed 416 surveys; greater than10 million audit log actions were collected (Mean [Standard Deviation] = 25,691 [14,331] actions per month, per physician). The workload feature set predicted burnout score with a mean absolute error (MAE) of 0.602 (95% Confidence Interval (CI), 0.412-0.826), and was able to predict burnout status with an average AUROC of 0.595 (95% CI 0.355-0.808) and average accuracy 0.567 (95% CI 0.393-0.742). The temporal feature set had a similar performance, with MAE 0.596 (95% CI 0.391-0.826), and AUROC 0.581 (95% CI 0.343-0.790). The addition of the baseline burnout score to the workload features improved the model performance to a mean AUROC of 0.829 (95% CI 0.607-0.996) and mean accuracy of 0.781 (95% CI 0.587-0.936); however, this performance was not meaningfully different than using the baseline burnout score alone. CONCLUSIONS: Current findings illustrate the complexities of predicting burnout exclusively based on clinical work activities as captured in the EHR, highlighting its multi-factorial and individualized nature. Future prediction studies of burnout should account for individual factors (e.g., resilience, physiological measurements such as sleep) and associated system-level factors (e.g., leadership).
BACKGROUND: Burnout is a significant public health concern affecting more than half of the healthcare workforce; however, passive screening tools to detect burnout are lacking. We investigated the ability of machine learning (ML) techniques to identify burnout using passively collected electronic health record (EHR)-based audit log data. METHOD: Physician trainees participated in a longitudinal study where they completed monthly burnout surveys and provided access to their EHR-based audit logs. Using the monthly burnout scores as the target outcome, we trained ML models using combinations of features derived from audit log data-aggregate measures of clinical workload, time series-based temporal measures of EHR use, and the baseline burnout score. Five ML models were constructed to predict burnout as a continuous score: penalized linear regression, support vector machine, neural network, random forest, and gradient boosting machine. RESULTS: 88 trainee physicians participated and completed 416 surveys; greater than10 million audit log actions were collected (Mean [Standard Deviation] = 25,691 [14,331] actions per month, per physician). The workload feature set predicted burnout score with a mean absolute error (MAE) of 0.602 (95% Confidence Interval (CI), 0.412-0.826), and was able to predict burnout status with an average AUROC of 0.595 (95% CI 0.355-0.808) and average accuracy 0.567 (95% CI 0.393-0.742). The temporal feature set had a similar performance, with MAE 0.596 (95% CI 0.391-0.826), and AUROC 0.581 (95% CI 0.343-0.790). The addition of the baseline burnout score to the workload features improved the model performance to a mean AUROC of 0.829 (95% CI 0.607-0.996) and mean accuracy of 0.781 (95% CI 0.587-0.936); however, this performance was not meaningfully different than using the baseline burnout score alone. CONCLUSIONS: Current findings illustrate the complexities of predicting burnout exclusively based on clinical work activities as captured in the EHR, highlighting its multi-factorial and individualized nature. Future prediction studies of burnout should account for individual factors (e.g., resilience, physiological measurements such as sleep) and associated system-level factors (e.g., leadership).
Authors: Tait D Shanafelt; Omar Hasan; Lotte N Dyrbye; Christine Sinsky; Daniel Satele; Jeff Sloan; Colin P West Journal: Mayo Clin Proc Date: 2015-12 Impact factor: 7.616
Authors: Tait D Shanafelt; Grace Gorringe; Ronald Menaker; Kristin A Storz; David Reeves; Steven J Buskirk; Jeff A Sloan; Stephen J Swensen Journal: Mayo Clin Proc Date: 2015-03-18 Impact factor: 7.616
Authors: Skyler Place; Danielle Blanch-Hartigan; Channah Rubin; Cristina Gorrostieta; Caroline Mead; John Kane; Brian P Marx; Joshua Feast; Thilo Deckersbach; Alex Sandy Pentland; Andrew Nierenberg; Ali Azarbayejani Journal: J Med Internet Res Date: 2017-03-16 Impact factor: 5.428
Authors: Eugenia McPeek-Hinz; Mina Boazak; J Bryan Sexton; Kathryn C Adair; Vivian West; Benjamin A Goldstein; Robert S Alphin; Sherif Idris; W Ed Hammond; Shelley E Hwang; Jonathan Bae Journal: JAMA Netw Open Date: 2021-04-01
Authors: Sunny S Lou; Seunghwan Kim; Derek Harford; Benjamin C Warner; Philip R O Payne; Joanna Abraham; Thomas Kannampallil Journal: Br J Anaesth Date: 2022-05-12 Impact factor: 11.719