Vaibhavi Shah1, Yeshwant R Chillakuru2, Alex Rybkin3, Youngho Seo3, Thienkhai Vu3, Jae Ho Sohn4. 1. Radiology and Biomedical Imaging, University of California, San Francisco (UCSF), San Francisco, California 94143; Department of Biological Engineering, Department of Science, Technology, and Society , Massachusetts Institute of Technology (MIT), Cambridge, MA 02139. 2. Radiology and Biomedical Imaging, University of California, San Francisco (UCSF), San Francisco, California 94143; The George Washington University, School of Medicine and Health Sciences, Washington, DC 20037. 3. Radiology and Biomedical Imaging, University of California, San Francisco (UCSF), San Francisco, California 94143. 4. Radiology and Biomedical Imaging, University of California, San Francisco (UCSF), San Francisco, California 94143. Electronic address: sohn87@gmail.com.
Abstract
RATIONALE AND OBJECTIVES: Radiology turnaround time is an important quality measure that can impact hospital workflow and patient outcomes. We aimed to develop a machine learning model to predict delayed turnaround time during non-business hours and identify factors that contribute to this delay. MATERIALS AND METHODS: This retrospective study consisted of 15,117 CT cases from May 2018 to May 2019 during non-business hours at two hospital campuses after applying exclusion criteria. Of these 15,177 cases, 7,532 were inpatient cases and 7,585 were emergency cases. Order time, scan time, first communication by radiologist, free-text indications, and other clinical metadata were extracted. A combined XGBoost classifier and Random Forest natural language processing model was trained with 85% of the data and tested with 15% of the data. The model predicted two measures of delay: when the exam was ordered to first communication (total time) and when the scan was completed to first communication (interpretation time). The model was analyzed with the area under the curve (AUC) of receiver operating characteristic (ROC) and feature importance. Source code: https://bit.ly/2UrLiVJ RESULTS: The algorithm reached an AUC of 0.85, with a 95% confidence interval [0.83, 0.87], when predicting delays greater than 245 minutes for "total time" and 0.71, with a 95% confidence interval [0.68, 0.73], when predicting delays greater than 57 minutes for "interpretation time". At our institution, CT scan description (e.g. "CTA chest pulmonary embolism protocol"), time of day, and year in training were more predictive features compared to body part, inpatient status, and hospital campus for both interpretation and total time delay. CONCLUSION: This algorithm can be applied clinically when a physician is ordering the scan to reasonably predict delayed turnaround time. Such a model can be leveraged to identify factors associated with delays and emphasize areas for improvement to patient outcomes.
RATIONALE AND OBJECTIVES: Radiology turnaround time is an important quality measure that can impact hospital workflow and patient outcomes. We aimed to develop a machine learning model to predict delayed turnaround time during non-business hours and identify factors that contribute to this delay. MATERIALS AND METHODS: This retrospective study consisted of 15,117 CT cases from May 2018 to May 2019 during non-business hours at two hospital campuses after applying exclusion criteria. Of these 15,177 cases, 7,532 were inpatient cases and 7,585 were emergency cases. Order time, scan time, first communication by radiologist, free-text indications, and other clinical metadata were extracted. A combined XGBoost classifier and Random Forest natural language processing model was trained with 85% of the data and tested with 15% of the data. The model predicted two measures of delay: when the exam was ordered to first communication (total time) and when the scan was completed to first communication (interpretation time). The model was analyzed with the area under the curve (AUC) of receiver operating characteristic (ROC) and feature importance. Source code: https://bit.ly/2UrLiVJ RESULTS: The algorithm reached an AUC of 0.85, with a 95% confidence interval [0.83, 0.87], when predicting delays greater than 245 minutes for "total time" and 0.71, with a 95% confidence interval [0.68, 0.73], when predicting delays greater than 57 minutes for "interpretation time". At our institution, CT scan description (e.g. "CTA chest pulmonary embolism protocol"), time of day, and year in training were more predictive features compared to body part, inpatient status, and hospital campus for both interpretation and total time delay. CONCLUSION: This algorithm can be applied clinically when a physician is ordering the scan to reasonably predict delayed turnaround time. Such a model can be leveraged to identify factors associated with delays and emphasize areas for improvement to patient outcomes.