BACKGROUND AND OBJECTIVES: Medical errors are a leading cause of death in the United States. Despite widespread adoption of patient safety reporting systems to address medical errors, making sense of the reports collected in these systems is challenging in practice. Event classification taxonomies used in many reporting systems can be complex and difficult to understand by frontline reporters, leading reporters to classify reports as "miscellaneous" as opposed to assigning a specific event-type category, which may facilitate analysis. METHODS: To assist patient safety analysts in their analysis of "miscellaneous" reports, we developed an ensemble machine learning natural language processing model to reclassify these reports. We integrated the model into a clinical workflow dashboard, evaluated user feedback, and compared differences in user thresholds for model performance. RESULTS AND CONCLUSIONS: Integrating an ensemble model to classify "miscellaneous" event reports with an interactive visualization was helpful to patient safety analysts review "miscellaneous" reports. However, patient safety analysts have different thresholds for model reclassification depending on their role and experience with "miscellaneous" event reports.
BACKGROUND AND OBJECTIVES: Medical errors are a leading cause of death in the United States. Despite widespread adoption of patient safety reporting systems to address medical errors, making sense of the reports collected in these systems is challenging in practice. Event classification taxonomies used in many reporting systems can be complex and difficult to understand by frontline reporters, leading reporters to classify reports as "miscellaneous" as opposed to assigning a specific event-type category, which may facilitate analysis. METHODS: To assist patient safety analysts in their analysis of "miscellaneous" reports, we developed an ensemble machine learning natural language processing model to reclassify these reports. We integrated the model into a clinical workflow dashboard, evaluated user feedback, and compared differences in user thresholds for model performance. RESULTS AND CONCLUSIONS: Integrating an ensemble model to classify "miscellaneous" event reports with an interactive visualization was helpful to patient safety analysts review "miscellaneous" reports. However, patient safety analysts have different thresholds for model reclassification depending on their role and experience with "miscellaneous" event reports.