Soumi Ray1,2, Dustin S McEvoy3, Skye Aaron1, Thu-Trang Hickman1, Adam Wright1,2,3. 1. Department of General Internal Medicine and Primary Care, Brigham & Women's Hospital, Boston, MA, USA. 2. Department of Medicine, Harvard Medical School, Boston, MA, USA. 3. Partners Healthcare, Information Systems, Somerville, MA, USA.
Abstract
Objective: Malfunctions in Clinical Decision Support (CDS) systems occur due to a multitude of reasons, and often go unnoticed, leading to potentially poor outcomes. Our goal was to identify malfunctions within CDS systems. Methods: We evaluated 6 anomaly detection models: (1) Poisson Changepoint Model, (2) Autoregressive Integrated Moving Average (ARIMA) Model, (3) Hierarchical Divisive Changepoint (HDC) Model, (4) Bayesian Changepoint Model, (5) Seasonal Hybrid Extreme Studentized Deviate (SHESD) Model, and (6) E-Divisive with Median (EDM) Model and characterized their ability to find known anomalies. We analyzed 4 CDS alerts with known malfunctions from the Longitudinal Medical Record (LMR) and Epic® (Epic Systems Corporation, Madison, WI, USA) at Brigham and Women's Hospital, Boston, MA. The 4 rules recommend lead testing in children, aspirin therapy in patients with coronary artery disease, pneumococcal vaccination in immunocompromised adults and thyroid testing in patients taking amiodarone. Results: Poisson changepoint, ARIMA, HDC, Bayesian changepoint and the SHESD model were able to detect anomalies in an alert for lead screening in children and in an alert for pneumococcal conjugate vaccine in immunocompromised adults. EDM was able to detect anomalies in an alert for monitoring thyroid function in patients on amiodarone. Conclusions: Malfunctions/anomalies occur frequently in CDS alert systems. It is important to be able to detect such anomalies promptly. Anomaly detection models are useful tools to aid such detections.
Objective: Malfunctions in Clinical Decision Support (CDS) systems occur due to a multitude of reasons, and often go unnoticed, leading to potentially poor outcomes. Our goal was to identify malfunctions within CDS systems. Methods: We evaluated 6 anomaly detection models: (1) Poisson Changepoint Model, (2) Autoregressive Integrated Moving Average (ARIMA) Model, (3) Hierarchical Divisive Changepoint (HDC) Model, (4) Bayesian Changepoint Model, (5) Seasonal Hybrid Extreme Studentized Deviate (SHESD) Model, and (6) E-Divisive with Median (EDM) Model and characterized their ability to find known anomalies. We analyzed 4 CDS alerts with known malfunctions from the Longitudinal Medical Record (LMR) and Epic® (Epic Systems Corporation, Madison, WI, USA) at Brigham and Women's Hospital, Boston, MA. The 4 rules recommend lead testing in children, aspirin therapy in patients with coronary artery disease, pneumococcal vaccination in immunocompromised adults and thyroid testing in patients taking amiodarone. Results: Poisson changepoint, ARIMA, HDC, Bayesian changepoint and the SHESD model were able to detect anomalies in an alert for lead screening in children and in an alert for pneumococcal conjugate vaccine in immunocompromised adults. EDM was able to detect anomalies in an alert for monitoring thyroid function in patients on amiodarone. Conclusions: Malfunctions/anomalies occur frequently in CDS alert systems. It is important to be able to detect such anomalies promptly. Anomaly detection models are useful tools to aid such detections.
Authors: Jerome A Osheroff; Jonathan M Teich; Blackford Middleton; Elaine B Steen; Adam Wright; Don E Detmer Journal: J Am Med Inform Assoc Date: 2007-01-09 Impact factor: 4.497
Authors: Allison B McCoy; Lemuel R Waitman; Julia B Lewis; Julie A Wright; David P Choma; Randolph A Miller; Josh F Peterson Journal: J Am Med Inform Assoc Date: 2011-08-17 Impact factor: 4.497
Authors: Tiffani J Bright; Anthony Wong; Ravi Dhurjati; Erin Bristow; Lori Bastian; Remy R Coeytaux; Gregory Samsa; Vic Hasselblad; John W Williams; Michael D Musty; Liz Wing; Amy S Kendrick; Gillian D Sanders; David Lobach Journal: Ann Intern Med Date: 2012-07-03 Impact factor: 25.391
Authors: Adam B Landman; Sukhjit S Takhar; Samuel L Wang; Anabela Cardoso; Joshua M Kosowsky; Ali S Raja; Ramin Khorasani; Eric G Poon Journal: J Am Med Inform Assoc Date: 2013-03-14 Impact factor: 4.497
Authors: Adam Wright; Thu-Trang T Hickman; Dustin McEvoy; Skye Aaron; Angela Ai; Jan Marie Andersen; Salman Hussain; Rachel Ramoni; Julie Fiskio; Dean F Sittig; David W Bates Journal: J Am Med Inform Assoc Date: 2016-03-28 Impact factor: 4.497
Authors: Dean F Sittig; Adam Wright; Enrico Coiera; Farah Magrabi; Raj Ratwani; David W Bates; Hardeep Singh Journal: Health Informatics J Date: 2018-12-11 Impact factor: 2.681
Authors: Amol A Verma; Sachin V Pasricha; Hae Young Jung; Vladyslav Kushnir; Denise Y F Mak; Radha Koppula; Yishan Guo; Janice L Kwan; Lauren Lapointe-Shaw; Shail Rawal; Terence Tang; Adina Weinerman; Fahad Razak Journal: J Am Med Inform Assoc Date: 2021-03-01 Impact factor: 4.497
Authors: Olufemi A Omitaomu; Ozgur Ozmen; Mohammed M Olama; Laura L Pullum; Teja Kuruganti; James Nutaro; Hilda B Klasky; Helia Zandi; Aneel Advani; Angela L Laurio; Merry Ward; Jeanie Scott; Jonathan R Nebeker Journal: Health Syst (Basingstoke) Date: 2019-05-07
Authors: Juan D Chaparro; Jonathan M Beus; Adam C Dziorny; Philip A Hagedorn; Sean Hernandez; Swaminathan Kandaswamy; Eric S Kirkendall; Allison B McCoy; Naveen Muthu; Evan W Orenstein Journal: Appl Clin Inform Date: 2022-05-25 Impact factor: 2.762
Authors: Vaishnavi Kannan; Mujeeb A Basit; Puneet Bajaj; Angela R Carrington; Irma B Donahue; Emily L Flahaven; Richard Medford; Tsedey Melaku; Brett A Moran; Luis E Saldana; Duwayne L Willett; Josh E Youngblood; Seth M Toomay Journal: J Am Med Inform Assoc Date: 2019-11-01 Impact factor: 4.497