Andrew J King1, Gregory F Cooper2, Gilles Clermont3, Harry Hochheiser2, Milos Hauskrecht4, Dean F Sittig5, Shyam Visweswaran6. 1. Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA; Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, PA, USA. 2. Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA; Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA. 3. Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, PA, USA. 4. Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA; Department of Computer Science, University of Pittsburgh, Pittsburgh, PA, USA. 5. Department of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX, USA. 6. Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA, USA; Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA, USA. Electronic address: shv3@pitt.edu.
Abstract
BACKGROUND: Electronic medical record (EMR) systems need functionality that decreases cognitive overload by drawing the clinician's attention to the right data, at the right time. We developed a Learning EMR (LEMR) system that learns statistical models of clinician information-seeking behavior and applies those models to direct the display of data in future patients. We evaluated the performance of the system in identifying relevant patient data in intensive care unit (ICU) patient cases. METHODS: To capture information-seeking behavior, we enlisted critical care medicine physicians who reviewed a set of patient cases and selected data items relevant to the task of presenting at morning rounds. Using patient EMR data as predictors, we built machine learning models to predict their relevancy. We prospectively evaluated the predictions of a set of high performing models. RESULTS: On an independent evaluation data set, 25 models achieved precision of 0.52, 95% CI [0.49, 0.54] and recall of 0.77, 95% CI [0.75, 0.80] in identifying relevant patient data items. For data items missed by the system, the reviewers rated the effect of not seeing those data from no impact to minor impact on patient care in about 82% of the cases. CONCLUSION: Data-driven approaches for adaptively displaying data in EMR systems, like the LEMR system, show promise in using information-seeking behavior of clinicians to identify and highlight relevant patient data.
BACKGROUND: Electronic medical record (EMR) systems need functionality that decreases cognitive overload by drawing the clinician's attention to the right data, at the right time. We developed a Learning EMR (LEMR) system that learns statistical models of clinician information-seeking behavior and applies those models to direct the display of data in future patients. We evaluated the performance of the system in identifying relevant patient data in intensive care unit (ICU) patient cases. METHODS: To capture information-seeking behavior, we enlisted critical care medicine physicians who reviewed a set of patient cases and selected data items relevant to the task of presenting at morning rounds. Using patient EMR data as predictors, we built machine learning models to predict their relevancy. We prospectively evaluated the predictions of a set of high performing models. RESULTS: On an independent evaluation data set, 25 models achieved precision of 0.52, 95% CI [0.49, 0.54] and recall of 0.77, 95% CI [0.75, 0.80] in identifying relevant patient data items. For data items missed by the system, the reviewers rated the effect of not seeing those data from no impact to minor impact on patient care in about 82% of the cases. CONCLUSION: Data-driven approaches for adaptively displaying data in EMR systems, like the LEMR system, show promise in using information-seeking behavior of clinicians to identify and highlight relevant patient data.
Authors: Shyam Visweswaran; Brian McLay; Nickie Cappella; Michele Morris; John T Milnes; Steven E Reis; Jonathan C Silverstein; Michael J Becich Journal: J Am Med Inform Assoc Date: 2022-03-15 Impact factor: 4.497
Authors: Andrew J King; Gregory F Cooper; Gilles Clermont; Harry Hochheiser; Milos Hauskrecht; Dean F Sittig; Shyam Visweswaran Journal: J Med Internet Res Date: 2020-04-02 Impact factor: 5.428
Authors: Mohammadamin Tajgardoon; Gregory F Cooper; Andrew J King; Gilles Clermont; Harry Hochheiser; Milos Hauskrecht; Dean F Sittig; Shyam Visweswaran Journal: JAMIA Open Date: 2020-12-31
Authors: Shyam Visweswaran; Andrew J King; Mohammadamin Tajgardoon; Luca Calzoni; Gilles Clermont; Harry Hochheiser; Gregory F Cooper Journal: JAMIA Open Date: 2021-08-02