Ainhoa Yera1, Javier Muguerza1, Olatz Arbelaitz1, Iñigo Perona1, Richard N Keers2, Darren M Ashcroft2, Richard Williams3, Niels Peek3, Caroline Jay4, Markel Vigo5. 1. Faculty of Informatics, University of the Basque Country UPV/EHU, Donostia/San Sebastián, Spain. 2. Division of Pharmacy and Optometry, University of Manchester, Manchester, United Kingdom; NIHR Greater Manchester Patient Safety Translational Research Centre, University of Manchester, Manchester Academic Health Science Centre, Manchester, United Kingdom. 3. Division of Informatics, Imaging and Data Sciences, University of Manchester, Manchester Academic Health Science Centre, Manchester, United Kingdom; NIHR Greater Manchester Patient Safety Translational Research Centre, University of Manchester, Manchester Academic Health Science Centre, Manchester, United Kingdom. 4. School of Computer Science, University of Manchester, Manchester, United Kingdom. 5. School of Computer Science, University of Manchester, Manchester, United Kingdom. Electronic address: markel.vigo@manchester.ac.uk.
Abstract
OBJECTIVE: To characterise the use of an electronic medication safety dashboard by exploring and contrasting interactions from primary users (i.e. pharmacists) who were leading the intervention and secondary users (i.e. non-pharmacist staff) who used the dashboard to engage in safe prescribing practices. MATERIALS AND METHODS: We conducted a 10-month observational study in which 35 health professionals used an instrumented medication safety dashboard for audit and feedback purposes in clinical practice as part of a wider intervention study. We modelled user interaction by computing features representing exploration and dwell time through user interface events that were logged on a remote database. We applied supervised learning algorithms to classify primary against secondary users. RESULTS: We observed values for accuracy above 0.8, indicating that 80% of the time we were able to distinguish a primary user from a secondary user. In particular, the Multilayer Perceptron (MLP) yielded the highest values of precision (0.88), recall (0.86) and F-measure (0.86). The behaviour of primary users was distinctive in that they spent less time between mouse clicks (lower dwell time) on the screens showing the overview of the practice and trends. Secondary users exhibited a higher dwell time and more visual search activity (higher exploration) on the screens displaying patients at risk and visualisations. DISCUSSION AND CONCLUSION: We were able to distinguish the interactive behaviour of primary and secondary users of a medication safety dashboard in primary care using timestamped mouse events. Primary users were more competent on population health monitoring activities, while secondary users struggled on activities involving a detailed breakdown of the safety of patients. Informed by these findings, we propose workflows that group these activities and adaptive nudges to increase user engagement.
OBJECTIVE: To characterise the use of an electronic medication safety dashboard by exploring and contrasting interactions from primary users (i.e. pharmacists) who were leading the intervention and secondary users (i.e. non-pharmacist staff) who used the dashboard to engage in safe prescribing practices. MATERIALS AND METHODS: We conducted a 10-month observational study in which 35 health professionals used an instrumented medication safety dashboard for audit and feedback purposes in clinical practice as part of a wider intervention study. We modelled user interaction by computing features representing exploration and dwell time through user interface events that were logged on a remote database. We applied supervised learning algorithms to classify primary against secondary users. RESULTS: We observed values for accuracy above 0.8, indicating that 80% of the time we were able to distinguish a primary user from a secondary user. In particular, the Multilayer Perceptron (MLP) yielded the highest values of precision (0.88), recall (0.86) and F-measure (0.86). The behaviour of primary users was distinctive in that they spent less time between mouse clicks (lower dwell time) on the screens showing the overview of the practice and trends. Secondary users exhibited a higher dwell time and more visual search activity (higher exploration) on the screens displaying patients at risk and visualisations. DISCUSSION AND CONCLUSION: We were able to distinguish the interactive behaviour of primary and secondary users of a medication safety dashboard in primary care using timestamped mouse events. Primary users were more competent on population health monitoring activities, while secondary users struggled on activities involving a detailed breakdown of the safety of patients. Informed by these findings, we propose workflows that group these activities and adaptive nudges to increase user engagement.
Authors: Nicholas A Rattray; Teresa M Damush; Edward J Miech; Barbara Homoya; Laura J Myers; Lauren S Penney; Jared Ferguson; Brenna Giacherio; Meetesh Kumar; Dawn M Bravata Journal: J Gen Intern Med Date: 2020-09-01 Impact factor: 5.128