Kai Zheng1, Rema Padman, Michael P Johnson, Herbert S Diamond. 1. H. John Heinz III School of Public Policy and Management, Carnegie Mellon University, Department of Medicine, Pittsburgh, PA 15213-3890, USA. kzheng@andrew.cmu.edu
Abstract
BACKGROUND: Evaluation studies of clinical decision support systems (CDSS) have tended to focus on assessments of system quality and clinical performance in a laboratory setting. Relatively few studies have used field trials to determine if CDSS are likely to be used in routine clinical settings and whether reminders generated are likely to be acted upon by end-users. Moreover, such studies when performed tend not to identify distinct user groups, nor to classify user feedback. AIM: To assess medical residents' acceptance and adoption of a clinical reminder system for chronic disease and preventive care management and to use expressed preferences for system attributes and functionality as a basis for system re-engineering. DESIGN OF STUDY: Longitudinal, correlational study using a novel developmental trajectory analysis (DTA) statistical method, followed by a qualitative analysis based on user satisfaction surveys and field interviews. SETTING: An ambulatory primary care clinic of an urban teaching hospital offering comprehensive healthcare services. 41 medical residents used a CDSS over 10 months in their daily practice. Use of this system was strongly recommended but not mandatory. METHODS: A group-based, semi-parametric statistical modeling method to identify distinct groups, with distinct usage trajectories, followed by qualitative instruments of usability and satisfaction surveys and structured interviews to validate insights derived from usage trajectories. RESULTS: Quantitative analysis delineates three types of user adoption behavior: "light", "moderate" and "heavy" usage. Qualitative analysis reveals that clinicians of distinct types tend to exhibit views of the system consistent with their demonstrated adoption behavior. Drawbacks in the design of the CDSS identified by users of all types (in different ways) motivate a redesign based on current physician workflows. CONCLUSION: We conclude that this mixed methodology has considerable promise to provide new insights into system usability and adoption issues that may benefit clinical decision support systems as well as information systems more generally.
BACKGROUND: Evaluation studies of clinical decision support systems (CDSS) have tended to focus on assessments of system quality and clinical performance in a laboratory setting. Relatively few studies have used field trials to determine if CDSS are likely to be used in routine clinical settings and whether reminders generated are likely to be acted upon by end-users. Moreover, such studies when performed tend not to identify distinct user groups, nor to classify user feedback. AIM: To assess medical residents' acceptance and adoption of a clinical reminder system for chronic disease and preventive care management and to use expressed preferences for system attributes and functionality as a basis for system re-engineering. DESIGN OF STUDY: Longitudinal, correlational study using a novel developmental trajectory analysis (DTA) statistical method, followed by a qualitative analysis based on user satisfaction surveys and field interviews. SETTING: An ambulatory primary care clinic of an urban teaching hospital offering comprehensive healthcare services. 41 medical residents used a CDSS over 10 months in their daily practice. Use of this system was strongly recommended but not mandatory. METHODS: A group-based, semi-parametric statistical modeling method to identify distinct groups, with distinct usage trajectories, followed by qualitative instruments of usability and satisfaction surveys and structured interviews to validate insights derived from usage trajectories. RESULTS: Quantitative analysis delineates three types of user adoption behavior: "light", "moderate" and "heavy" usage. Qualitative analysis reveals that clinicians of distinct types tend to exhibit views of the system consistent with their demonstrated adoption behavior. Drawbacks in the design of the CDSS identified by users of all types (in different ways) motivate a redesign based on current physician workflows. CONCLUSION: We conclude that this mixed methodology has considerable promise to provide new insights into system usability and adoption issues that may benefit clinical decision support systems as well as information systems more generally.
Authors: Marie-Pierre Gagnon; Marie Desmartis; Michel Labrecque; Josip Car; Claudia Pagliari; Pierre Pluye; Pierre Frémont; Johanne Gagnon; Nadine Tremblay; France Légaré Journal: J Med Syst Date: 2010-03-30 Impact factor: 4.460
Authors: Ana Lucia Hincapie; Terri L Warholak; Anita C Murcko; Marion Slack; Daniel C Malone Journal: J Am Med Inform Assoc Date: 2010-11-24 Impact factor: 4.497
Authors: Mary Regina Boland; Alexander Rusanov; Yat So; Carlos Lopez-Jimenez; Linda Busacca; Richard C Steinman; Suzanne Bakken; J Thomas Bigger; Chunhua Weng Journal: J Biomed Inform Date: 2013-12-12 Impact factor: 6.317
Authors: Constance H Fung; Jerry S Tsai; Armine Lulejian; Peter Glassman; Emily Patterson; Brad N Doebbeling; Steven M Asch Journal: J Gen Intern Med Date: 2008-04 Impact factor: 5.128