I Cho1, J Lee2, H Han3, S Phansalkar4, D W Bates5. 1. Department of Nursing, School of Medicine, Inha University , Incheon, Korea ; Division of General Internal Medicine, Brigham and Women's Hospital , Boston, MA, USA ; Harvard Medical School , Boston, MA, USA. 2. Division of General Internal Medicine, Brigham and Women's Hospital , Boston, MA, USA ; Harvard Medical School , Boston, MA, USA ; Department of Emergency Medicine, University of Ulsan College of Medicine, Asan Medical Center , Seoul, Korea ; Department of Biomedical Informatics, Asan Medical Center , Seoul, Korea. 3. Department of Pharmacy, Asan Medical Center , Seoul, Korea. 4. Division of General Internal Medicine, Brigham and Women's Hospital , Boston, MA, USA ; Harvard Medical School , Boston, MA, USA ; Partners Healthcare Systems , Wellesley, MA, USA ; Wolters Kluwer Health , Indianapolis, IN, USA. 5. Division of General Internal Medicine, Brigham and Women's Hospital , Boston, MA, USA ; Harvard Medical School , Boston, MA, USA ; Partners Healthcare Systems , Wellesley, MA, USA.
Abstract
OBJECTIVE: The Instrument for Evaluating Human-Factor Principles in Medication-Related Decision Support Alerts (I-MeDeSA) was developed recently in the US with a view towards improving considerations of human-factor principles when designing alerts for clinical decision support (CDS) systems. This study evaluated the generalizability of this tool, in cooperation with its authors, across cultures by applying it to a Korean system. We also examined opportunities to promote user acceptance of the system. METHODS: We developed a Korean version of the I-MeDeSA (K-I-MeDeSA) and used it to evaluate drug-drug interaction alerts in a large academic tertiary hospital in Seoul. We involved four reviewers (A, B, C, and D). Two (A and B) conducted the initial independent scoring, while the other two (C and D) performed a final review and assessed feedback from the initial reviewers. The obtained scores were compared with those from 13 previously reported CDS systems. The feedback was summarized qualitatively. RESULTS: The translation of the I-MeDeSA had excellent interrater agreement in terms of face validity (scale-level content validity index = 0.95). The system's K-I-MeDeSA score was 10 out of 26, with a good agreement between reviewers (κ = 0.77), which showed a lack of human-factor considerations. The reviewers readily identified two of the nine principles that needed primary improvement: prioritization and text-based information. The reviewers also expressed difficulty judging the following four principles: alarm philosophy, visibility, color, and learnability and confusability. CONCLUSION: The K-I-MeDeSA was semantically and operationally equivalent to the original tool. Only minor cultural problems were identified, leading the reviewers to suggest the need for clarification of certain words plus a more detailed description of the tool's rationale and exemplars. Further evaluation is needed to empirically assess whether the implementation of changes in an electronic health record system could improve the adoption of CDS alerts.
OBJECTIVE: The Instrument for Evaluating Human-Factor Principles in Medication-Related Decision Support Alerts (I-MeDeSA) was developed recently in the US with a view towards improving considerations of human-factor principles when designing alerts for clinical decision support (CDS) systems. This study evaluated the generalizability of this tool, in cooperation with its authors, across cultures by applying it to a Korean system. We also examined opportunities to promote user acceptance of the system. METHODS: We developed a Korean version of the I-MeDeSA (K-I-MeDeSA) and used it to evaluate drug-drug interaction alerts in a large academic tertiary hospital in Seoul. We involved four reviewers (A, B, C, and D). Two (A and B) conducted the initial independent scoring, while the other two (C and D) performed a final review and assessed feedback from the initial reviewers. The obtained scores were compared with those from 13 previously reported CDS systems. The feedback was summarized qualitatively. RESULTS: The translation of the I-MeDeSA had excellent interrater agreement in terms of face validity (scale-level content validity index = 0.95). The system's K-I-MeDeSA score was 10 out of 26, with a good agreement between reviewers (κ = 0.77), which showed a lack of human-factor considerations. The reviewers readily identified two of the nine principles that needed primary improvement: prioritization and text-based information. The reviewers also expressed difficulty judging the following four principles: alarm philosophy, visibility, color, and learnability and confusability. CONCLUSION: The K-I-MeDeSA was semantically and operationally equivalent to the original tool. Only minor cultural problems were identified, leading the reviewers to suggest the need for clarification of certain words plus a more detailed description of the tool's rationale and exemplars. Further evaluation is needed to empirically assess whether the implementation of changes in an electronic health record system could improve the adoption of CDS alerts.
Entities:
Keywords:
Clinical decision support; alert fatigue; computerized order entry; human-computer interaction; interfaces and usability
Authors: Nidhi R Shah; Andrew C Seger; Diane L Seger; Julie M Fiskio; Gilad J Kuperman; Barry Blumenfeld; Elaine G Recklet; David W Bates; Tejal K Gandhi Journal: J Am Med Inform Assoc Date: 2005-10-12 Impact factor: 4.497
Authors: Joan S Ash; Dean F Sittig; Richard H Dykstra; Kenneth Guappone; James D Carpenter; Veena Seshadri Journal: Int J Med Inform Date: 2006-06-21 Impact factor: 4.046
Authors: Basit Chaudhry; Jerome Wang; Shinyi Wu; Margaret Maglione; Walter Mojica; Elizabeth Roth; Sally C Morton; Paul G Shekelle Journal: Ann Intern Med Date: 2006-04-11 Impact factor: 25.391
Authors: William M Tierney; J Marc Overhage; Michael D Murray; Lisa E Harris; Xiao-Hua Zhou; George J Eckert; Faye E Smith; Nancy Nienaber; Clement J McDonald; Fredric D Wolinsky Journal: J Gen Intern Med Date: 2003-12 Impact factor: 5.128