Kellen R Maicher1, Laura Zimmerman2, Bruce Wilcox3, Beth Liston4, Holly Cronau5, Allison Macerollo5, Lifeng Jin6, Evan Jaffe6, Michael White7, Eric Fosler-Lussier6,7, William Schuler6, David P Way8, Douglas R Danforth2. 1. The James Cancer Hospital and Solove Research Institute, The Ohio State University Wexner Medical Center , Columbus , OH , USA. 2. The Department of Obstetrics and Gynecology, The Ohio State University , Columbus , OH , USA. 3. Brillig Understanding, Inc. , Atascadero , CA , USA. 4. Hospital Medicine, The Ohio State University , Columbus , OH , USA. 5. Family Medicine, The Ohio State University , Columbus , OH , USA. 6. The Department of Linguistics, The Ohio State University , Columbus , OH , USA. 7. The Department of Computer Science and Engineering, The Ohio State University , Columbus , OH , USA. 8. Emergency Medicine, College of Medicine, The Ohio State University , Columbus , OH , USA.
Abstract
Introduction: Practicing a medical history using standardized patients is an essential component of medical school curricula. Recent advances in technology now allow for newer approaches for practicing and assessing communication skills. We describe herein a virtual standardized patient (VSP) system that allows students to practice their history taking skills and receive immediate feedback. Methods: Our VSPs consist of artificially intelligent, emotionally responsive 3D characters which communicate with students using natural language. The system categorizes the input questions according to specific domains and summarizes the encounter. Automated assessment by the computer was compared to manual assessment by trained raters to assess accuracy of the grading system. Results: Twenty dialogs chosen randomly from 102 total encounters were analyzed by three human and one computer rater. Overall scores calculated by the computer were not different than those provided by the human raters, and overall accuracy of the computer system was 87%, compared with 90% for human raters. Inter-rater reliability was high across 19 of 21 categories. Conclusions: We have developed a virtual standardized patient system that can understand, respond, categorize, and assess student performance in gathering information during a typical medical history, thus enabling students to practice their history-taking skills and receive immediate feedback.
Introduction: Practicing a medical history using standardized patients is an essential component of medical school curricula. Recent advances in technology now allow for newer approaches for practicing and assessing communication skills. We describe herein a virtual standardized patient (VSP) system that allows students to practice their history taking skills and receive immediate feedback. Methods: Our VSPs consist of artificially intelligent, emotionally responsive 3D characters which communicate with students using natural language. The system categorizes the input questions according to specific domains and summarizes the encounter. Automated assessment by the computer was compared to manual assessment by trained raters to assess accuracy of the grading system. Results: Twenty dialogs chosen randomly from 102 total encounters were analyzed by three human and one computer rater. Overall scores calculated by the computer were not different than those provided by the human raters, and overall accuracy of the computer system was 87%, compared with 90% for human raters. Inter-rater reliability was high across 19 of 21 categories. Conclusions: We have developed a virtual standardized patient system that can understand, respond, categorize, and assess student performance in gathering information during a typical medical history, thus enabling students to practice their history-taking skills and receive immediate feedback.
Authors: Roberto De Ponti; Jacopo Marazzato; Andrea M Maresca; Francesca Rovera; Giulio Carcano; Marco M Ferrario Journal: BMC Med Educ Date: 2020-09-25 Impact factor: 2.463