BACKGROUND: A case-based online assessment tool has been accomplished at the University of Ulm by simulating consultations with virtual patients in General Practice setting. After a successful pilot study, more data on validity were needed as one important step when considering introducing this assessment format to replace the regular written course exam. METHODS: Five distinct sources of validity evidence are addressed: content, response process, internal structure, relationship to other variables and consequences. An examination simulation was conducted including all students (147) of the General Practice course, scores of three virtual patient cases compared with the written course exam and data gathered regarding usability, acceptance, content and structural characteristics, and consequences. RESULTS: All five aspects were supported by collected data. Correlation to the written exam was moderate with r = 0.36, correlation between the cases higher with r = 0.50-0.56. CONCLUSION: Although a direct comparison of written course exams and dealing with virtual patients is difficult in many aspects and the study setting as an exam simulation caused confounders, this new assessment format is supported by many aspects of validity evidence. Solving virtual clinical scenarios addresses other abilities, skills and knowledge than traditional written exams and since "assessment drives learning" it could have positive implications on teaching and learning alike.
BACKGROUND: A case-based online assessment tool has been accomplished at the University of Ulm by simulating consultations with virtual patients in General Practice setting. After a successful pilot study, more data on validity were needed as one important step when considering introducing this assessment format to replace the regular written course exam. METHODS: Five distinct sources of validity evidence are addressed: content, response process, internal structure, relationship to other variables and consequences. An examination simulation was conducted including all students (147) of the General Practice course, scores of three virtual patient cases compared with the written course exam and data gathered regarding usability, acceptance, content and structural characteristics, and consequences. RESULTS: All five aspects were supported by collected data. Correlation to the written exam was moderate with r = 0.36, correlation between the cases higher with r = 0.50-0.56. CONCLUSION: Although a direct comparison of written course exams and dealing with virtual patients is difficult in many aspects and the study setting as an exam simulation caused confounders, this new assessment format is supported by many aspects of validity evidence. Solving virtual clinical scenarios addresses other abilities, skills and knowledge than traditional written exams and since "assessment drives learning" it could have positive implications on teaching and learning alike.
Authors: Katerina Bogomolova; Amir H Sam; Adam T Misky; Chinmay M Gupte; Paul H Strutton; Thomas J Hurkxkens; Beerend P Hierck Journal: Anat Sci Educ Date: 2021-03-05 Impact factor: 5.958