INTRODUCTION: The Corona Virus Disease-19 (COVID-19) pandemic disrupted medical education across the world. Online teaching has grown rapidly under lockdown. Yet the online approach for assessment presents a number of challenges, particularly when evaluating clinical competencies. The aim of this study was to investigate the feasibility, acceptability, reliability and validity of an online Virtual Clinical Encounter Examination (VICEE) to assess non-psychomotor competencies (non-procedure or manual skills) of medical students. METHOD: Sixty-one final year medical students took the VICEE as part of the final summative examination. A panel of faculty experts developed the exam cases and competencies. They administered the test online via real-time interaction with artificial intelligence (AI) based virtual patients, along with faculty and IT support. RESULTS: Student and faculty surveys demonstrated satisfaction with the experience. Confirmatory factor analysis supported convergent validity of VICEE with Direct Observation Clinical Encounter Examination (DOCEE), a previously validated clinical examination. The observed sensitivity was 81.8%, specificity 64.1% and likelihood ratio 12.6, supporting the ability of VICEE to diagnose 'clinical incompetence' among students. CONCLUSION: Our results suggest that online AI-based virtual patient high fidelity simulation may be used as an alternative tool to assess some aspects of non-psychometric competencies.
INTRODUCTION: The Corona Virus Disease-19 (COVID-19) pandemic disrupted medical education across the world. Online teaching has grown rapidly under lockdown. Yet the online approach for assessment presents a number of challenges, particularly when evaluating clinical competencies. The aim of this study was to investigate the feasibility, acceptability, reliability and validity of an online Virtual Clinical Encounter Examination (VICEE) to assess non-psychomotor competencies (non-procedure or manual skills) of medical students. METHOD: Sixty-one final year medical students took the VICEE as part of the final summative examination. A panel of faculty experts developed the exam cases and competencies. They administered the test online via real-time interaction with artificial intelligence (AI) based virtual patients, along with faculty and IT support. RESULTS: Student and faculty surveys demonstrated satisfaction with the experience. Confirmatory factor analysis supported convergent validity of VICEE with Direct Observation Clinical Encounter Examination (DOCEE), a previously validated clinical examination. The observed sensitivity was 81.8%, specificity 64.1% and likelihood ratio 12.6, supporting the ability of VICEE to diagnose 'clinical incompetence' among students. CONCLUSION: Our results suggest that online AI-based virtual patient high fidelity simulation may be used as an alternative tool to assess some aspects of non-psychometric competencies.