OBJECTIVE: Objective structured clinical examination (OSCE) is a key part of medical student assessment. Currently, assessment is performed by medical examiners in situ. Our objective was to determine whether assessment by videotaped OSCE is as reliable as live OSCE assessment. METHODS:Participants were 95 undergraduate medical students attending their musculoskeletal week at Freeman Hospital, Newcastle (UK). Student performance on OSCE stations for shoulder or knee examinations was assessed by experienced rheumatologists. The stations were also videotaped and scored by a rheumatologist independently. The examinations consisted of a 14-item checklist and a global rating scale (GRS). RESULTS: Mean values for the shoulder OSCE checklist were 17.9 by live assessment and 17.4 by video (n = 50), and 20.9 and 20.0 for live and video knee assessment, respectively (n = 45). Intraclass correlation coefficients for shoulder and knee checklists were 0.55 and 0.58, respectively, indicating moderate reliability between live and video scores for the OSCE checklists. GRS scores were less reliable than checklist scores. There was 84% agreement in the classification of examination grades between live and video checklist scores for the shoulder and 87% agreement for the knee (kappa = 0.43 and 0.51, respectively; P < 0.001). CONCLUSION:Video OSCE has the potential to be reliable and offers some advantages over live OSCE including more efficient use of examiners' time, increased fairness, and better monitoring of standards across various schools/sites. However, further work is needed to support our findings and to implement and evaluate the quality assurance issues identified in this work before justifiable recommendations can be made.
RCT Entities:
OBJECTIVE: Objective structured clinical examination (OSCE) is a key part of medical student assessment. Currently, assessment is performed by medical examiners in situ. Our objective was to determine whether assessment by videotaped OSCE is as reliable as live OSCE assessment. METHODS:Participants were 95 undergraduate medical students attending their musculoskeletal week at Freeman Hospital, Newcastle (UK). Student performance on OSCE stations for shoulder or knee examinations was assessed by experienced rheumatologists. The stations were also videotaped and scored by a rheumatologist independently. The examinations consisted of a 14-item checklist and a global rating scale (GRS). RESULTS: Mean values for the shoulder OSCE checklist were 17.9 by live assessment and 17.4 by video (n = 50), and 20.9 and 20.0 for live and video knee assessment, respectively (n = 45). Intraclass correlation coefficients for shoulder and knee checklists were 0.55 and 0.58, respectively, indicating moderate reliability between live and video scores for the OSCE checklists. GRS scores were less reliable than checklist scores. There was 84% agreement in the classification of examination grades between live and video checklist scores for the shoulder and 87% agreement for the knee (kappa = 0.43 and 0.51, respectively; P < 0.001). CONCLUSION: Video OSCE has the potential to be reliable and offers some advantages over live OSCE including more efficient use of examiners' time, increased fairness, and better monitoring of standards across various schools/sites. However, further work is needed to support our findings and to implement and evaluate the quality assurance issues identified in this work before justifiable recommendations can be made.
Authors: Michal Kalli Hose; John Fontanesi; Manjulika Woytowitz; Diego Jarrin; Anna Quan Journal: J Gen Intern Med Date: 2017-08-07 Impact factor: 5.128
Authors: Gunther Weitz; Christian Vinzentius; Christoph Twesten; Hendrik Lehnert; Hendrik Bonnemeier; Inke R König Journal: GMS Z Med Ausbild Date: 2014-11-17