BACKGROUND: Implementing large-scale multi-site objective structured clinical examination (OSCEs) for national competency examination in a low resource country is challenging. AIMS: To describe the first national OSCE for national competency examination of medical doctors in Indonesia and evaluate the reliability, validity, feasibility, acceptability, and educational impact. METHODS: We collected electronically the OSCE scores from 49 out of 73 medical schools that participated to assess reliability and validity. We conducted electronic survey to examiners, examinees, SP trainers, and OSCE coordinators to assess feasibility, acceptability, and educational impact. RESULTS: The Cronbach's alpha coefficient across station was 0.79. There was strong correlation between rubric and global rating scores in each station (coefficient correlation ranges from 0.705 to 0.82). The content validity ratio was 0.97. The coefficient correlation between OSCE and MCQ was 0.335 (p = 0.00). All 49 medical schools were able to conduct OSCE simultaneously. Examiners, examinees, SP trainers, and OSCE coordinators had good perception regarding feasibility and acceptability of OSCE. Both examiners and examinees indicated good educational impact of OSCE application. The cutting score based on the borderline regression method was 61.96%. There were 67.39% of the examinees achieved similar or above the cutting score. CONCLUSION: With 12 stations 15 min each, the reliability coefficient across station is intermediate. Content validity is good. It is feasible and acceptable to implement large-scale multi-site OSCEs in Indonesia. Examiners and examinees perceive good educational impact on OSCE implementation.
BACKGROUND: Implementing large-scale multi-site objective structured clinical examination (OSCEs) for national competency examination in a low resource country is challenging. AIMS: To describe the first national OSCE for national competency examination of medical doctors in Indonesia and evaluate the reliability, validity, feasibility, acceptability, and educational impact. METHODS: We collected electronically the OSCE scores from 49 out of 73 medical schools that participated to assess reliability and validity. We conducted electronic survey to examiners, examinees, SP trainers, and OSCE coordinators to assess feasibility, acceptability, and educational impact. RESULTS: The Cronbach's alpha coefficient across station was 0.79. There was strong correlation between rubric and global rating scores in each station (coefficient correlation ranges from 0.705 to 0.82). The content validity ratio was 0.97. The coefficient correlation between OSCE and MCQ was 0.335 (p = 0.00). All 49 medical schools were able to conduct OSCE simultaneously. Examiners, examinees, SP trainers, and OSCE coordinators had good perception regarding feasibility and acceptability of OSCE. Both examiners and examinees indicated good educational impact of OSCE application. The cutting score based on the borderline regression method was 61.96%. There were 67.39% of the examinees achieved similar or above the cutting score. CONCLUSION: With 12 stations 15 min each, the reliability coefficient across station is intermediate. Content validity is good. It is feasible and acceptable to implement large-scale multi-site OSCEs in Indonesia. Examiners and examinees perceive good educational impact on OSCE implementation.