BACKGROUND: This study examines the effect of using multiple modalities to evaluate medical students. METHODS: Thirty-four students were evaluated by a complex model utilizing National Board of Medical Examiners (NBME) shelf examination, Objective Structured Clinical Examination (OSCE), Computer Patient Simulation (CPS), faculty and peer evaluation. Results were compared with a traditional model based on NBME and faculty evaluation alone. RESULTS: Reliability (coefficient alpha) of the complex and traditional models were 0.72 and 0.47, respectively. Item correlations suggested that NBME was most discriminating (r = 0.75), followed by OSCE (r = 0.52), peer evaluation (r = 0.43), CPS (r = 0.39), and faculty evaluation (r = 0.32). Rank order correlation (Spearman's rho) between scores calculated using each model was 0.87. CONCLUSIONS: Although the complex model has improved reliability, both models rank students similarly. However, neither model fully captures and reflects the information provided by each of the specific evaluation methods.
BACKGROUND: This study examines the effect of using multiple modalities to evaluate medical students. METHODS: Thirty-four students were evaluated by a complex model utilizing National Board of Medical Examiners (NBME) shelf examination, Objective Structured Clinical Examination (OSCE), Computer Patient Simulation (CPS), faculty and peer evaluation. Results were compared with a traditional model based on NBME and faculty evaluation alone. RESULTS: Reliability (coefficient alpha) of the complex and traditional models were 0.72 and 0.47, respectively. Item correlations suggested that NBME was most discriminating (r = 0.75), followed by OSCE (r = 0.52), peer evaluation (r = 0.43), CPS (r = 0.39), and faculty evaluation (r = 0.32). Rank order correlation (Spearman's rho) between scores calculated using each model was 0.87. CONCLUSIONS: Although the complex model has improved reliability, both models rank students similarly. However, neither model fully captures and reflects the information provided by each of the specific evaluation methods.
Authors: Aisha M Al-Osail; Mona H Al-Sheikh; Emad M Al-Osail; Mohannad A Al-Ghamdi; Abdulaziz M Al-Hawas; Abdullah S Al-Bahussain; Ahmed A Al-Dajani Journal: BMC Res Notes Date: 2015-10-19