BACKGROUND: The Medical Council of Canada (MCC) administers an objective structured clinical examination for licensure. Traditionally, physician examiners (PE) have evaluated these examinees. Recruitment of physicians is becoming more difficult. Determining if alternate scorers can be used is of increasing importance. METHOD: In 2003, the MCC ran a study using trained assessors (TA) simultaneously with PEs. Four examination centers and three history-taking stations were selected. Health care workers were recruited as the TAs. RESULTS: A 3x2x4 mixed analyses of variance indicated no significant difference between scorers (F1,462=.01, p=.94). There were significant interaction effects, which were, localized to site 1/station 3, site 3/station 2, and site 4/station1. Pass/fail decisions would have misclassified 14.4-25.01% of examinees. CONCLUSION: Trained assessors may be a valid alternative to PE for completing checklists in history-taking stations, but their role in completing global ratings is not supported by this study.
BACKGROUND: The Medical Council of Canada (MCC) administers an objective structured clinical examination for licensure. Traditionally, physician examiners (PE) have evaluated these examinees. Recruitment of physicians is becoming more difficult. Determining if alternate scorers can be used is of increasing importance. METHOD: In 2003, the MCC ran a study using trained assessors (TA) simultaneously with PEs. Four examination centers and three history-taking stations were selected. Health care workers were recruited as the TAs. RESULTS: A 3x2x4 mixed analyses of variance indicated no significant difference between scorers (F1,462=.01, p=.94). There were significant interaction effects, which were, localized to site 1/station 3, site 3/station 2, and site 4/station1. Pass/fail decisions would have misclassified 14.4-25.01% of examinees. CONCLUSION: Trained assessors may be a valid alternative to PE for completing checklists in history-taking stations, but their role in completing global ratings is not supported by this study.
Authors: Brian E Mavis; Dianne P Wagner; Rebecca C Henry; Laura Carravallah; Jon Gold; Joel Maurer; Asad Mohmand; Janet Osuch; Steven Roskos; Andrew Saxe; Aron Sousa; Vince Winkler Prins Journal: Med Educ Online Date: 2013-07-22