BACKGROUND: Despite the goal of objective structured clinical examinations (OSCEs) to be objective, examiner biases may influence scores. Examiner familiarity with candidates is a potential bias that has not been well studied. METHOD: To determine the effect of familiarity, OSCE scores for 158 internal medicine residents were analyzed by whether examiners were familiar with them, based on previous clinical encounters, and if previous impressions were positive or negative. A hierarchical multivariable analysis of variance was performed to control for resident, examiner, and level of training. RESULTS: Across 480 interactions (50 examiners, 158 residents), multivariable analysis showed that positive familiarity was associated with a significant increase in ratings (+0.37 on a 5-point scale), comparable to the difference between first- and third/fourth-year residents. CONCLUSIONS: Familiarity with candidates is a significant source of examiner bias in OSCE scores. Consideration should be paid to the influence of examiners' previous knowledge of examinees and attempts made to mitigate this bias.
BACKGROUND: Despite the goal of objective structured clinical examinations (OSCEs) to be objective, examiner biases may influence scores. Examiner familiarity with candidates is a potential bias that has not been well studied. METHOD: To determine the effect of familiarity, OSCE scores for 158 internal medicine residents were analyzed by whether examiners were familiar with them, based on previous clinical encounters, and if previous impressions were positive or negative. A hierarchical multivariable analysis of variance was performed to control for resident, examiner, and level of training. RESULTS: Across 480 interactions (50 examiners, 158 residents), multivariable analysis showed that positive familiarity was associated with a significant increase in ratings (+0.37 on a 5-point scale), comparable to the difference between first- and third/fourth-year residents. CONCLUSIONS: Familiarity with candidates is a significant source of examiner bias in OSCE scores. Consideration should be paid to the influence of examiners' previous knowledge of examinees and attempts made to mitigate this bias.
Authors: Michael D Jain; George A Tomlinson; Danica Lam; Jessica Liu; Deepti Damaraju; Allan S Detsky; Luke A Devine Journal: J Grad Med Educ Date: 2014-09
Authors: Boaz Shulruf; Arvin Damodaran; Phil Jones; Sean Kennedy; George Mangos; Anthony J O'Sullivan; Joel Rhee; Silas Taylor; Gary Velan; Peter Harris Journal: BMC Med Educ Date: 2018-01-06 Impact factor: 2.463
Authors: Boaz Shulruf; Barbara-Ann Adelstein; Arvin Damodaran; Peter Harris; Sean Kennedy; Anthony O'Sullivan; Silas Taylor Journal: BMC Med Educ Date: 2018-11-20 Impact factor: 2.463