J A Colliver1, M H Swartz, R S Robbs. 1. Southern Illinois University School of Medicine, Springfield, IL 62794, U.S.A. colliver@siumed.edu
Abstract
CONTEXT: Ethnicity has been a continuing concern for the valid assessment of clinical performance with standardized patients (SPs). The concern is that examinee ethnicity and SP ethnicity might interact, such that examinees might score higher in encounters with SPs of the same ethnicity. OBJECTIVE: To test for an interaction of examinee ethnicity and SP ethnicity on clinical performance in an SP examination. MAIN OUTCOME MEASURES: History-taking and physical-examination scores and interpersonal-and communication-skills scores, both based on checklists completed by SPs. Poststation scores for answers to case-related questions concerning pathophysiology, diagnosis, test selection, and test interpretation. SETTING AND PARTICIPANTS: Two graduating classes of over 1,000 fourth-year medical students each in the New York City Consortium were tested on the SP assessment administered at The Morchand Center of Mount Sinai School of Medicine. DESIGN AND ANALYSIS: The primary analyses were two-way (2 x 2) analyses, to test the main and interaction effects of examinee ethnicity and SP ethnicity. Effect-size measures (standardized mean differences, d) were computed to provide a sharper picture of the effects. RESULTS: Of the 24 interaction analyses, only three were statistically significant (not significantly more than expected by chance) and the results were mixed: one analysis showed better examinee performance in encounters with SPs of the same ethnic background and the other two showed the opposite. For all 24 interactions, significant or not, the results showed weak effects and no consistent pattern. White examinees scored on average 0.12 standard deviations above black examinees in encounters with white SPs, and 0.11 standard deviations higher in encounters with black SPs. CONCLUSIONS: These initial results are encouraging and should dispel some of the concern about ethnicity in SP assessment, at least about the operation of an examinee-by-SP-ethnicity interaction that would pose a serious threat to the validity of the examination scores.
CONTEXT: Ethnicity has been a continuing concern for the valid assessment of clinical performance with standardized patients (SPs). The concern is that examinee ethnicity and SP ethnicity might interact, such that examinees might score higher in encounters with SPs of the same ethnicity. OBJECTIVE: To test for an interaction of examinee ethnicity and SP ethnicity on clinical performance in an SP examination. MAIN OUTCOME MEASURES: History-taking and physical-examination scores and interpersonal-and communication-skills scores, both based on checklists completed by SPs. Poststation scores for answers to case-related questions concerning pathophysiology, diagnosis, test selection, and test interpretation. SETTING AND PARTICIPANTS: Two graduating classes of over 1,000 fourth-year medical students each in the New York City Consortium were tested on the SP assessment administered at The Morchand Center of Mount Sinai School of Medicine. DESIGN AND ANALYSIS: The primary analyses were two-way (2 x 2) analyses, to test the main and interaction effects of examinee ethnicity and SP ethnicity. Effect-size measures (standardized mean differences, d) were computed to provide a sharper picture of the effects. RESULTS: Of the 24 interaction analyses, only three were statistically significant (not significantly more than expected by chance) and the results were mixed: one analysis showed better examinee performance in encounters with SPs of the same ethnic background and the other two showed the opposite. For all 24 interactions, significant or not, the results showed weak effects and no consistent pattern. White examinees scored on average 0.12 standard deviations above black examinees in encounters with white SPs, and 0.11 standard deviations higher in encounters with black SPs. CONCLUSIONS: These initial results are encouraging and should dispel some of the concern about ethnicity in SP assessment, at least about the operation of an examinee-by-SP-ethnicity interaction that would pose a serious threat to the validity of the examination scores.
Authors: Adam L Wendling; Shivashankar Halan; Patrick Tighe; Linda Le; Tammy Euliano; Benjamin Lok Journal: Acad Med Date: 2011-03 Impact factor: 6.893