Danette W McKinley1, John R Boulet. 1. Educational Commission for Foreign Medical Graduates, Philadelphia, Pennsylvania 19104, USA. dmckinley@ecfmg.org
Abstract
BACKGROUND: If the sequence of administration of encounters in performance-based examinations has an appreciable impact on examinee performance, the validity of any associated assessment decisions could be compromised. PURPOSE: The purpose of this article was to determine if the order in which encounters occurred in a multistation standardized patient assessment had an effect on examinee performance and, if so, could this be explained by examinee experience and familiarity with the assessment. METHODS: Analysis of the scores of over 11,000 examinees who took a 10-station clinical skills assessment was performed to determine trends across the cases. RESULTS: The results showed that administration sequence does have significant, albeit small, impact on examinee performance. In general, examinees perform slightly better as they proceed through the assessment, especially in the first few encounters. CONCLUSIONS: The possible reasons for this effect were difficult to establish, but it is likely that comfort with the examination format plays a significant role. For a given assessment, all examinees saw the same cases in a fixed sequence. Therefore, as long as performance gains are relatively small and consistent, it is unlikely that sequence effects compromised the fairness of the assessment.
BACKGROUND: If the sequence of administration of encounters in performance-based examinations has an appreciable impact on examinee performance, the validity of any associated assessment decisions could be compromised. PURPOSE: The purpose of this article was to determine if the order in which encounters occurred in a multistation standardized patient assessment had an effect on examinee performance and, if so, could this be explained by examinee experience and familiarity with the assessment. METHODS: Analysis of the scores of over 11,000 examinees who took a 10-station clinical skills assessment was performed to determine trends across the cases. RESULTS: The results showed that administration sequence does have significant, albeit small, impact on examinee performance. In general, examinees perform slightly better as they proceed through the assessment, especially in the first few encounters. CONCLUSIONS: The possible reasons for this effect were difficult to establish, but it is likely that comfort with the examination format plays a significant role. For a given assessment, all examinees saw the same cases in a fixed sequence. Therefore, as long as performance gains are relatively small and consistent, it is unlikely that sequence effects compromised the fairness of the assessment.
Authors: Lubna A Baig; Tanya N Beran; Andrea Vallevand; Zarrukh A Baig; Mauricio Monroy-Cuadros Journal: BMC Med Educ Date: 2014-05-19 Impact factor: 2.463