Kristi J Ferguson1, Clarence D Kreiter. 1. Office of Consultation and Research in Medical Education, University of Iowa College of Medicine, Iowa City, IA 52242, USA. kristi-ferguson@uiowa.edu
Abstract
PURPOSE: To examine the validity of using scores from a clinical evaluation form as an assessment of clinical competence. METHOD: Investigators collected a longitudinal clinical skills assessment database that included scores reflecting performance on standardized patient interactions, case-based learning performance, scores on multiple-choice clinical examinations, and preceptors' ratings of students during their clerkships. Pearson correlation coefficients and coefficients corrected for attenuation were calculated between the mean preceptor rating CEF score and the other measures collected during years one and two. RESULTS: Estimates from an earlier across-clerkship generalizability study of the CEF demonstrated that a mean rating computed across 28 forms and four clerkships (the average measurement frequencies observed in this study), yielded an estimated G-coefficient of 0.62 (Kreiter and Ferguson, 2001). Reliabilities for the other measures ranged from 0.23 to 0.56. Nine of the twelve clinical skill measures correlated with the CEF. For those significantly correlated measures where reliabilities could be calculated, correlations corrected for attenuation ranged from 0.46 to 0.58. CONCLUSION: This study indicates that the skills measured by the CEF are related to other clinical performance measures and, conversely, that pre-clinical measures of skills that are believed to be important during clinical years are in fact predictors of preceptors' ratings of clinical performance later in medical school. In addition, the magnitude of the disattenuated coefficients suggests that ratings on the CEF are dependent on important aspects such as clinical knowledge. This study suggests that when averaged over a large number of observations, mean CEF scores demonstrate validity coefficients large enough to support their use as part of an evaluation of students' clinical performance.
PURPOSE: To examine the validity of using scores from a clinical evaluation form as an assessment of clinical competence. METHOD: Investigators collected a longitudinal clinical skills assessment database that included scores reflecting performance on standardized patient interactions, case-based learning performance, scores on multiple-choice clinical examinations, and preceptors' ratings of students during their clerkships. Pearson correlation coefficients and coefficients corrected for attenuation were calculated between the mean preceptor rating CEF score and the other measures collected during years one and two. RESULTS: Estimates from an earlier across-clerkship generalizability study of the CEF demonstrated that a mean rating computed across 28 forms and four clerkships (the average measurement frequencies observed in this study), yielded an estimated G-coefficient of 0.62 (Kreiter and Ferguson, 2001). Reliabilities for the other measures ranged from 0.23 to 0.56. Nine of the twelve clinical skill measures correlated with the CEF. For those significantly correlated measures where reliabilities could be calculated, correlations corrected for attenuation ranged from 0.46 to 0.58. CONCLUSION: This study indicates that the skills measured by the CEF are related to other clinical performance measures and, conversely, that pre-clinical measures of skills that are believed to be important during clinical years are in fact predictors of preceptors' ratings of clinical performance later in medical school. In addition, the magnitude of the disattenuated coefficients suggests that ratings on the CEF are dependent on important aspects such as clinical knowledge. This study suggests that when averaged over a large number of observations, mean CEF scores demonstrate validity coefficients large enough to support their use as part of an evaluation of students' clinical performance.