Michael S Ryan1, Steven Bishop, Joel Browning, Rahul J Anand, Elizabeth Waterhouse, Fidelma Rigby, Cheryl S Al-Mateen, Clifton Lee, Melissa Bradner, Jorie M Colbert-Getz. 1. M.S. Ryan is associate professor and assistant dean for clinical medical education, Department of Pediatrics, Virginia Commonwealth University, Richmond, Virginia. S. Bishop is assistant professor and clerkship director, Department of Internal Medicine, Virginia Commonwealth University, Richmond, Virginia. J. Browning is director, Academic Information Systems, Virginia Commonwealth University, Richmond, Virginia. R.J. Anand is associate professor and residency program director, Department of Surgery, Virginia Commonwealth University, Richmond, Virginia. E. Waterhouse is professor and clerkship director, Department of Neurology, Virginia Commonwealth University, Richmond, Virginia. F. Rigby is associate professor and clerkship director, Department of Obstetrics and Gynecology, Virginia Commonwealth University, Richmond, Virginia. C.S. Al-Mateen is professor and clerkship director, Department of Psychiatry, Virginia Commonwealth University, Richmond, Virginia. C. Lee is associate professor and clerkship director, Department of Pediatrics, Virginia Commonwealth University, Richmond, Virginia. M. Bradner is associate professor and clerkship director, Department of Family Medicine, Virginia Commonwealth University, Richmond, Virginia. J.M. Colbert-Getz is director, Medical Education Research, and assistant professor, Department of Internal Medicine, University of Utah School of Medicine, Salt Lake City, Utah.
Abstract
PURPOSE: The National Board of Medical Examiners' Clinical Science Subject Examinations are a component used by most U.S. medical schools to determine clerkship grades. The purpose of this study was to examine the validity of this practice. METHOD: This was a retrospective cohort study of medical students at the Virginia Commonwealth University School of Medicine who completed clerkships in 2012 through 2014. Linear regression was used to determine how well United States Medical Licensing Examination Step 1 scores predicted Subject Examination scores in seven clerkships. The authors then substituted each student's Subject Examination standard scores with his or her Step 1 standard score. Clerkship grades based on the Step 1 substitution were compared with actual grades with the Wilcoxon rank test. RESULTS: A total of 2,777 Subject Examination scores from 432 students were included in the analysis. Step 1 scores significantly predicted between 23% and 44% of the variance in Subject Examination scores, P < .001 for all clerkship regression equations. Mean differences between expected and actual Subject Examination scores were small (≤ 0.2 points). There was a match between 73% of Step 1 substituted final clerkship grades and actual final clerkship grades. CONCLUSIONS: The results of this study suggest that performance on Step 1 can be used to identify and counsel students at risk for poor performance on the Subject Examinations. In addition, these findings call into the question the validity of using scores from Subject Examinations as a high-stakes assessment of learning in individual clerkships.
PURPOSE: The National Board of Medical Examiners' Clinical Science Subject Examinations are a component used by most U.S. medical schools to determine clerkship grades. The purpose of this study was to examine the validity of this practice. METHOD: This was a retrospective cohort study of medical students at the Virginia Commonwealth University School of Medicine who completed clerkships in 2012 through 2014. Linear regression was used to determine how well United States Medical Licensing Examination Step 1 scores predicted Subject Examination scores in seven clerkships. The authors then substituted each student's Subject Examination standard scores with his or her Step 1 standard score. Clerkship grades based on the Step 1 substitution were compared with actual grades with the Wilcoxon rank test. RESULTS: A total of 2,777 Subject Examination scores from 432 students were included in the analysis. Step 1 scores significantly predicted between 23% and 44% of the variance in Subject Examination scores, P < .001 for all clerkship regression equations. Mean differences between expected and actual Subject Examination scores were small (≤ 0.2 points). There was a match between 73% of Step 1 substituted final clerkship grades and actual final clerkship grades. CONCLUSIONS: The results of this study suggest that performance on Step 1 can be used to identify and counsel students at risk for poor performance on the Subject Examinations. In addition, these findings call into the question the validity of using scores from Subject Examinations as a high-stakes assessment of learning in individual clerkships.