CONTEXT: Identifying medical students who will perform poorly during residency is difficult. OBJECTIVE: Determine whether commonly available data predicts low performance ratings during internship by residency program directors. DESIGN: Prospective cohort involving medical school data from graduates of the Uniformed Services University (USU), surveys about experiences at USU, and ratings of their performance during internship by their program directors. SETTING: Uniformed Services University. PARTICIPANTS: One thousand sixty-nine graduates between 1993 and 2002. MAIN OUTCOME MEASURE(S): Residency program directors completed an 18-item survey assessing intern performance. Factor analysis of these items collapsed to 2 domains: knowledge and professionalism. These domains were scored and performance dichotomized at the 10th percentile. RESULTS: Many variables showed a univariate relationship with ratings in the bottom 10% of both domains. Multivariable logistic regression modeling revealed that grades earned during the third year predicted low ratings in both knowledge (odds ratio [OR] = 4.9; 95%CI = 2.7-9.2) and professionalism (OR = 7.3; 95%CI = 4.1-13.0). USMLE step 1 scores (OR = 1.03; 95%CI = 1.01-1.05) predicted knowledge but not professionalism. The remaining variables were not independently predictive of performance ratings. The predictive ability for the knowledge and professionalism models was modest (respective area under ROC curves = 0.735 and 0.725). CONCLUSIONS: A strong association exists between the third year GPA and internship ratings by program directors in professionalism and knowledge. In combination with third year grades, either the USMLE step 1 or step 2 scores predict poor knowledge ratings. Despite a wealth of available markers and a large data set, predicting poor performance during internship remains difficult.
CONTEXT: Identifying medical students who will perform poorly during residency is difficult. OBJECTIVE: Determine whether commonly available data predicts low performance ratings during internship by residency program directors. DESIGN: Prospective cohort involving medical school data from graduates of the Uniformed Services University (USU), surveys about experiences at USU, and ratings of their performance during internship by their program directors. SETTING: Uniformed Services University. PARTICIPANTS: One thousand sixty-nine graduates between 1993 and 2002. MAIN OUTCOME MEASURE(S): Residency program directors completed an 18-item survey assessing intern performance. Factor analysis of these items collapsed to 2 domains: knowledge and professionalism. These domains were scored and performance dichotomized at the 10th percentile. RESULTS: Many variables showed a univariate relationship with ratings in the bottom 10% of both domains. Multivariable logistic regression modeling revealed that grades earned during the third year predicted low ratings in both knowledge (odds ratio [OR] = 4.9; 95%CI = 2.7-9.2) and professionalism (OR = 7.3; 95%CI = 4.1-13.0). USMLE step 1 scores (OR = 1.03; 95%CI = 1.01-1.05) predicted knowledge but not professionalism. The remaining variables were not independently predictive of performance ratings. The predictive ability for the knowledge and professionalism models was modest (respective area under ROC curves = 0.735 and 0.725). CONCLUSIONS: A strong association exists between the third year GPA and internship ratings by program directors in professionalism and knowledge. In combination with third year grades, either the USMLE step 1 or step 2 scores predict poor knowledge ratings. Despite a wealth of available markers and a large data set, predicting poor performance during internship remains difficult.
Authors: Steven J Durning; Louis N Pangaro; Linda L Lawrence; Donna Waechter; John McManigle; Jeffrey L Jackson Journal: Acad Med Date: 2005-10 Impact factor: 6.893
Authors: Maxine A Papadakis; Arianne Teherani; Mary A Banach; Timothy R Knettler; Susan L Rattner; David T Stern; J Jon Veloski; Carol S Hodgson Journal: N Engl J Med Date: 2005-12-22 Impact factor: 91.245
Authors: Hossam Hamdy; Kameshwar Prasad; M Brownell Anderson; Albert Scherpbier; Reed Williams; Rein Zwierstra; Helen Cuddihy Journal: Med Teach Date: 2006-03 Impact factor: 3.650
Authors: Michael W Cullen; Darcy A Reed; Andrew J Halvorsen; Christopher M Wittich; Lisa M Baumann Kreuziger; Mira T Keddis; Furman S McDonald; Thomas J Beckman Journal: Mayo Clin Proc Date: 2011-03 Impact factor: 7.616
Authors: Michael W Cullen; Thomas J Beckman; Kristine M Baldwin; Gregory J Engstler; Jay Mandrekar; Christopher G Scott; Kyle W Klarich Journal: Tex Heart Inst J Date: 2020-08-01