Christine Dauphine1, Angela L Neville2, Ashkan Moazzez2, Dennis Y Kim2, Eric R Simms3, George Singer2, Christian de Virgilio2. 1. Department of Surgery, Harbor-UCLA Medical Center, Torrance, California; Los Angeles Biomedical Research Institute, Torrance, California. Electronic address: cdauphine@dhs.lacounty.gov. 2. Department of Surgery, Harbor-UCLA Medical Center, Torrance, California; Los Angeles Biomedical Research Institute, Torrance, California. 3. Department of Surgery, Harbor-UCLA Medical Center, Torrance, California.
Abstract
OBJECTIVE: Identifying gaps in medical knowledge, patient management, and procedural competence is difficult early in surgical residency. We designed and implemented an end-of-year examination for our postgraduate year 1 residents, entitled Surgical Trainee Assessment of Readiness (STAR). Our objective in this study was to determine whether STAR scores correlated with other available indicators of resident performance, such as the American Board of Surgery in-training exam (ABSITE) and Milestone scores, and if they provided evidence of additional discriminatory value. STUDY DESIGN: Overall and component scores of the STAR exam were compared to the ABSITE and Milestone assessment scores for the 17 categorical residents that took the exam in 2016 and 2017. SETTING: Harbor-UCLA Medical Center, a university-affiliated academic medical center. PARTICIPANTS: Seventeen categorical general surgery residents. RESULTS: The STAR Total Test Score (β = 2.77, p = 0.006) was an independent predictor of the ABSITE taken the same year, and components of the STAR were independent predictors of ABSITE taken the following year. The STAR Total Test Score was lowest in the 3 residents who had at least 1 low Milestone score assessed in the same year; and 2 of these 3 residents had at least 1 low Milestone score assigned the next year after STAR. Lastly, the Patient Care 1 and 2 Milestones assessed in the same year as STAR were uniformly scored as appropriate for level of training, yet the corresponding STAR component for those milestones demonstrated 3 residents as having deficiencies. CONCLUSIONS: We have created a multifaceted standardized STAR exam, which correlates with performance on the ABSITE and early milestone scores. It also appears to discriminate resident performance where milestone assessments do not. Further evaluation of the STAR exam with longer term follow-up is needed to confirm these initial findings.
OBJECTIVE: Identifying gaps in medical knowledge, patient management, and procedural competence is difficult early in surgical residency. We designed and implemented an end-of-year examination for our postgraduate year 1 residents, entitled Surgical Trainee Assessment of Readiness (STAR). Our objective in this study was to determine whether STAR scores correlated with other available indicators of resident performance, such as the American Board of Surgery in-training exam (ABSITE) and Milestone scores, and if they provided evidence of additional discriminatory value. STUDY DESIGN: Overall and component scores of the STAR exam were compared to the ABSITE and Milestone assessment scores for the 17 categorical residents that took the exam in 2016 and 2017. SETTING: Harbor-UCLA Medical Center, a university-affiliated academic medical center. PARTICIPANTS: Seventeen categorical general surgery residents. RESULTS: The STAR Total Test Score (β = 2.77, p = 0.006) was an independent predictor of the ABSITE taken the same year, and components of the STAR were independent predictors of ABSITE taken the following year. The STAR Total Test Score was lowest in the 3 residents who had at least 1 low Milestone score assessed in the same year; and 2 of these 3 residents had at least 1 low Milestone score assigned the next year after STAR. Lastly, the Patient Care 1 and 2 Milestones assessed in the same year as STAR were uniformly scored as appropriate for level of training, yet the corresponding STAR component for those milestones demonstrated 3 residents as having deficiencies. CONCLUSIONS: We have created a multifaceted standardized STAR exam, which correlates with performance on the ABSITE and early milestone scores. It also appears to discriminate resident performance where milestone assessments do not. Further evaluation of the STAR exam with longer term follow-up is needed to confirm these initial findings.
Keywords:
Interpersonal and Communication Skills; Medical Knowledge; Patient Care; Professionalism; educational assessment; feedback; graduate medical education; medical residency; milestones; resident evaluation