Faith J Ross1, David G Metro2, Shawn T Beaman3, James G Cain4, Monique M Dowdy5, Abraham Apfel6, Jong-Hyeon Jeong7, James W Ibinson8. 1. University of Pittsburgh Department of Anesthesiology, 3471 Fifth Ave, Kaufmann Bldg Suite 910, Pittsburgh, PA, 15213. Electronic address: faithjordanross@gmail.com. 2. University of Pittsburgh Department of Anesthesiology, 3471 Fifth Ave, Kaufmann Bldg Suite 910, Pittsburgh, PA, 15213. Electronic address: metrodg@anes.upmc.edu. 3. University of Pittsburgh Department of Anesthesiology, 3471 Fifth Ave, Kaufmann Bldg Suite 910, Pittsburgh, PA, 15213. Electronic address: beamst@upmc.edu. 4. University of Pittsburgh Department of Anesthesiology, 3471 Fifth Ave, Kaufmann Bldg Suite 910, Pittsburgh, PA, 15213. Electronic address: cainj@upmc.edu. 5. University of Pittsburgh Department of Anesthesiology, 3471 Fifth Ave, Kaufmann Bldg Suite 910, Pittsburgh, PA, 15213. Electronic address: Dowdymm@upmc.edu. 6. University of Pittsburgh Graduate School of Public Health, Epidemiology Data Coordinating Center, 130 DeSoto St, 127 Parran Hall, Pittsburgh, PA, 15261. Electronic address: aba44@pitt.edu. 7. University of Pittsburgh Graduate School of Public Health, Epidemiology Data Coordinating Center, A444 Crabtree, 130 DeSoto St, Pittsburgh, PA, 15261. Electronic address: jjeong@pitt.edu. 8. University of Pittsburgh Department of Anesthesiology, 3471 Fifth Ave, Kaufmann Bldg Suite 910, Pittsburgh, PA, 15213. Electronic address: ibinsonjw@upmc.edu.
Abstract
STUDY OBJECTIVE: The objective was to determine if there is a correlation between resident postgraduate year (PGY) of training and self-evaluation of performance using the Accreditation Council for Graduate Medical Education milestones. DESIGN: Survey. SETTING: Residency program at a large academic center. PATIENTS: Residents and Faculty Clinical Competency Committee (CCC). INTERVENTIONS: None. MEASUREMENTS: Resident and CCC milestone scores. MAIN RESULTS: Correlation coefficients for average score for each milestone vs PGY level ranged from 0.80 for receiving and giving feedback to 0.95 for anesthetic choice and conduct. All milestones showed a relatively linear relationship with PGY of training, and none were found to be consistently reached very late or very early in training. When examining variation across the scores for the individual residents, the distributions for PGY-2 and -3 appeared to be wider than those for PGY-1 and -4. The intraclass correlation coefficients ranged from 0.718 to 0.928. CONCLUSIONS: There was a remarkable degree of consistency in the relationship between level of training and resident self-assessment score for every milestone, as well as strong agreement between the resident and CCC faculty scores. Examination of the variance in the scores, when interpreted in light of our particular training program's characteristics, suggests that the milestones accurately reflect the progression in skill across the residency. In addition, given the concordance between the self-evaluation scores and the CCC faculty scores, self-evaluation may be a reasonable starting point as programs begin the daunting task of determining scores for each of the 25 milestones as part of the biannual evaluation process.
STUDY OBJECTIVE: The objective was to determine if there is a correlation between resident postgraduate year (PGY) of training and self-evaluation of performance using the Accreditation Council for Graduate Medical Education milestones. DESIGN: Survey. SETTING: Residency program at a large academic center. PATIENTS: Residents and Faculty Clinical Competency Committee (CCC). INTERVENTIONS: None. MEASUREMENTS: Resident and CCC milestone scores. MAIN RESULTS: Correlation coefficients for average score for each milestone vs PGY level ranged from 0.80 for receiving and giving feedback to 0.95 for anesthetic choice and conduct. All milestones showed a relatively linear relationship with PGY of training, and none were found to be consistently reached very late or very early in training. When examining variation across the scores for the individual residents, the distributions for PGY-2 and -3 appeared to be wider than those for PGY-1 and -4. The intraclass correlation coefficients ranged from 0.718 to 0.928. CONCLUSIONS: There was a remarkable degree of consistency in the relationship between level of training and resident self-assessment score for every milestone, as well as strong agreement between the resident and CCC faculty scores. Examination of the variance in the scores, when interpreted in light of our particular training program's characteristics, suggests that the milestones accurately reflect the progression in skill across the residency. In addition, given the concordance between the self-evaluation scores and the CCC faculty scores, self-evaluation may be a reasonable starting point as programs begin the daunting task of determining scores for each of the 25 milestones as part of the biannual evaluation process.
Authors: Daniel J Schumacher; Beth King; Michelle M Barnes; Sean P Elliott; Kathleen Gibbs; Jon F McGreevy; Javier Gonzalez Del Rey; Tanvi Sharma; Catherine Michelson; Alan Schwartz Journal: J Grad Med Educ Date: 2018-08
Authors: Anna L Waterbrook; Karen C Spear Ellinwood; T Gail Pritchard; Karen Bertels; Ariel C Johnson; Alice Min; Lisa R Stoneking Journal: Adv Med Educ Pract Date: 2018-05-04