Ryan S Watson1, Andrew J Borgert2, Colette T O Heron1, Kara J Kallies2, Richard A Sidwell3, John D Mellinger4, Amit R Joshi5, Joseph M Galante6, Lowell W Chambers7, Jon B Morris8, Robert K Josloff9, Marc L Melcher10, George M Fuhrman11, Kyla P Terhune12, Lily Chang13, Elizabeth M Ferguson14, Edward D Auyang15, Kevin R Patel16, Benjamin T Jarman17. 1. Department of Medical Education, General Surgery Residency, Gundersen Medical Foundation, La Crosse, Wisconsin. 2. Department of Medical Research, Gundersen Medical Foundation, La Crosse, Wisconsin. 3. General Surgery Residency Program, Iowa Methodist Medical Center, Des Moines, Iowa. 4. Department of Surgery, Southern Illinois University School of Medicine, Springfield, Illinois. 5. Department of Surgery, Einstein Healthcare Network, Philadelphia, Pennsylvania. 6. General Surgery Residency, University of California-Davis Health System, Sacramento, California. 7. Department of Surgery, Mount Carmel Health System, Columbus, Ohio. 8. Department of Surgery, Hospital of the University of Pennsylvania, Philadelphia, Pennsylvania. 9. Department of Surgery, Abington Memorial Hospital, Abington, Pennsylvania. 10. Department of Surgery, Stanford University Medical Center, Palo Alto, California. 11. Department of Surgery, Ochsner Clinic, New Orleans, Louisiana. 12. Department of Surgery, Vanderbilt University Medical Center, Nashville, Tennessee. 13. Department of Surgery, Virginia Mason Medical Center, Seattle, Washington. 14. Department of Surgery, Maricopa Medical Center, Phoenix, Arizona. 15. Department of Surgery, University of New Mexico, Albuquerque, New Mexico. 16. Department of General Surgery, Kaiser Permanente Los Angeles Medical Center, Los Angeles, California. 17. Department of Medical Education, General Surgery Residency, Gundersen Medical Foundation, La Crosse, Wisconsin; Department of General Surgery, Gundersen Health System, La Crosse, Wisconsin. Electronic address: btjarman@gundersenhealth.org.
Abstract
OBJECTIVE: The Accreditation Council for Graduate Medical Education requires accredited residency programs to implement competency-based assessments of medical trainees based upon nationally established Milestones. Clinical competency committees (CCC) are required to prepare biannual reports using the Milestones and ensure reporting to the Accreditation Council for Graduate Medical Education. Previous research demonstrated a strong correlation between CCC and resident scores on the Milestones at 1 institution. We sought to evaluate a national sampling of general surgery residency programs and hypothesized that CCC and resident assessments are similar. DESIGN: Details regarding the makeup and process of each CCC were obtained. Major disparities were defined as an absolute mean difference of ≥0.5 on the 4-point scale. A negative assessment disparity indicated that the residents evaluated themselves at a lower level than did the CCC. Statistical analysis included Wilcoxon rank sum and Sign tests. SETTING: CCCs and categorical general surgery residents from 15 residency programs completed the Milestones document independently during the spring of 2016. RESULTS: Overall, 334 residents were included; 44 (13%) and 43 (13%) residents scored themselves ≥0.5 points higher and lower than the CCC, respectively. Female residents scored themselves a mean of 0.08 points lower, and male residents scored themselves a mean of 0.03 points higher than the CCC. Median assessment differences for postgraduate year (PGY) 1-5 were 0.03 (range: -0.94 to 1.28), -0.11 (range: -1.22 to 1.22), -0.08 (range: -1.28 to 0.81), 0.02 (range: -0.91 to 1.00), and -0.19 (range: -1.16 to 0.50), respectively. Residents in university vs. independent programs had higher rates of negative assessment differences in medical knowledge (15% vs. 6%; P = 0.015), patient care (17% vs. 5%; P = 0.002), professionalism (23% vs. 14%; P = 0.013), and system-based practice (18% vs. 9%; P = 0.031) competencies. Major assessment disparities by sex or PGY were similar among individual competencies. CONCLUSIONS: Surgery residents in this national cohort demonstrated self-awareness when compared to assessments by their respective CCCs. This was independent of program type, sex, or level of training. PGY 5 residents, female residents, and those from university programs consistently rated themselves lower than the CCC, but these were not major disparities and the significance of this is unclear.
OBJECTIVE: The Accreditation Council for Graduate Medical Education requires accredited residency programs to implement competency-based assessments of medical trainees based upon nationally established Milestones. Clinical competency committees (CCC) are required to prepare biannual reports using the Milestones and ensure reporting to the Accreditation Council for Graduate Medical Education. Previous research demonstrated a strong correlation between CCC and resident scores on the Milestones at 1 institution. We sought to evaluate a national sampling of general surgery residency programs and hypothesized that CCC and resident assessments are similar. DESIGN: Details regarding the makeup and process of each CCC were obtained. Major disparities were defined as an absolute mean difference of ≥0.5 on the 4-point scale. A negative assessment disparity indicated that the residents evaluated themselves at a lower level than did the CCC. Statistical analysis included Wilcoxon rank sum and Sign tests. SETTING: CCCs and categorical general surgery residents from 15 residency programs completed the Milestones document independently during the spring of 2016. RESULTS: Overall, 334 residents were included; 44 (13%) and 43 (13%) residents scored themselves ≥0.5 points higher and lower than the CCC, respectively. Female residents scored themselves a mean of 0.08 points lower, and male residents scored themselves a mean of 0.03 points higher than the CCC. Median assessment differences for postgraduate year (PGY) 1-5 were 0.03 (range: -0.94 to 1.28), -0.11 (range: -1.22 to 1.22), -0.08 (range: -1.28 to 0.81), 0.02 (range: -0.91 to 1.00), and -0.19 (range: -1.16 to 0.50), respectively. Residents in university vs. independent programs had higher rates of negative assessment differences in medical knowledge (15% vs. 6%; P = 0.015), patient care (17% vs. 5%; P = 0.002), professionalism (23% vs. 14%; P = 0.013), and system-based practice (18% vs. 9%; P = 0.031) competencies. Major assessment disparities by sex or PGY were similar among individual competencies. CONCLUSIONS: Surgery residents in this national cohort demonstrated self-awareness when compared to assessments by their respective CCCs. This was independent of program type, sex, or level of training. PGY 5 residents, female residents, and those from university programs consistently rated themselves lower than the CCC, but these were not major disparities and the significance of this is unclear.
Authors: Stanley J Hamstra; Kenji Yamazaki; Melissa A Barton; Sally A Santen; Michael S Beeson; Eric S Holmboe Journal: Acad Med Date: 2019-10 Impact factor: 6.893
Authors: Amanda Barone Pritchard; Christina Sloan-Heggen; Catherine E Keegan; Shane C Quinonez Journal: Genet Med Date: 2021-02-12 Impact factor: 8.864
Authors: Mary Ellen J Goldhamer; Maria Martinez-Lage; W Stephen Black-Schaffer; Jennifer T Huang; John Patrick T Co; Debra F Weinstein; Martin V Pusic Journal: J Gen Intern Med Date: 2022-04-20 Impact factor: 6.473