BACKGROUND: The Accreditation Council for Graduate Medical Education Milestones were created as a criterion-based framework to promote competency-based education during graduate medical education. Despite widespread implementation across subspecialty programs, extensive validity evidence supporting the use of milestones within fellowship training is lacking. OBJECTIVE: We assessed the construct and response process validity of milestones in subspecialty fellowship programs in an academic medical center. METHODS: From 2014-2016, we performed a single center retrospective cohort analysis of milestone data from fellows across 5 programs. We analyzed summary statistics and performed multivariable linear regression to assess change in milestone ratings by training year and variability in ratings across fellowship programs. Finally, we examined a subset of Professionalism and Interpersonal and Communication Skills subcompetencies from the first 6 months of training to identify the proportion of fellows deemed "ready for independent practice" in these domains. RESULTS: Milestone data were available for 68 fellows, with 75 933 unique subcompetency ratings. Multivariable linear regression, adjusted for subcompetency and subspecialty, revealed an increase of 0.17 (0.16-0.19) in ratings with each postgraduate year level increase (P < .005), as well as significant variation in milestone ratings across subspecialties. For the Professionalism and Interpersonal and Communication Skills domains, mean ratings within the first 6 months of training were 3.78 and 3.95, respectively. CONCLUSIONS: We noted a minimal upward trend of milestone ratings in subspecialty training programs, and significant variability in implementing milestones across differing subspecialties. This may suggest possible difficulties with the construct validity and response process of the milestone system in certain medical subspecialties.
BACKGROUND: The Accreditation Council for Graduate Medical Education Milestones were created as a criterion-based framework to promote competency-based education during graduate medical education. Despite widespread implementation across subspecialty programs, extensive validity evidence supporting the use of milestones within fellowship training is lacking. OBJECTIVE: We assessed the construct and response process validity of milestones in subspecialty fellowship programs in an academic medical center. METHODS: From 2014-2016, we performed a single center retrospective cohort analysis of milestone data from fellows across 5 programs. We analyzed summary statistics and performed multivariable linear regression to assess change in milestone ratings by training year and variability in ratings across fellowship programs. Finally, we examined a subset of Professionalism and Interpersonal and Communication Skills subcompetencies from the first 6 months of training to identify the proportion of fellows deemed "ready for independent practice" in these domains. RESULTS: Milestone data were available for 68 fellows, with 75 933 unique subcompetency ratings. Multivariable linear regression, adjusted for subcompetency and subspecialty, revealed an increase of 0.17 (0.16-0.19) in ratings with each postgraduate year level increase (P < .005), as well as significant variation in milestone ratings across subspecialties. For the Professionalism and Interpersonal and Communication Skills domains, mean ratings within the first 6 months of training were 3.78 and 3.95, respectively. CONCLUSIONS: We noted a minimal upward trend of milestone ratings in subspecialty training programs, and significant variability in implementing milestones across differing subspecialties. This may suggest possible difficulties with the construct validity and response process of the milestone system in certain medical subspecialties.
Authors: Michael L Green; Eva M Aagaard; Kelly J Caverzagie; Davoren A Chick; Eric Holmboe; Gregory Kane; Cynthia D Smith; William Iobst Journal: J Grad Med Educ Date: 2009-09
Authors: Jason R Frank; Linda S Snell; Olle Ten Cate; Eric S Holmboe; Carol Carraccio; Susan R Swing; Peter Harris; Nicholas J Glasgow; Craig Campbell; Deepak Dath; Ronald M Harden; William Iobst; Donlin M Long; Rani Mungroo; Denyse L Richardson; Jonathan Sherbino; Ivan Silver; Sarah Taber; Martin Talbot; Kenneth A Harris Journal: Med Teach Date: 2010 Impact factor: 3.650
Authors: Michael S Beeson; Wallace A Carter; Theodore A Christopher; Jonathan W Heidt; James H Jones; Lynne E Meyer; Susan B Promes; Kevin G Rodgers; Philip H Shayne; Susan R Swing; Mary Jo Wagner Journal: Acad Emerg Med Date: 2013-06-19 Impact factor: 3.451
Authors: Eva Aagaard; Gregory C Kane; Lisa Conforti; Sarah Hood; Kelly J Caverzagie; Cynthia Smith; Davoren A Chick; Eric S Holmboe; William F Iobst Journal: J Grad Med Educ Date: 2013-09
Authors: Michael S Beeson; Eric S Holmboe; Robert C Korte; Thomas J Nasca; Timothy Brigham; Chad M Russ; Cameron T Whitley; Earl J Reisdorff Journal: Acad Emerg Med Date: 2015-06-25 Impact factor: 3.451
Authors: Susan R Swing; Michael S Beeson; Carol Carraccio; Michael Coburn; William Iobst; Nathan R Selden; Peter J Stern; Kay Vydareny Journal: J Grad Med Educ Date: 2013-03
Authors: Taylor Sawyer; Megan Gray; Shilpi Chabra; Lindsay C Johnston; Melissa M Carbajal; Maria Gillam-Krakauer; Jennifer M Brady; Heather French Journal: J Grad Med Educ Date: 2021-06-14