Ashley M Maranich1,2, Paul A Hemmer1,3, Sebastian Uijtdehaage1,4, Alexis Battista1,5. 1. All authors are with the Uniformed Services University. 2. is Assistant Dean for Clinical Sciences and Associate Professor of Pediatrics. 3. is Professor of Medicine and Professor of Health Professions Education. 4. is Professor of Medicine and Professor of Health Professions Education, and Associate Editor, Journal of Graduate Medical Education. 5. is Assistant Professor of Medicine.
Abstract
Background: Since the Accreditation Council for Graduate Medical Education (ACGME) introduced the Milestones in 2013, the body of validity evidence supporting their use has grown, but there is a gap with regard to response process. Objective: The purpose of this study is to qualitatively explore validity evidence pertaining to the response process of individual Clinical Competency Committee (CCC) members when assigning Milestone ratings to a resident. Methods: Using a constructivist paradigm, we conducted a thematic analysis of semi-structured interviews with 8 Transitional Year (TY) CCC members from 4 programs immediately following a CCC meeting between November and December 2020. Participants were queried about their response process in their application of Milestone assessment. Analysis was iterative, including coding, constant comparison, and theming. Results: Participant interviews identified an absence of formal training and a perception that Milestones are a tool for resident assessment without recognizing their role in program evaluation. In describing their thought process, participants reported comparing averaged assessment data to peers and time in training to generate Milestone ratings. Meaningful narrative comments, when available, differentiated resident performance from peers. When assessment data were absent, participants assumed an average performance. Conclusions: Our study found that the response process used by TY CCC members was not always consistent with the dual purpose of the Milestones to improve educational outcomes at the levels of residents and the program.
Background: Since the Accreditation Council for Graduate Medical Education (ACGME) introduced the Milestones in 2013, the body of validity evidence supporting their use has grown, but there is a gap with regard to response process. Objective: The purpose of this study is to qualitatively explore validity evidence pertaining to the response process of individual Clinical Competency Committee (CCC) members when assigning Milestone ratings to a resident. Methods: Using a constructivist paradigm, we conducted a thematic analysis of semi-structured interviews with 8 Transitional Year (TY) CCC members from 4 programs immediately following a CCC meeting between November and December 2020. Participants were queried about their response process in their application of Milestone assessment. Analysis was iterative, including coding, constant comparison, and theming. Results: Participant interviews identified an absence of formal training and a perception that Milestones are a tool for resident assessment without recognizing their role in program evaluation. In describing their thought process, participants reported comparing averaged assessment data to peers and time in training to generate Milestone ratings. Meaningful narrative comments, when available, differentiated resident performance from peers. When assessment data were absent, participants assumed an average performance. Conclusions: Our study found that the response process used by TY CCC members was not always consistent with the dual purpose of the Milestones to improve educational outcomes at the levels of residents and the program.
Authors: Paul A Hemmer; Gregory A Dadekian; Christopher Terndrup; Louis N Pangaro; Allison B Weisbrod; Mark D Corriere; Rechell Rodriguez; Patricia Short; William F Kelly Journal: J Gen Intern Med Date: 2015-09 Impact factor: 5.128
Authors: Sebastiano Cassaro; Benjamin T Jarman; Amit R T Joshi; Sidra Goldman-Mellor; William W Hope; Samir Johna; Theodor Kaufman; Kevin J Grannan Journal: J Surg Educ Date: 2019-09-28 Impact factor: 2.891
Authors: Karen E Hauer; Jerome Clauser; Rebecca S Lipner; Eric S Holmboe; Kelly Caverzagie; Stanley J Hamstra; Sarah Hood; William Iobst; Eric Warm; Furman S McDonald Journal: Ann Intern Med Date: 2016-05-10 Impact factor: 25.391
Authors: Stanley J Hamstra; Kenji Yamazaki; Melissa A Barton; Sally A Santen; Michael S Beeson; Eric S Holmboe Journal: Acad Med Date: 2019-10 Impact factor: 6.893
Authors: Robin Klein; Nneka N Ufere; Sowmya R Rao; Jennifer Koch; Anna Volerman; Erin D Snyder; Sarah Schaeffer; Vanessa Thompson; Ana Sofia Warner; Katherine A Julian; Kerri Palamara Journal: JAMA Netw Open Date: 2020-07-01