Nicholas A Yaghmour1, Lauren J Poulin2, Elizabeth C Bernabeo3, Andem Ekpenyong4, Su-Ting T Li5, Aimee R Eden6, Karen E Hauer7, Aleksandr M Tichter8, Stanley J Hamstra9, Eric S Holmboe10. 1. is Associate Director, Milestones and Well-Being Research, Physician Engagement, Recruitment, and Wellness, Accreditation Council for Graduate Medical Education (ACGME). 2. is Milestones Administrator, Milestones Department, ACGME. 3. is Research Consultant, ACGME. 4. is Associate Professor, Department of Internal Medicine, Rush University Medical Center. 5. is Professor, Vice-Chair of Education, and Residency Program Director, Division of Pediatric Hospital Medicine, Department of Pediatrics, University of California Davis. 6. is Medical Anthropologist, American Board of Family Medicine. 7. is Associate Dean, Competency Assessment and Professional Standards, and Professor of Medicine, University of California, San Francisco. 8. is Assistant Professor Program Director, Department of Emergency Medicine, Baylor College of Medicine; at the time of research. 9. was Vice President, Milestones Research and Evaluation, ACGME, and is now Professor, Department of Surgery, University of Toronto, Adjunct Professor, Department of Medical Education, Feinberg School of Medicine, Northwestern University, and Research Consultant, ACGME. 10. is Chief Research, Milestone Development, and Evaluation Officer, ACGME.
Abstract
BACKGROUND: Since 2013, US residency programs have used the competency-based framework of the Milestones to report resident progress and to provide feedback to residents. The implementation of Milestones-based assessments, clinical competency committee (CCC) meetings, and processes for providing feedback varies among programs and warrants systematic examination across specialties. OBJECTIVE: We sought to determine how varying assessment, CCC, and feedback implementation strategies result in different outcomes in resource expenditure and stakeholder engagement, and to explore the contextual forces that moderate these outcomes. METHODS: From 2017 to 2018, interviews were conducted of program directors, CCC chairs, and residents in emergency medicine (EM), internal medicine (IM), pediatrics, and family medicine (FM), querying their experiences with Milestone processes in their respective programs. Interview transcripts were coded using template analysis, with the initial template derived from previous research. The research team conducted iterative consensus meetings to ensure that the evolving template accurately represented phenomena described by interviewees. RESULTS: Forty-four individuals were interviewed across 16 programs (5 EM, 4 IM, 5 pediatrics, 3 FM). We identified 3 stages of Milestone-process implementation, including a resource-intensive early stage, an increasingly efficient transition stage, and a final stage for fine-tuning. CONCLUSIONS: Residency program leaders can use these findings to place their programs along an implementation continuum and gain an understanding of the strategies that have enabled their peers to progress to improved efficiency and increased resident and faculty engagement.
BACKGROUND: Since 2013, US residency programs have used the competency-based framework of the Milestones to report resident progress and to provide feedback to residents. The implementation of Milestones-based assessments, clinical competency committee (CCC) meetings, and processes for providing feedback varies among programs and warrants systematic examination across specialties. OBJECTIVE: We sought to determine how varying assessment, CCC, and feedback implementation strategies result in different outcomes in resource expenditure and stakeholder engagement, and to explore the contextual forces that moderate these outcomes. METHODS: From 2017 to 2018, interviews were conducted of program directors, CCC chairs, and residents in emergency medicine (EM), internal medicine (IM), pediatrics, and family medicine (FM), querying their experiences with Milestone processes in their respective programs. Interview transcripts were coded using template analysis, with the initial template derived from previous research. The research team conducted iterative consensus meetings to ensure that the evolving template accurately represented phenomena described by interviewees. RESULTS: Forty-four individuals were interviewed across 16 programs (5 EM, 4 IM, 5 pediatrics, 3 FM). We identified 3 stages of Milestone-process implementation, including a resource-intensive early stage, an increasingly efficient transition stage, and a final stage for fine-tuning. CONCLUSIONS: Residency program leaders can use these findings to place their programs along an implementation continuum and gain an understanding of the strategies that have enabled their peers to progress to improved efficiency and increased resident and faculty engagement.
Authors: Lisa N Conforti; Nicholas A Yaghmour; Stanley J Hamstra; Eric S Holmboe; Benjamin Kennedy; Jesse J Liu; Heidi Waldo; Nathan R Selden Journal: J Surg Educ Date: 2017-06-22 Impact factor: 2.891
Authors: Laura J Damschroder; David C Aron; Rosalind E Keith; Susan R Kirsh; Jeffery A Alexander; Julie C Lowery Journal: Implement Sci Date: 2009-08-07 Impact factor: 7.327
Authors: Michael J Taylor; Chris McNicholas; Chris Nicolay; Ara Darzi; Derek Bell; Julie E Reed Journal: BMJ Qual Saf Date: 2013-09-11 Impact factor: 7.035