Yara Mikhaeil-Demo1, Eric Holmboe2, Elizabeth E Gerard3, Diane B Wayne4, Elaine R Cohen5, Kenji Yamazaki6, Jessica W Templer7, Danny Bega8, George W Culler9, Amar B Bhatt10, Neelofer Shafi11, Jeffrey H Barsuk12. 1. is Assistant Professor, Department of Neurology, Northwestern University, Feinberg School of Medicine. 2. is Chief Research, Milestone Development, and Evaluation Officer, Accreditation Council for Graduate Medical Education (ACGME). 3. is Director, Clinical Neurophysiology Fellowship, and Associate Professor, Department of Neurology, Northwestern University, Feinberg School of Medicine. 4. is Vice Dean for Education, Chair, Department of Medical Education, and Professor of Medicine and Medical Education, Northwestern University, Feinberg School of Medicine. 5. is Research Associate, Department of Medicine, Northwestern University, Feinberg School of Medicine. 6. is Senior Analyst, Milestones Research and Evaluation, ACGME. 7. is Director, Epilepsy Fellowship, and Assistant Professor, Department of Neurology, Northwestern University, Feinberg School of Medicine. 8. is Director, Neurology Residency Program, and Assistant Professor, Department of Neurology, Northwestern University, Feinberg School of Medicine. 9. is Epilepsy Fellow, Department of Neurology, Northwestern University, Feinberg School of Medicine. 10. is Assistant Professor, Department of Neurological Sciences, Rush University. 11. is Director, Students and Faculty Development, and Assistant Professor, Department of Neurology and Rehabilitation, University of Illinois Chicago. 12. is Director, Simulation and Patient Safety, and Professor of Medicine and Medical Education, Northwestern University, Feinberg School of Medicine.
Abstract
BACKGROUND: The American Board of Psychiatry and Neurology and the Accreditation Council for Graduate Medical Education (ACGME) developed Milestones that provide a framework for residents' assessment. However, Milestones do not provide a description for how programs should perform assessments. OBJECTIVES: We evaluated graduating residents' status epilepticus (SE) identification and management skills and how they correlate with ACGME Milestones reported for epilepsy and management/treatment by their program's clinical competency committee (CCC). METHODS: We performed a cohort study of graduating neurology residents from 3 academic medical centers in Chicago in 2018. We evaluated residents' skills identifying and managing SE using a simulation-based assessment (26-item checklist). Simulation-based assessment scores were compared to experience (number of SE cases each resident reported identifying and managing during residency), self-confidence in identifying and managing these cases, and their end of residency Milestones assigned by a CCC based on end-of-rotation evaluations. RESULTS: Sixteen of 21 (76%) eligible residents participated in the study. Average SE checklist score was 15.6 of 26 checklist items correct (60%, SD 12.2%). There were no significant correlations between resident checklist performance and experience or self-confidence. The average participant's level of Milestone for epilepsy and management/treatment was high at 4.3 of 5 (SD 0.4) and 4.4 of 5 (SD 0.4), respectively. There were no significant associations between checklist skills performance and level of Milestone assigned. CONCLUSIONS: Simulated SE skills performance of graduating neurology residents was poor. Our study suggests that end-of-rotation evaluations alone are inadequate for assigning Milestones for high-stakes clinical skills such as identification and management of SE.
BACKGROUND: The American Board of Psychiatry and Neurology and the Accreditation Council for Graduate Medical Education (ACGME) developed Milestones that provide a framework for residents' assessment. However, Milestones do not provide a description for how programs should perform assessments. OBJECTIVES: We evaluated graduating residents' status epilepticus (SE) identification and management skills and how they correlate with ACGME Milestones reported for epilepsy and management/treatment by their program's clinical competency committee (CCC). METHODS: We performed a cohort study of graduating neurology residents from 3 academic medical centers in Chicago in 2018. We evaluated residents' skills identifying and managing SE using a simulation-based assessment (26-item checklist). Simulation-based assessment scores were compared to experience (number of SE cases each resident reported identifying and managing during residency), self-confidence in identifying and managing these cases, and their end of residency Milestones assigned by a CCC based on end-of-rotation evaluations. RESULTS: Sixteen of 21 (76%) eligible residents participated in the study. Average SE checklist score was 15.6 of 26 checklist items correct (60%, SD 12.2%). There were no significant correlations between resident checklist performance and experience or self-confidence. The average participant's level of Milestone for epilepsy and management/treatment was high at 4.3 of 5 (SD 0.4) and 4.4 of 5 (SD 0.4), respectively. There were no significant associations between checklist skills performance and level of Milestone assigned. CONCLUSIONS: Simulated SE skills performance of graduating neurology residents was poor. Our study suggests that end-of-rotation evaluations alone are inadequate for assigning Milestones for high-stakes clinical skills such as identification and management of SE.
Authors: Diane B Wayne; John Butter; Viva J Siddall; Monica J Fudala; Leonard D Wade; Joe Feinglass; William C McGaghie Journal: Med Teach Date: 2006-06 Impact factor: 3.650
Authors: Yara Mikhaeil-Demo; Jeffrey H Barsuk; George W Culler; Danny Bega; David H Salzman; Elaine R Cohen; Jessica W Templer; Elizabeth E Gerard Journal: Epilepsy Behav Date: 2020-06-27 Impact factor: 2.937
Authors: Jeffrey H Barsuk; Elaine R Cohen; John A Vozenilek; Lanty M O'Connor; William C McGaghie; Diane B Wayne Journal: J Grad Med Educ Date: 2012-03
Authors: Jeffrey H Barsuk; Elaine R Cohen; Duyhuu Nguyen; Debi Mitra; Kelly O'Hara; Yasuharu Okuda; Joe Feinglass; Kenzie A Cameron; William C McGaghie; Diane B Wayne Journal: Crit Care Med Date: 2016-10 Impact factor: 7.598