Lydia Weyenberg1, Ronald J Prince2, Ann Evensen2. 1. University of Wisconsin School of Medicine and Public Health. 2. University of Wisconsin School of Medicine and Public Health, Department of Family Medicine and Community Health.
Abstract
INTRODUCTION: Pelvic examination training and competency-based assessment are expensive and time consuming. Our goal was to use the results of skill evaluation early in residency to identify residents who required training. METHODS: Incoming residents performed pelvic examinations with gynecological teaching assistants. Faculty observed residents performing examinations with clinic patients to assess for competency. Written assessment of residents by teaching assistants and faculty were completed. A regression-based software tool was used to determine items in early resident performance to best predict subsequent competency. RESULTS: Sixty-eight residents were evaluated. Thirty-eight (56%) residents were not able to demonstrate competency in three clinical exams and therefore received more observation. One third of evaluations were completed by faculty performing ≤1 evaluation per year. Two items were found most likely to identify a resident who required further training ("identifies cervix" and "properly assembles equipment"). The model based on these items had a sensitivity of 100% (95% confidence interval [CI]=86-100%) and specificity of 36.8% (95% CI=22-54%). CONCLUSIONS: A model using early skills assessments was not sufficiently specific to identify residents who needed more training. Next steps include limiting the number of faculty assessors, faculty development to improve discriminatory capacity, and creating separate processes for (1) providing feedback and identifying learning needs in new residents, and (2) documenting competency in performance of pelvic exams.
INTRODUCTION: Pelvic examination training and competency-based assessment are expensive and time consuming. Our goal was to use the results of skill evaluation early in residency to identify residents who required training. METHODS: Incoming residents performed pelvic examinations with gynecological teaching assistants. Faculty observed residents performing examinations with clinic patients to assess for competency. Written assessment of residents by teaching assistants and faculty were completed. A regression-based software tool was used to determine items in early resident performance to best predict subsequent competency. RESULTS: Sixty-eight residents were evaluated. Thirty-eight (56%) residents were not able to demonstrate competency in three clinical exams and therefore received more observation. One third of evaluations were completed by faculty performing ≤1 evaluation per year. Two items were found most likely to identify a resident who required further training ("identifies cervix" and "properly assembles equipment"). The model based on these items had a sensitivity of 100% (95% confidence interval [CI]=86-100%) and specificity of 36.8% (95% CI=22-54%). CONCLUSIONS: A model using early skills assessments was not sufficiently specific to identify residents who needed more training. Next steps include limiting the number of faculty assessors, faculty development to improve discriminatory capacity, and creating separate processes for (1) providing feedback and identifying learning needs in new residents, and (2) documenting competency in performance of pelvic exams.
Authors: Kirsten Bibbins-Domingo; David C Grossman; Susan J Curry; Michael J Barry; Karina W Davidson; Chyke A Doubeni; John W Epling; Francisco A R García; Alex R Kemper; Alex H Krist; Ann E Kurth; C Seth Landefeld; Carol M Mangione; William R Phillips; Maureen G Phipps; Michael Silverstein; Melissa Simon; Albert L Siu; Chien-Wen Tseng Journal: JAMA Date: 2017-03-07 Impact factor: 56.272