Patricia J Hicks1, Melissa J Margolis2, Carol L Carraccio3, Brian E Clauser2, Kathleen Donnelly4, H Barrett Fromme5, Kimberly A Gifford6, Sue E Poynter7, Daniel J Schumacher7, Alan Schwartz8. 1. a Children's Hospital of Philadelphia and University of Pennsylvania , Philadelphia , PA , USA. 2. b National Board of Medical Examiners , Philadelphia , PA , USA. 3. c American Board of Pediatrics , Chapel Hill , NC , USA. 4. d Inova Children's Hospital , Falls Church , VA , USA. 5. e Pritzker School of Medicine , University of Chicago , Chicago , IL , USA. 6. f Geisel School of Medicine at Dartmouth , Hanover , NH , USA. 7. g Pediatrics , Cincinnati Children's Hospital Medical Center/University of Cincinnati College of Medicine , Cincinnati , OH , USA. 8. h Medical Education and Pediatrics , University of Illinois at Chicago and Association of Pediatric Program Directors , Chicago , IL , USA.
Abstract
BACKGROUND: Increased recognition of the importance of competency-based education and assessment has led to the need for practical and reliable methods to assess relevant skills in the workplace. METHODS: A novel milestone-based workplace assessment system was implemented in 15 pediatrics residency programs. The system provided: (1) web-based multisource feedback (MSF) and structured clinical observation (SCO) instruments that could be completed on any computer or mobile device; and (2) monthly feedback reports that included competency-level scores and recommendations for improvement. RESULTS: For the final instruments, an average of five MSF and 3.7 SCO assessment instruments were completed for each of 292 interns; instruments required an average of 4-8 min to complete. Generalizability coefficients >0.80 were attainable with six MSF observations. Users indicated that the new system added value to their existing assessment program; the need to complete the local assessments in addition to the new assessments was identified as a burden of the overall process. CONCLUSIONS: Outcomes - including high participation rates and high reliability compared to what has traditionally been found with workplace-based assessment - provide evidence for the validity of scores resulting from this novel competency-based assessment system. The development of this assessment model is generalizable to other specialties.
BACKGROUND: Increased recognition of the importance of competency-based education and assessment has led to the need for practical and reliable methods to assess relevant skills in the workplace. METHODS: A novel milestone-based workplace assessment system was implemented in 15 pediatrics residency programs. The system provided: (1) web-based multisource feedback (MSF) and structured clinical observation (SCO) instruments that could be completed on any computer or mobile device; and (2) monthly feedback reports that included competency-level scores and recommendations for improvement. RESULTS: For the final instruments, an average of five MSF and 3.7 SCO assessment instruments were completed for each of 292 interns; instruments required an average of 4-8 min to complete. Generalizability coefficients >0.80 were attainable with six MSF observations. Users indicated that the new system added value to their existing assessment program; the need to complete the local assessments in addition to the new assessments was identified as a burden of the overall process. CONCLUSIONS: Outcomes - including high participation rates and high reliability compared to what has traditionally been found with workplace-based assessment - provide evidence for the validity of scores resulting from this novel competency-based assessment system. The development of this assessment model is generalizable to other specialties.
Authors: Cleo Keppens; Elke Boone; Paula Gameiro; Véronique Tack; Elisabeth Moreau; Elizabeth Hodges; Paul Evans; Monika Brüggemann; Ian Carter; Dido Lenze; Maria Eugenia Sarasquete; Markus Möbs; Hongxiang Liu; Elisabeth M C Dequeker; Patricia J T A Groenen Journal: Virchows Arch Date: 2021-03-08 Impact factor: 4.064
Authors: Ana G Palis; Jesús Barrio-Barrio; Eduardo P Mayorga; Ilhem Mili-Boussen; Christelle D Noche; Meenakshi Swaminathan; Karl C Golnik Journal: Indian J Ophthalmol Date: 2021-01 Impact factor: 1.848