Janelle Rekman1, Stanley J Hamstra2, Nancy Dudek3, Timothy Wood4, Christine Seabrook5, Wade Gofton5. 1. Department of Surgical Education, The University of Ottawa, Ottawa, Ontario, Canada. Electronic address: jrekman@toh.on.ca. 2. Milestones Research and Evaluation at the Accreditation Council for Graduate Medical Education, Chicago, Illinois. 3. Department of Medicine, The Ottawa Hospital Rehabilitation Center, The University of Ottawa, Ottawa, Ontario, Canada. 4. Department of Innovation in Medical Education, University of Ottawa, Ottawa, Ontario, Canada. 5. Department of Surgical Education, The University of Ottawa, Ottawa, Ontario, Canada.
Abstract
BACKGROUND: The shift toward competency-based medical education has created a demand for feasible workplace-based assessment tools. Perhaps, more important than competence to assess an individual patient is the ability to successfully manage a surgical clinic. Trainee performance in clinic is a critical component of learning to manage a surgical practice, yet no assessment tool currently exists to assess daily performance in outpatient clinics for surgery residents. The development of a competency-based assessment tool, the Ottawa Clinic Assessment Tool (OCAT), is described here to address this gap. STUDY DESIGN: A consensus group of experts was gathered to generate dimensions of performance reflective of a competent "generalist" surgeon in clinic. A 6-month pilot study of the OCAT was conducted in orthopedics, general surgery, and obstetrics and gynecology with quantitative and qualitative evidence of validity collected. In all, 2 subsequent feedback sessions and a survey for staff and residents evaluated the OCAT for clarity and utility. RESULTS: The OCAT is a 9-item tool, with a global assessment item and 2 short-answer questions. Among the 2 divisions, 44 staff surgeons completed 132 OCAT assessments of 79 residents. Psychometric data was collected as evidence of validity. Analysis of feedback indicated that the entrustability rating scale was useful for surgeons and residents and that the items could be correlated with individual competencies. CONCLUSIONS: Multiple sources of validity evidence collected in this study demonstrate that the OCAT can measure resident clinic competency in a valid and feasible manner.
BACKGROUND: The shift toward competency-based medical education has created a demand for feasible workplace-based assessment tools. Perhaps, more important than competence to assess an individual patient is the ability to successfully manage a surgical clinic. Trainee performance in clinic is a critical component of learning to manage a surgical practice, yet no assessment tool currently exists to assess daily performance in outpatient clinics for surgery residents. The development of a competency-based assessment tool, the Ottawa Clinic Assessment Tool (OCAT), is described here to address this gap. STUDY DESIGN: A consensus group of experts was gathered to generate dimensions of performance reflective of a competent "generalist" surgeon in clinic. A 6-month pilot study of the OCAT was conducted in orthopedics, general surgery, and obstetrics and gynecology with quantitative and qualitative evidence of validity collected. In all, 2 subsequent feedback sessions and a survey for staff and residents evaluated the OCAT for clarity and utility. RESULTS: The OCAT is a 9-item tool, with a global assessment item and 2 short-answer questions. Among the 2 divisions, 44 staff surgeons completed 132 OCAT assessments of 79 residents. Psychometric data was collected as evidence of validity. Analysis of feedback indicated that the entrustability rating scale was useful for surgeons and residents and that the items could be correlated with individual competencies. CONCLUSIONS: Multiple sources of validity evidence collected in this study demonstrate that the OCAT can measure resident clinic competency in a valid and feasible manner.
Authors: Daniel J Schumacher; Beth King; Michelle M Barnes; Sean P Elliott; Kathleen Gibbs; Jon F McGreevy; Javier Gonzalez Del Rey; Tanvi Sharma; Catherine Michelson; Alan Schwartz Journal: J Grad Med Educ Date: 2018-08
Authors: Jennie B Jarrett; Kristen L Goliak; Stuart T Haines; Elizabeth Trolli; Alan Schwartz Journal: Am J Pharm Educ Date: 2021-07-22 Impact factor: 2.047