Yang Lu1, Ruth Miranda2, Chi Quach1, Mark Girgis1, Catherine E Lewis1, Areti Tillou1, Formosa Chen3. 1. Department of Surgery, Ronald Reagan UCLA Medical Center, Los Angeles, California. 2. David Geffen UCLA School of Medicine, Los Angeles, California. 3. Department of Surgery, Ronald Reagan UCLA Medical Center, Los Angeles, California. Electronic address: Fochen@mednet.ucla.edu.
Abstract
INTRODUCTION: Mock oral examinations (MOE) are used to prepare residents and assess their readiness for the American Board of Surgery Certifying Exam (ABSCE). Delivery of MOEs varies by institution and previous studies have demonstrated significant implementation barriers such as availability of faculty examiners and exam scenarios. OBJECTIVE: To assess the value and participant satisfaction of a standardized multi-institutional MOE for general surgery residents. PARTICIPANTS: Thirty-three general surgery residents and 37 faculty members from 3 institutions participated in a regional MOE. Residents were examined in three 20-minute sessions. Faculty examiners were given a wide selection of prescripted exam scenarios and instructed to use standardized grading rubrics during a brief orientation on the day of the exam. All participants were surveyed on their overall experience. RESULTS: Of 33 participating residents, 26 (79%) passed the MOE (92% of R5, 91% R4, and 50% of R3). Response rates were 91% for residents, and 57% for faculty members respectively. Most respondents were satisfied with the overall exam experience (88%), standardized question quality (86%) and question variety (82%). A total of 92% of respondents agreed that the time, effort, and cost of the MOE was justified by its educational value to residents. Higher medical knowledge ratings assigned by faculty examiners correlated with stronger trainee performance (β = 0.48; 95% confidence interval [CI] 0.29-0.66), while patient care and interpersonal communication skill ratings were not associated with trainee performance. The standardized grading rubric achieved moderate inter-rater reliability among examiner pairs with 70.6% agreement (Kappa 0.47). CONCLUSIONS: General Surgery residents and faculty perceived the standardized multi-institutional MOE to be a highly satisfactory educational experience and valuable assessment tool. Developing a repertoire of scripted exam scenarios made it feasible to recruit sufficient faculty participants, and standardizing grading rubrics allowed for a consistent exam experience with moderate inter-rater reliability.
INTRODUCTION: Mock oral examinations (MOE) are used to prepare residents and assess their readiness for the American Board of Surgery Certifying Exam (ABSCE). Delivery of MOEs varies by institution and previous studies have demonstrated significant implementation barriers such as availability of faculty examiners and exam scenarios. OBJECTIVE: To assess the value and participant satisfaction of a standardized multi-institutional MOE for general surgery residents. PARTICIPANTS: Thirty-three general surgery residents and 37 faculty members from 3 institutions participated in a regional MOE. Residents were examined in three 20-minute sessions. Faculty examiners were given a wide selection of prescripted exam scenarios and instructed to use standardized grading rubrics during a brief orientation on the day of the exam. All participants were surveyed on their overall experience. RESULTS: Of 33 participating residents, 26 (79%) passed the MOE (92% of R5, 91% R4, and 50% of R3). Response rates were 91% for residents, and 57% for faculty members respectively. Most respondents were satisfied with the overall exam experience (88%), standardized question quality (86%) and question variety (82%). A total of 92% of respondents agreed that the time, effort, and cost of the MOE was justified by its educational value to residents. Higher medical knowledge ratings assigned by faculty examiners correlated with stronger trainee performance (β = 0.48; 95% confidence interval [CI] 0.29-0.66), while patient care and interpersonal communication skill ratings were not associated with trainee performance. The standardized grading rubric achieved moderate inter-rater reliability among examiner pairs with 70.6% agreement (Kappa 0.47). CONCLUSIONS: General Surgery residents and faculty perceived the standardized multi-institutional MOE to be a highly satisfactory educational experience and valuable assessment tool. Developing a repertoire of scripted exam scenarios made it feasible to recruit sufficient faculty participants, and standardizing grading rubrics allowed for a consistent exam experience with moderate inter-rater reliability.
Keywords:
American Board of Surgery Certifying Exam; General surgery, mock orals; Interpersonal and communication skills; Mock oral examination; Surgical education
Authors: Ivy A Huang; Yang Lu; Justin P Wagner; Chi Quach; Timothy R Donahue; Areti Tillou; Formosa Chen; James Wu Journal: Am J Surg Date: 2020-11-13 Impact factor: 2.565