Peter Szasz1, Marisa Louridas2, Kenneth A Harris3, Teodor P Grantcharov2. 1. Department of Surgery, St. Michael's Hospital, Toronto, Canada. Electronic address: peter.szasz@utoronto.ca. 2. Department of Surgery, St. Michael's Hospital, Toronto, Canada. 3. Royal College Physicians and Surgeons of Canada, Ottawa, Canada.
Abstract
BACKGROUND: Competency-based education necessitates assessments that determine whether trainees have acquired specific competencies. The evidence on the ability of internal raters (staff surgeons) to provide accurate assessments is mixed; however, this has not yet been directly explored in the operating room. This study's objective is to compare the ratings given by internal raters vs an expert external rater (independent to the training process) in the operating room. METHODS: Raters assessed general surgery residents during a laparoscopic cholecystectomy for their technical and nontechnical performance. RESULTS: Fifteen cases were observed. There was a moderately positive correlation (rs = .618, P = .014) for technical performance and a strong positive correlation (rs = .731, P = .002) for nontechnical performance. The internal raters were less stringent for technical (mean rank 3.33 vs 8.64, P = .007) and nontechnical (mean rank 3.83 vs 8.50, P = .01) performances. CONCLUSIONS: This study provides evidence to help operationalize competency-based assessments.
BACKGROUND: Competency-based education necessitates assessments that determine whether trainees have acquired specific competencies. The evidence on the ability of internal raters (staff surgeons) to provide accurate assessments is mixed; however, this has not yet been directly explored in the operating room. This study's objective is to compare the ratings given by internal raters vs an expert external rater (independent to the training process) in the operating room. METHODS: Raters assessed general surgery residents during a laparoscopic cholecystectomy for their technical and nontechnical performance. RESULTS: Fifteen cases were observed. There was a moderately positive correlation (rs = .618, P = .014) for technical performance and a strong positive correlation (rs = .731, P = .002) for nontechnical performance. The internal raters were less stringent for technical (mean rank 3.33 vs 8.64, P = .007) and nontechnical (mean rank 3.83 vs 8.50, P = .01) performances. CONCLUSIONS: This study provides evidence to help operationalize competency-based assessments.