Steven E Swift1, James F Carter. 1. Department of Obstetrics and Gynecology, Medical University of South Carolina, Charleston, SC, USA.
Abstract
OBJECTIVE: This study was undertaken to further evaluate the construct validity of the observed structured assessment of technical skills (OSATS) by comparing resident scores to faculty scores. STUDY DESIGN: This study is a prospective blinded observational study. Four residents from each year (1-4) and 5 faculty members were examined. The OSATS examination was in the form of a dry laboratory with 10 stations: 4 laparoscopic and 6 open surgical skills. The sessions were videotaped and graded by the senior authors who used a task-specific checklist. The scoring of the videos was performed by the 2 senior authors blinded to the other examiners results. The examinations were videotaped, and the identity of the participants was blinded. The scores for each station were determined by adding all the numbers from the skills rating with a time score. Higher scores denote superior performance. Statistical analysis was performed with a nonparametric test (Kruskal-Wallis) for a total score with resident years divided into junior residents (first and second year), senior residents (third and fourth year), and faculty status as the independent variable. To determine the interrelater reliability between the 2 scores, the Kendall tau beta statistic was used. RESULTS: The results show definite trends for 7 stations, with junior residents performing the worst and faculty performing the best. This trend was statistically significant for 6 of the 10 stations. CONCLUSION: The OSATS examination has good construct validity that extends beyond residency to faculty.
OBJECTIVE: This study was undertaken to further evaluate the construct validity of the observed structured assessment of technical skills (OSATS) by comparing resident scores to faculty scores. STUDY DESIGN: This study is a prospective blinded observational study. Four residents from each year (1-4) and 5 faculty members were examined. The OSATS examination was in the form of a dry laboratory with 10 stations: 4 laparoscopic and 6 open surgical skills. The sessions were videotaped and graded by the senior authors who used a task-specific checklist. The scoring of the videos was performed by the 2 senior authors blinded to the other examiners results. The examinations were videotaped, and the identity of the participants was blinded. The scores for each station were determined by adding all the numbers from the skills rating with a time score. Higher scores denote superior performance. Statistical analysis was performed with a nonparametric test (Kruskal-Wallis) for a total score with resident years divided into junior residents (first and second year), senior residents (third and fourth year), and faculty status as the independent variable. To determine the interrelater reliability between the 2 scores, the Kendall tau beta statistic was used. RESULTS: The results show definite trends for 7 stations, with junior residents performing the worst and faculty performing the best. This trend was statistically significant for 6 of the 10 stations. CONCLUSION: The OSATS examination has good construct validity that extends beyond residency to faculty.
Authors: Luc Joyeux; Allan Javaux; Mary P Eastwood; Felix R De Bie; Gert Van den Bergh; Rebecca S Degliuomini; Simen Vergote; Talita Micheletti; Geertje Callewaert; Sebastien Ourselin; Paolo De Coppi; Frank Van Calenbergh; Emmanuel Vander Poorten; Jan Deprest Journal: Sci Rep Date: 2021-03-17 Impact factor: 4.379