OBJECTIVE: To evaluate a method to assess surgical skills of veterinary students that is based on digital recording of their performance during closure of a celiotomy in canine cadavers. SAMPLE POPULATION: Second year veterinary students without prior experience with live animal or simulated surgical procedure (n = 19) METHODS: Each student completed a 3-layer closure of a celiotomy on a canine cadaver. Each procedure was digitally recorded with a single small wide-angle camera mounted to the overhead surgical light. The performance was scored by 2 of 5 trained raters who were unaware of the identity of the students. Scores were based on an 8-item rubric that was created to evaluate surgical skills that are required to close a celiotomy. The reliability of scores was tested with Cronbach's α, intraclass correlation, and a generalizability study. RESULTS: The internal consistency of the grading rubric, as measured by α, was .76. Interrater reliability, as measured by intraclass correlation, was 0.64. The generalizability coefficient was 0.56. CONCLUSION: Reliability measures of 0.60 and above have been suggested as adequate to assess low-stakes skills. The task-specific grading rubric used in this study to evaluate veterinary surgical skills captured by a single wide-angle camera mounted to an overhead surgical light produced scores with acceptable internal consistency, substantial interrater reliability, and marginal generalizability. IMPACT: Evaluation of veterinary students' surgical skills by using digital recordings with a validated rubric improves flexibility when designing accurate assessments.
OBJECTIVE: To evaluate a method to assess surgical skills of veterinary students that is based on digital recording of their performance during closure of a celiotomy in canine cadavers. SAMPLE POPULATION: Second year veterinary students without prior experience with live animal or simulated surgical procedure (n = 19) METHODS: Each student completed a 3-layer closure of a celiotomy on a canine cadaver. Each procedure was digitally recorded with a single small wide-angle camera mounted to the overhead surgical light. The performance was scored by 2 of 5 trained raters who were unaware of the identity of the students. Scores were based on an 8-item rubric that was created to evaluate surgical skills that are required to close a celiotomy. The reliability of scores was tested with Cronbach's α, intraclass correlation, and a generalizability study. RESULTS: The internal consistency of the grading rubric, as measured by α, was .76. Interrater reliability, as measured by intraclass correlation, was 0.64. The generalizability coefficient was 0.56. CONCLUSION: Reliability measures of 0.60 and above have been suggested as adequate to assess low-stakes skills. The task-specific grading rubric used in this study to evaluate veterinary surgical skills captured by a single wide-angle camera mounted to an overhead surgical light produced scores with acceptable internal consistency, substantial interrater reliability, and marginal generalizability. IMPACT: Evaluation of veterinary students' surgical skills by using digital recordings with a validated rubric improves flexibility when designing accurate assessments.
Authors: Robin M Farrell; Gregory E Gilbert; Larry Betance; Jennifer Huck; Julie A Hunt; James Dundas; Eric Pope Journal: Vet Surg Date: 2022-03-08 Impact factor: 1.618