BACKGROUND: We previously described a feasible, reliable, valid, and acceptable clinical assessment form for basic surgical trainees (BSTs). We now recently assessed tissue-handling skills using real-time assessment (RTA) and video assessment (VA) and addressed feasibility, reliability, validity, and trainer-trainee agreement using the same assessment form. METHODS: Nine BSTs were videotaped as they performed open inguinal hernia repairs at 6 and 12 months of surgical training. Edited videotapes were independently rated by 7 consultants and 5 trainees using the technical part of the Edinburgh BST Assessment Form (EBSTAF-Tech), the Toronto Global Rating Scale of Surgical Performance (Toronto scale), and a visual analogue scale. RESULTS: RTA and VA both proved to be reliable tools (RTA alpha > or = .85; VA alpha > or = .76; interclass correlation coefficient [ICC] > or = .69, rho > or = .694 [P < or = .004]), but RTA was not construct valid. VA scores distinguished consultant from trainee operators (Mann-Whitney P < .02), with trainers able to discriminate between trainee levels (Wilcoxon P = .01 to .023). Concurrent validity was demonstrated by the VA (trainers tau-b = .71 to .79 [P < .001] and trainees tau-b = .79 to .82 [P < .001]) with good trainer-trainee agreement (EBSTAF-Tech tau-b = .35 [P = .03), Toronto tau-b = .46 [P = .006), and visual analogue scale tau-b = .46 [P = .006]). CONCLUSIONS: VA of BST tissue-handling skills is feasible, reliable, valid, and highly sensitive. It may also improve trainee self-assessment skills by promoting reflective practice.
BACKGROUND: We previously described a feasible, reliable, valid, and acceptable clinical assessment form for basic surgical trainees (BSTs). We now recently assessed tissue-handling skills using real-time assessment (RTA) and video assessment (VA) and addressed feasibility, reliability, validity, and trainer-trainee agreement using the same assessment form. METHODS: Nine BSTs were videotaped as they performed open inguinal hernia repairs at 6 and 12 months of surgical training. Edited videotapes were independently rated by 7 consultants and 5 trainees using the technical part of the Edinburgh BST Assessment Form (EBSTAF-Tech), the Toronto Global Rating Scale of Surgical Performance (Toronto scale), and a visual analogue scale. RESULTS: RTA and VA both proved to be reliable tools (RTA alpha > or = .85; VA alpha > or = .76; interclass correlation coefficient [ICC] > or = .69, rho > or = .694 [P < or = .004]), but RTA was not construct valid. VA scores distinguished consultant from trainee operators (Mann-Whitney P < .02), with trainers able to discriminate between trainee levels (Wilcoxon P = .01 to .023). Concurrent validity was demonstrated by the VA (trainers tau-b = .71 to .79 [P < .001] and trainees tau-b = .79 to .82 [P < .001]) with good trainer-trainee agreement (EBSTAF-Tech tau-b = .35 [P = .03), Toronto tau-b = .46 [P = .006), and visual analogue scale tau-b = .46 [P = .006]). CONCLUSIONS: VA of BST tissue-handling skills is feasible, reliable, valid, and highly sensitive. It may also improve trainee self-assessment skills by promoting reflective practice.
Authors: Dariusz Laski; Tomasz J Stefaniak; Wojciech Makarewicz; Monika Proczko; Zbigniew Gruca; Zbigniew Sledziński Journal: Wideochir Inne Tech Maloinwazyjne Date: 2013-03-06 Impact factor: 1.195