BACKGROUND: There are no valid and reliable tools to assess competency in advanced laparoscopic surgery at a specialist level. The observational clinical human reliability analysis (OCHRA) may have the required characteristics of such a tool. The aim of this study was to evaluate construct and concurrent validity of OCHRA for competency assessment at a specialist level. METHODS: Thirty-two video-recorded laparoscopic colorectal resections, performed by experts and delegates of the National Training Program in England, were evaluated. Each video was analysed using OCHRA by identifying errors enacted during surgery. The number of tissue-handling, instrument-misuse, and consequential errors was recorded using video-rating software. Times spent on dissecting (D) and on exposing (E) tissues were also measured (D/E ratio). In addition, two independent expert surgeons globally assessed each video regarding competency (pass vs. fail). Logistic regression was used to predict outcomes. RESULTS: A total of 399 errors were identified. There was a significant difference when comparing the expert, pass, and fail groups for total errors (median counts for experts = 4, pass = 10, fail = 17; P < 0.001). When comparing the pass and fail groups excluding experts, differences could be found for tissue-handling errors (7 vs. 12; P = 0.005), but not for consequential errors (4 vs. 7; P = 0.059) and instrument-handling errors (4 vs. 5; P = 0.320). The D/E ratio was significantly lower for delegates than for experts (0.6 vs. 1.0; P = 0.001). When all four independent variables were used to predict delegates who passed or failed, the area under the receiver operating characteristic curve was 0.867, sensitivity was 71.4%, and specificity was 90.9%. CONCLUSION: OCHRA is a valid tool for assessing competency at a specialist level in advanced laparoscopic surgery. It has the potential to be used for recertification and revalidation of specialists.
BACKGROUND: There are no valid and reliable tools to assess competency in advanced laparoscopic surgery at a specialist level. The observational clinical human reliability analysis (OCHRA) may have the required characteristics of such a tool. The aim of this study was to evaluate construct and concurrent validity of OCHRA for competency assessment at a specialist level. METHODS: Thirty-two video-recorded laparoscopic colorectal resections, performed by experts and delegates of the National Training Program in England, were evaluated. Each video was analysed using OCHRA by identifying errors enacted during surgery. The number of tissue-handling, instrument-misuse, and consequential errors was recorded using video-rating software. Times spent on dissecting (D) and on exposing (E) tissues were also measured (D/E ratio). In addition, two independent expert surgeons globally assessed each video regarding competency (pass vs. fail). Logistic regression was used to predict outcomes. RESULTS: A total of 399 errors were identified. There was a significant difference when comparing the expert, pass, and fail groups for total errors (median counts for experts = 4, pass = 10, fail = 17; P < 0.001). When comparing the pass and fail groups excluding experts, differences could be found for tissue-handling errors (7 vs. 12; P = 0.005), but not for consequential errors (4 vs. 7; P = 0.059) and instrument-handling errors (4 vs. 5; P = 0.320). The D/E ratio was significantly lower for delegates than for experts (0.6 vs. 1.0; P = 0.001). When all four independent variables were used to predict delegates who passed or failed, the area under the receiver operating characteristic curve was 0.867, sensitivity was 71.4%, and specificity was 90.9%. CONCLUSION: OCHRA is a valid tool for assessing competency at a specialist level in advanced laparoscopic surgery. It has the potential to be used for recertification and revalidation of specialists.
Authors: M C Vassiliou; G A Ghitulescu; L S Feldman; D Stanbridge; K Leffondré; H H Sigman; G M Fried Journal: Surg Endosc Date: 2006-02-27 Impact factor: 4.584
Authors: Ahmed R Ahmed; Danilo Miskovic; Thormela Vijayaseelan; William O'Malley; George B Hanna Journal: Surg Obes Relat Dis Date: 2011-01-14 Impact factor: 4.734
Authors: Melody Ni; Hugh Mackenzie; Adam Widdison; John T Jenkins; Steve Mansfield; Tony Dixon; Dominic Slade; Mark G Coleman; George B Hanna Journal: Surg Endosc Date: 2015-06-23 Impact factor: 4.584
Authors: Hugh Mackenzie; Danilo Miskovic; Melody Ni; Amjad Parvaiz; Austin G Acheson; John T Jenkins; John Griffith; Mark G Coleman; George B Hanna Journal: Surg Endosc Date: 2013-02-08 Impact factor: 4.584
Authors: J D Foster; P Ewings; S Falk; E J Cooper; H Roach; N P West; B A Williams-Yesson; G B Hanna; N K Francis Journal: Tech Coloproctol Date: 2016-08-10 Impact factor: 3.781