Melody Ni1, Hugh Mackenzie1, Adam Widdison2, John T Jenkins3, Steve Mansfield4, Tony Dixon5, Dominic Slade6, Mark G Coleman7, George B Hanna8. 1. Division of Surgery, Department of Surgery and Cancer, Imperial College London, 10th Floor, QEQM, St Mary's Hospital, Praed St, London, W2 1NY, UK. 2. Department of Surgery, Royal Cornwall Hospitals Trust, Cornwall, UK. 3. Department of Surgery, St Mark's Hospital, London, UK. 4. Department of Surgery, Royal Devon and Exeter NHS Foundation Trust, Exeter, UK. 5. Department of Colorectal Surgery, North Bristol NHS Trust, Southmead Hospital, Bristol, UK. 6. Department of Surgery, Salford Royal NHS Foundation Trust, Salford, UK. 7. Department of Surgery, Plymouth Hospitals NHS Trust, Plymouth, UK. 8. Division of Surgery, Department of Surgery and Cancer, Imperial College London, 10th Floor, QEQM, St Mary's Hospital, Praed St, London, W2 1NY, UK. g.hanna@imperial.ac.uk.
Abstract
BACKGROUND: The National Training Programme for laparoscopic colorectal surgery in England was implemented to ensure training was supervised, structured, safe and effective. Delegates were required to pass a competency assessment (sign-off) before undertaking independent practice. This study described the types of errors identified and associated these errors with competency to progress to independent laparoscopic colorectal practice. METHODS: All sign-off submissions from the start of the process in January 2008 until July 2013 were included. Content analysis was used to categorise errors. Bayes factor (BF) was used to measure the impact of individual error on assessment outcome. A smaller BF indicates that an error has stronger associations with unsuccessful assessments. Bayesian network was employed to graphically represent the reasoning process whereby the chance of successful assessment diminished with the identification of each error. Quality of the error feedback was measured by the area under the ROC curve which linked the predictions from the Bayesian model to the expert verdict. RESULTS: Among 370 assessments analysed, 240 passed and 130 failed. On average, 2.5 different types of error were identified in each assessment. Cases that were more likely to fail had three or more different types of error (χ(2) = 72, p < 0.0001) and demonstrated poorer technical skills (CAT score <2.7, χ(2) = 164, p < 0.0001). Case complexity or right- versus left-sided resection did not have a significant impact. Errors associated with dissection (BF = 0.18), anastomosis (BF = 0.23) and oncological quality (BF = 0.19) were critical determinants of surgical competence, each reducing the odds of pass by at least fourfold. The area under the ROC curve was 0.84. CONCLUSIONS: Errors associated with dissection, anastomosis and oncological quality were critical determinants of surgical competency. The detailed error analysis reported in this study can guide the design of future surgical education and clinical training programmes.
BACKGROUND: The National Training Programme for laparoscopic colorectal surgery in England was implemented to ensure training was supervised, structured, safe and effective. Delegates were required to pass a competency assessment (sign-off) before undertaking independent practice. This study described the types of errors identified and associated these errors with competency to progress to independent laparoscopic colorectal practice. METHODS: All sign-off submissions from the start of the process in January 2008 until July 2013 were included. Content analysis was used to categorise errors. Bayes factor (BF) was used to measure the impact of individual error on assessment outcome. A smaller BF indicates that an error has stronger associations with unsuccessful assessments. Bayesian network was employed to graphically represent the reasoning process whereby the chance of successful assessment diminished with the identification of each error. Quality of the error feedback was measured by the area under the ROC curve which linked the predictions from the Bayesian model to the expert verdict. RESULTS: Among 370 assessments analysed, 240 passed and 130 failed. On average, 2.5 different types of error were identified in each assessment. Cases that were more likely to fail had three or more different types of error (χ(2) = 72, p < 0.0001) and demonstrated poorer technical skills (CAT score <2.7, χ(2) = 164, p < 0.0001). Case complexity or right- versus left-sided resection did not have a significant impact. Errors associated with dissection (BF = 0.18), anastomosis (BF = 0.23) and oncological quality (BF = 0.19) were critical determinants of surgical competence, each reducing the odds of pass by at least fourfold. The area under the ROC curve was 0.84. CONCLUSIONS: Errors associated with dissection, anastomosis and oncological quality were critical determinants of surgical competency. The detailed error analysis reported in this study can guide the design of future surgical education and clinical training programmes.
Authors: John D Birkmeyer; Jonathan F Finks; Amanda O'Reilly; Mary Oerline; Arthur M Carlin; Andre R Nunn; Justin Dimick; Mousumi Banerjee; Nancy J O Birkmeyer Journal: N Engl J Med Date: 2013-10-10 Impact factor: 91.245
Authors: Danilo Miskovic; Melody Ni; Susannah M Wyles; Robin H Kennedy; Nader K Francis; Amjad Parvaiz; Chris Cunningham; Timothy A Rockall; Andrew M Gudgeon; Mark G Coleman; George B Hanna Journal: Ann Surg Date: 2013-03 Impact factor: 12.969
Authors: Konstantinos Perivoliotis; Ioannis Baloyiannis; Ioannis Mamaloudis; Georgios Volakakis; Alex Valaroutsos; George Tzovaras Journal: World J Gastrointest Endosc Date: 2022-06-16