Susannah M Wyles1, Danilo Miskovic2, Zhifang Ni3, Ara W Darzi4, Roland M Valori5, Mark G Coleman6, George B Hanna7. 1. Department of Surgery and Cancer, St Mary's Hospital, Imperial College London, London, UK. susannah.wyles@imperial.ac.uk. 2. St James University Hospital, Leeds, UK. 3. National Training Programme, Department of Surgery and Cancer, St Mary's Hospital, Imperial College London, London, UK. 4. Department of Surgery and Cancer, St Mary's Hospital, Imperial College London, London, UK. 5. Department of Gastroenterology, Gloucestershire Hospitals NHS Foundation Trust, Gloucester, UK. 6. National Training Programme, Derriford Hospital, Plymouth, UK. 7. National Training Programme, Department of Surgery and Cancer, St Mary's Hospital, Imperial College London, Praed St, London, W2 1NY, UK. g.hanna@imperial.ac.uk.
Abstract
BACKGROUND: There is a lack of educational tools available for surgical teaching critique, particularly for advanced laparoscopic surgery. The aim was to develop and implement a tool that assesses training quality and structures feedback for trainers in the English National Training Programme for laparoscopic colorectal surgery. METHODS: Semi-structured interviews were performed and analysed, and items were extracted. Through the Delphi process, essential items pertaining to desirable trainer characteristics, training structure and feedback were determined. An assessment tool (Structured Training Trainer Assessment Report-STTAR) was developed and tested for feasibility, acceptability and educational impact. RESULTS: Interview transcripts (29 surgical trainers, 10 trainees, four educationalists) were analysed, and item lists created and distributed for consensus opinion (11 trainers and seven trainees). The STTAR consisted of 64 factors, and its web-based version, the mini-STTAR, included 21 factors that were categorised into four groups (training structure, training behaviour, trainer attributes and role modelling) and structured around a training session timeline (beginning, middle and end). The STTAR (six trainers, 48 different assessments) demonstrated good internal consistency (α = 0.88) and inter-rater reliability (ICC = 0.75). The mini-STTAR demonstrated good inter-item reliability (α = 0.79) and intra-observer reliability on comparison of 85 different trainer/trainee combinations (r = 0.701, p = <0.001). Both were found to be feasible and acceptable. The educational report for trainers was found to be useful (4.4 out of 5). CONCLUSIONS: An assessment tool that evaluates training quality was developed and shown to be reliable, acceptable and of educational value. It has been successfully implemented into the English National Training Programme for laparoscopic colorectal surgery.
BACKGROUND: There is a lack of educational tools available for surgical teaching critique, particularly for advanced laparoscopic surgery. The aim was to develop and implement a tool that assesses training quality and structures feedback for trainers in the English National Training Programme for laparoscopic colorectal surgery. METHODS: Semi-structured interviews were performed and analysed, and items were extracted. Through the Delphi process, essential items pertaining to desirable trainer characteristics, training structure and feedback were determined. An assessment tool (Structured Training Trainer Assessment Report-STTAR) was developed and tested for feasibility, acceptability and educational impact. RESULTS: Interview transcripts (29 surgical trainers, 10 trainees, four educationalists) were analysed, and item lists created and distributed for consensus opinion (11 trainers and seven trainees). The STTAR consisted of 64 factors, and its web-based version, the mini-STTAR, included 21 factors that were categorised into four groups (training structure, training behaviour, trainer attributes and role modelling) and structured around a training session timeline (beginning, middle and end). The STTAR (six trainers, 48 different assessments) demonstrated good internal consistency (α = 0.88) and inter-rater reliability (ICC = 0.75). The mini-STTAR demonstrated good inter-item reliability (α = 0.79) and intra-observer reliability on comparison of 85 different trainer/trainee combinations (r = 0.701, p = <0.001). Both were found to be feasible and acceptable. The educational report for trainers was found to be useful (4.4 out of 5). CONCLUSIONS: An assessment tool that evaluates training quality was developed and shown to be reliable, acceptable and of educational value. It has been successfully implemented into the English National Training Programme for laparoscopic colorectal surgery.
Keywords:
Assessment; Education; Feedback; Surgery; Teaching; Training
Authors: Hugh Mackenzie; Tamzin Cuming; Danilo Miskovic; Susannah M Wyles; Laura Langsford; John Anderson; Siwan Thomas-Gibson; Roland Valori; George B Hanna; Mark G Coleman; Nader Francis Journal: Ann Surg Date: 2015-01 Impact factor: 12.969
Authors: Nick Sevdalis; Rachel Davis; Mary Koutantji; Shabnam Undre; Ara Darzi; Charles A Vincent Journal: Am J Surg Date: 2008-06-16 Impact factor: 2.565
Authors: Hugh Mackenzie; Danilo Miskovic; Melody Ni; Wah-Siew Tan; Deborah S Keller; Choong-Leong Tang; Conor P Delaney; Mark G Coleman; George B Hanna Journal: Ann Surg Date: 2015-02 Impact factor: 12.969
Authors: Susannah M Wyles; Erin Schwarz; Jonathan Dort; Nabil Tariq; Tom Cecil; Mark G Coleman; John Paige; Brian J Dunkin Journal: Surg Endosc Date: 2017-03-13 Impact factor: 4.584