Nabil Hussein1, Andrew Lim2, Osami Honjo1, Christoph Haller1, John G Coles1, Glen Van Arsdell3, Shi-Joon Yoo4. 1. Division of Cardiology, Department of Paediatrics, The Hospital for Sick Children, University of Toronto, Toronto, Ontario, Canada; Division of Cardiovascular Surgery, Department of Surgery, The Hospital for Sick Children, University of Toronto, Toronto, Ontario, Canada. 2. Center for Image-Guided Innovation and Therapeutic Intervention, The Hospital for Sick Children, University of Toronto, Toronto, Ontario, Canada. 3. Department of Surgery, David Geffen School of Medicine at UCLA and UCLA Mattel Children's Hospital, Los Angeles, Calif. 4. Division of Cardiology, Department of Paediatrics, The Hospital for Sick Children, University of Toronto, Toronto, Ontario, Canada; Department of Diagnostic Imaging, The Hospital for Sick Children, University of Toronto, Toronto, Ontario, Canada. Electronic address: shi-joon.yoo@sickkids.ca.
Abstract
BACKGROUND: Hands-on surgical simulation has been sought to address training limitations within congenital heart surgery (CHS). However, there is a need for objective assessment methods to measure surgeons' performance to justify its global adoption. This study aimed to validate a procedure-specific assessment tool for the simulation of the arterial switch operation on 3D-printed models and to evaluate the consistency of scoring among evaluators with different levels of experience in CHS. METHODS: Five "expert" and 5 "junior" surgeons performed the arterial switch procedure on 3D-printed models with transposition of the great arteries during 2 hands-on surgical training courses. Their performance was retrospectively assessed by 9 evaluators with varying experience in CHS (staff surgeons, resident surgeons, and non-MD raters). Assessments were done using 2 assessment tools: the Hands-On Surgical Training-Congenital Heart Surgery (HOST-CHS) assessment tool and the global rating scale (GRS). RESULTS: The HOST-CHS tool showed a higher interrater and intrarater reliability compared with the GRS. Total scores for expert surgeons were highly consistent across all evaluators. Non-MD raters' total scores for junior surgeons were slightly higher than those of residents and staff evaluators. All grades of evaluator were able to discriminate between junior and expert surgeons. CONCLUSIONS: This study demonstrates the development and validation of an objective, procedure-specific assessment tool for the arterial switch operation with consistency among evaluators with different experience. There is now a platform for quantifying and accurately evaluating performance, which will be highly beneficial in training and developing the next generation of congenital heart surgeons.
BACKGROUND: Hands-on surgical simulation has been sought to address training limitations within congenital heart surgery (CHS). However, there is a need for objective assessment methods to measure surgeons' performance to justify its global adoption. This study aimed to validate a procedure-specific assessment tool for the simulation of the arterial switch operation on 3D-printed models and to evaluate the consistency of scoring among evaluators with different levels of experience in CHS. METHODS: Five "expert" and 5 "junior" surgeons performed the arterial switch procedure on 3D-printed models with transposition of the great arteries during 2 hands-on surgical training courses. Their performance was retrospectively assessed by 9 evaluators with varying experience in CHS (staff surgeons, resident surgeons, and non-MD raters). Assessments were done using 2 assessment tools: the Hands-On Surgical Training-Congenital Heart Surgery (HOST-CHS) assessment tool and the global rating scale (GRS). RESULTS: The HOST-CHS tool showed a higher interrater and intrarater reliability compared with the GRS. Total scores for expert surgeons were highly consistent across all evaluators. Non-MD raters' total scores for junior surgeons were slightly higher than those of residents and staff evaluators. All grades of evaluator were able to discriminate between junior and expert surgeons. CONCLUSIONS: This study demonstrates the development and validation of an objective, procedure-specific assessment tool for the arterial switch operation with consistency among evaluators with different experience. There is now a platform for quantifying and accurately evaluating performance, which will be highly beneficial in training and developing the next generation of congenital heart surgeons.
Authors: Jose Antonio Calvo-Haro; Javier Pascau; José Manuel Asencio-Pascual; Felipe Calvo-Manuel; Maria José Cancho-Gil; Juan Francisco Del Cañizo López; María Fanjul-Gómez; Roberto García-Leal; Guillermo González-Casaurrán; Manuel González-Leyte; Juan Antonio León-Luis; Lydia Mediavilla-Santos; Santiago Ochandiano-Caicoya; Ramón Pérez-Caballero; Almudena Ribed-Sánchez; Javier Río-Gómez; Eduardo Sánchez-Pérez; Javier Serrano-Andreu; Manuel Tousidonis-Rial; Javier Vaquero-Martín; Sonia García San José; Rubén Perez-Mañanes Journal: 3D Print Med Date: 2021-04-22