Roger D Dias1, William Riley2, Kenneth Shann3, Donald S Likosky4, David Fitzgerald5, Steven Yule6. 1. STRATUS Center for Medical Simulation, Brigham and Women's Hospital, Boston, Mass; Department of Emergency Medicine, Harvard Medical School, Boston, Mass. Electronic address: rdias@bwh.harvard.edu. 2. Cardiovascular Center, Tufts Medical Center, Boston, Mass. 3. Division of Cardiac Surgery, Massachusetts General Hospital, Boston, Mass. 4. Department of Cardiac Surgery, Michigan Medicine, University of Michigan, Ann Arbor, Mich. 5. College of Health Professions, Medical University of South Carolina, Charleston, SC. 6. Department of Surgery, Brigham and Women's Hospital/Harvard Medical School, Boston, Mass; Department of Clinical Surgery, University of Edinburgh, Edinburgh, Scotland.
Abstract
OBJECTIVES: This study aimed to develop the Perfusionists' Intraoperative Non-Technical Skills tool, specifically to the perfusionists' context, and test its inter-rater reliability. METHODS: An expert panel was convened to review existing surgical nontechnical skills taxonomies and develop the Perfusionists' Intraoperative Non-Technical Skills tool. During a workshop held at a national meeting, perfusionists completed the Perfusionists' Intraoperative Non-Technical Skills ratings after watching 4 videos displaying simulated cardiac operations. Two videos showed "good performance," and 2 videos showed "poor performance." Inter-rater reliability analysis was performed and intraclass correlation coefficient was reported. RESULTS: The final version of the Perfusionists' Intraoperative Non-Technical Skills taxonomy contains 4 behavioral categories (decision making, situation awareness, task management and leadership, teamwork and communication) with 4 behavioral elements each. Categories and elements are rated using an 8-point Likert scale ranging from 0.5 to 4.0. A total of 60 perfusionist raters were included and the comparison between rating distribution on "poor performance" and "good performance" videos yielded a statistically significant difference between groups, with a P value less than .001. A similar difference was found in all behavioral categories and elements. Reliability analysis showed moderate inter-rater reliability across overall ratings (intraclass correlation coefficient, 0.735; 95% confidence interval, 0.674-0.796; P < .001). Similar inter-rater reliability was found when raters were stratified by experience level. CONCLUSIONS: The Perfusionists' Intraoperative Non-Technical Skills tool presented moderate inter-rater reliability among perfusionists with varied levels of experience. This tool can be used to train and assess perfusionists in relevant nontechnical skills, with the potential to enhance safety and improve surgical outcomes.
OBJECTIVES: This study aimed to develop the Perfusionists' Intraoperative Non-Technical Skills tool, specifically to the perfusionists' context, and test its inter-rater reliability. METHODS: An expert panel was convened to review existing surgical nontechnical skills taxonomies and develop the Perfusionists' Intraoperative Non-Technical Skills tool. During a workshop held at a national meeting, perfusionists completed the Perfusionists' Intraoperative Non-Technical Skills ratings after watching 4 videos displaying simulated cardiac operations. Two videos showed "good performance," and 2 videos showed "poor performance." Inter-rater reliability analysis was performed and intraclass correlation coefficient was reported. RESULTS: The final version of the Perfusionists' Intraoperative Non-Technical Skills taxonomy contains 4 behavioral categories (decision making, situation awareness, task management and leadership, teamwork and communication) with 4 behavioral elements each. Categories and elements are rated using an 8-point Likert scale ranging from 0.5 to 4.0. A total of 60 perfusionist raters were included and the comparison between rating distribution on "poor performance" and "good performance" videos yielded a statistically significant difference between groups, with a P value less than .001. A similar difference was found in all behavioral categories and elements. Reliability analysis showed moderate inter-rater reliability across overall ratings (intraclass correlation coefficient, 0.735; 95% confidence interval, 0.674-0.796; P < .001). Similar inter-rater reliability was found when raters were stratified by experience level. CONCLUSIONS: The Perfusionists' Intraoperative Non-Technical Skills tool presented moderate inter-rater reliability among perfusionists with varied levels of experience. This tool can be used to train and assess perfusionists in relevant nontechnical skills, with the potential to enhance safety and improve surgical outcomes.
Authors: Louise Hull; Sonal Arora; Rajesh Aggarwal; Ara Darzi; Charles Vincent; Nick Sevdalis Journal: J Am Coll Surg Date: 2011-12-24 Impact factor: 6.113
Authors: Steven Yule; Rhona Flin; Nicola Maran; David Rowley; George Youngson; Simon Paterson-Brown Journal: World J Surg Date: 2008-04 Impact factor: 3.352
Authors: S Yule; A Gupta; D Gazarian; A Geraghty; D S Smink; J Beard; T Sundt; G Youngson; C McIlhenny; S Paterson-Brown Journal: Br J Surg Date: 2018-03-30 Impact factor: 6.939
Authors: Mohamad Alkhouli; Fahad Alqahtani; Ankur Kalra; Sameer Gafoor; Mohamed Alhajji; Mohammed Alreshidan; David R Holmes; Amir Lerman Journal: JAMA Netw Open Date: 2020-02-05
Authors: Robert A Baker; Shahna L Bronson; Timothy A Dickinson; David C Fitzgerald; Donald S Likosky; Nicholas B Mellas; Kenneth G Shann Journal: J Extra Corpor Technol Date: 2013-09