Roy Phitayakorn1, Rebecca Minehart2, May C M Pian-Smith2, Maureen W Hemingway3, Tanya Milosh-Zinkus4, Danika Oriol-Morway4, Emil Petrusa5. 1. Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts; MGH Learning Laboratory, Massachusetts General Hospital, Boston, Massachusetts. Electronic address: rphitayakorn@mgh.harvard.edu. 2. MGH Learning Laboratory, Massachusetts General Hospital, Boston, Massachusetts; Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts. 3. Department of Perioperative Services, Massachusetts General Hospital, Boston, Massachusetts; Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts. 4. MGH Learning Laboratory, Massachusetts General Hospital, Boston, Massachusetts. 5. Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts; MGH Learning Laboratory, Massachusetts General Hospital, Boston, Massachusetts.
Abstract
BACKGROUND: High-quality teamwork among operating room (OR) professionals is a key to efficient and safe practice. Quantification of teamwork facilitates feedback, assessment, and improvement. Several valid and reliable instruments are available for assessing separate OR disciplines and teams. We sought to determine the most feasible approach for routine documentation of teamwork in in-situ OR simulations. We compared rater agreement, hypothetical training costs, and feasibility ratings from five clinicians and two nonclinicians with instruments for assessment of separate OR groups and teams. MATERIALS AND METHODS: Five teams of anesthesia or surgery residents and OR nurses (RN) or surgical technicians were videotaped in simulations of an epigastric hernia repair where the patient develops malignant hyperthermia. Two anesthesiologists, one OR clinical RN specialist, one educational psychologist, one simulation specialist, and one general surgeon discussed and then independently completed Anesthesiologists' Non-Technical Skills, Non-Technical Skills for Surgeons, Scrub Practitioners' List of Intraoperative Non-Technical Skills, and Observational Teamwork Assessment for Surgery forms to rate nontechnical performance of anesthesiologists, surgeons, nurses, technicians, and the whole team. RESULTS: Intraclass correlations of agreement ranged from 0.17-0.85. Clinicians' agreements were not different from nonclinicians'. Published rater training was 4 h for Anesthesiologists' Non-Technical Skills and Scrub Practitioners' List of Intraoperative Non-Technical Skills, 2.5 h for Non-Technical Skills for Surgeons, and 15.5 h for Observational Teamwork Assessment for Surgery. Estimated costs to train one rater to use all instruments ranged from $442 for a simulation specialist to $6006 for a general surgeon. CONCLUSIONS: Additional training is needed to achieve higher levels of agreement; however, costs may be prohibitive. The most cost-effective model for real-time OR teamwork assessment may be to use a simulation technician combined with one clinical rater to allow complete documentation of all participants.
BACKGROUND: High-quality teamwork among operating room (OR) professionals is a key to efficient and safe practice. Quantification of teamwork facilitates feedback, assessment, and improvement. Several valid and reliable instruments are available for assessing separate OR disciplines and teams. We sought to determine the most feasible approach for routine documentation of teamwork in in-situ OR simulations. We compared rater agreement, hypothetical training costs, and feasibility ratings from five clinicians and two nonclinicians with instruments for assessment of separate OR groups and teams. MATERIALS AND METHODS: Five teams of anesthesia or surgery residents and OR nurses (RN) or surgical technicians were videotaped in simulations of an epigastric hernia repair where the patient develops malignant hyperthermia. Two anesthesiologists, one OR clinical RN specialist, one educational psychologist, one simulation specialist, and one general surgeon discussed and then independently completed Anesthesiologists' Non-Technical Skills, Non-Technical Skills for Surgeons, Scrub Practitioners' List of Intraoperative Non-Technical Skills, and Observational Teamwork Assessment for Surgery forms to rate nontechnical performance of anesthesiologists, surgeons, nurses, technicians, and the whole team. RESULTS: Intraclass correlations of agreement ranged from 0.17-0.85. Clinicians' agreements were not different from nonclinicians'. Published rater training was 4 h for Anesthesiologists' Non-Technical Skills and Scrub Practitioners' List of Intraoperative Non-Technical Skills, 2.5 h for Non-Technical Skills for Surgeons, and 15.5 h for Observational Teamwork Assessment for Surgery. Estimated costs to train one rater to use all instruments ranged from $442 for a simulation specialist to $6006 for a general surgeon. CONCLUSIONS: Additional training is needed to achieve higher levels of agreement; however, costs may be prohibitive. The most cost-effective model for real-time OR teamwork assessment may be to use a simulation technician combined with one clinical rater to allow complete documentation of all participants.
Keywords:
Feasibility of OR team assessment; Nontechnical OR performance; OR teamwork assessment; OR teamwork assessment tools; Transprofessional/transdisciplinary OR simulations