Jason C Pradarelli1, Emily George2, Jane Kavanagh2, Yves Sonnay2, Tan Hiang Khoon3, Joaquim M Havens4. 1. Ariadne Labs at Brigham and Women's Hospital and the Harvard T.H. Chan School of Public Health, Boston, Massachusetts; Department of Surgery, Brigham and Women's Hospital, Boston, Massachusetts. Electronic address: jpradarelli@partners.org. 2. Ariadne Labs at Brigham and Women's Hospital and the Harvard T.H. Chan School of Public Health, Boston, Massachusetts. 3. Division of Surgery and Surgical Oncology, Singapore General Hospital, Singapore. 4. Ariadne Labs at Brigham and Women's Hospital and the Harvard T.H. Chan School of Public Health, Boston, Massachusetts; Department of Surgery, Brigham and Women's Hospital, Boston, Massachusetts.
Abstract
OBJECTIVE: To our knowledge, no curricula have been described for training novice, nonclinician raters of nontechnical skills in the operating room (OR). We aimed to report the reliability of Oxford Non-Technical Skills (NOTECHS) ratings provided by novice raters who underwent a scalable curriculum for learning to assess nontechnical skills of OR teams. DESIGN: In-person training course to apply the NOTECHS framework to assessing OR teams' nontechnical skill performance, led by 2 facilitators and involving 5 partial-day sessions of didactic presentations, video simulation, and live OR observation with postassessment debriefing. NOTECHS ratings were submitted after each of 11 video scenarios and 8 live operations for the total NOTECHS team rating (including surgical/anesthesiology/nursing subteams) and for each NOTECHS skill category-situation awareness, problem solving and decision making, teamwork and cooperation, leadership and management. Inter-rater reliability was determined by calculating the intraclass correlation coefficient (ICC, range 0-1). SETTING: Training for outcome measurement during a quality improvement initiative focused on surgical safety in 3 public hospitals in Singapore. Two trainings were conducted in May 2019 and January 2020. PARTICIPANTS: Ten novice raters who were existing hospital staff and had overall minimal OR experience and no prior experience with nontechnical skill assessment. RESULTS: ICC for the total NOTECHS team rating was 0.89 (95% confidence interval [CI], 0.87-0.91). ICCs for each NOTECHS category were as follows: situation awareness, 0.83 (95% CI, 0.78-0.88); problem solving and decision-making, 0.76 (95% CI, 0.70-0.83); teamwork and cooperation, 0.84 (95% CI, 0.79-0.88); leadership and management, 0.81 (95% CI, 0.75-0.86). CONCLUSIONS: This training curriculum for nontechnical skill assessments of OR teams was associated with high inter-rater reliability from novice raters with minimal collective OR experience. Using scalable training materials to produce reliable measurements of OR team performance, this nontechnical skills assessment curriculum may contribute to future QI projects aimed at improving surgical safety.
OBJECTIVE: To our knowledge, no curricula have been described for training novice, nonclinician raters of nontechnical skills in the operating room (OR). We aimed to report the reliability of Oxford Non-Technical Skills (NOTECHS) ratings provided by novice raters who underwent a scalable curriculum for learning to assess nontechnical skills of OR teams. DESIGN: In-person training course to apply the NOTECHS framework to assessing OR teams' nontechnical skill performance, led by 2 facilitators and involving 5 partial-day sessions of didactic presentations, video simulation, and live OR observation with postassessment debriefing. NOTECHS ratings were submitted after each of 11 video scenarios and 8 live operations for the total NOTECHS team rating (including surgical/anesthesiology/nursing subteams) and for each NOTECHS skill category-situation awareness, problem solving and decision making, teamwork and cooperation, leadership and management. Inter-rater reliability was determined by calculating the intraclass correlation coefficient (ICC, range 0-1). SETTING: Training for outcome measurement during a quality improvement initiative focused on surgical safety in 3 public hospitals in Singapore. Two trainings were conducted in May 2019 and January 2020. PARTICIPANTS: Ten novice raters who were existing hospital staff and had overall minimal OR experience and no prior experience with nontechnical skill assessment. RESULTS: ICC for the total NOTECHS team rating was 0.89 (95% confidence interval [CI], 0.87-0.91). ICCs for each NOTECHS category were as follows: situation awareness, 0.83 (95% CI, 0.78-0.88); problem solving and decision-making, 0.76 (95% CI, 0.70-0.83); teamwork and cooperation, 0.84 (95% CI, 0.79-0.88); leadership and management, 0.81 (95% CI, 0.75-0.86). CONCLUSIONS: This training curriculum for nontechnical skill assessments of OR teams was associated with high inter-rater reliability from novice raters with minimal collective OR experience. Using scalable training materials to produce reliable measurements of OR team performance, this nontechnical skills assessment curriculum may contribute to future QI projects aimed at improving surgical safety.
Authors: Jason C Pradarelli; Steven Yule; Stuart R Lipsitz; Nikhil Panda; Molly Craig; Kurt W Lowery; Stanley W Ashley; Denise W Gee; Peter M Waters; Jim Knight; Douglas S Smink Journal: Surg Endosc Date: 2020-07-08 Impact factor: 4.584