| Literature DB >> 36199481 |
Sarah C Skinner1,2, Stéphanie Mazza1, Matthew J Carty3, Jean-Christophe Lifante1,4, Antoine Duclos1,2,3.
Abstract
To characterize quantitative studies on coaching interventions for professional surgeons to understand how surgical coaching is defined; examine how different coaching programs are designed, implemented, and evaluated; and identify any relevant research gaps. Background: Surgical coaching is gaining attention as an approach that could help surgeons optimize performance and improve overall wellbeing. However, surgical coaching programs and definitions of coaching vary widely between studies.Entities:
Keywords: coaching; nontechnical skills; performance; technical skills; wellbeing
Year: 2022 PMID: 36199481 PMCID: PMC9508984 DOI: 10.1097/AS9.0000000000000179
Source DB: PubMed Journal: Ann Surg Open ISSN: 2691-3593
List of Articles Selected for Inclusion and Their Definitions of Coaching
| No. | Title | Study ID | Coaching Definitions(s) | Study Objectives |
|---|---|---|---|---|
| 1 | A statewide surgical coaching program provides opportunity for continuous professional development | Greenberg et al[ | Coaching—“an experiential process for improving any aspect of surgical performance, including technical, cognitive, and interpersonal skills… based upon a partnership between 2 surgeons in which one facilitates the other’s pursuit of self-identified goals through collaborative analysis, peer support, and constructive feedback. Coaching emphasizes the development and refinement of the learner’s existing skills and his/her empowerment to make changes to practice” | To develop and implement an evidence-based peer coaching program for board-eligible/certified surgeons across practice settings in the state of Wisconsin. The four main objectives were to identify the goals of surgeons participating in the peer coaching, evaluate the extent that the coaching session adhered to the stated goal, evaluate the effectiveness of surgeon coaches to employ activities of coaching; evaluate the perceived value of participation in a surgical coaching program |
| 2 | Targeted surgical coaching can improve operative self-assessment ability: A single-blinded nonrandomized trial | Bull et al[ | Surgical coaching—“a constructive relationship that provides objective feedback to individuals about a broad range of factors influencing operative performance. There is a focus on improvement and refinement of existing skills, rather than teaching new techniques or philosophies. Individualized approaches are required to effectively implement these objectives. Excellent coaching interactions encourage discussion, affirm positive beliefs, and challenge assumptions” | To investigate the effect of targeted surgical coaching on self-assessment of laparoscopic skill |
| 3 | Team Leader Coaching Intervention: An Investigation of the Impact on Team Processes and Performance Within a Surgical Context | Maynard et al[ | Coaching—“a process of equipping people with the tools, knowledge, and opportunities they need to develop themselves and become more effective.” | To determine whether a team leader coaching intervention can improve episodic team processes and enhance operating team outcomes |
| 4 | Surgical Coaching for Operative Performance Enhancement (SCOPE): skill ratings and impact on surgeons’ practice | Pradarelli et al[ | Surgical peer coaching—a “series of structured one-on-one discussions over time… provides space for surgeons to reflect on their performance with another surgeon (i.e., a peer coach) and make changes to their own practice” | To better understand the impact of coaching on surgeons’ performance, to measure surgeons’ technical and nontechnical skills throughout a longitudinal surgical coaching program |
| 5 | Feasibility of Surgeon-Delivered Audit and Feedback Incorporating Peer Surgical Coaching to Reduce Fistula Incidence following Cleft Palate Repair: A Pilot Trial | Sitzman et al[ | Surgical coaching—“a social interaction that aims to develop expertise by setting specific goals and providing feedback to achieve those goals” | To determine the feasibility of a surgeon-delivered audit and feedback intervention incorporating peer surgical coaching. |
| 6 | Association of a Statewide Surgical Coaching Program with Clinical Outcomes and Surgeon Perceptions | Greenberg et al[ | N/A | To assess the association between participation in a surgical coaching program and risk-adjusted outcomes |
| 7 | Evaluation of Coaching Impact on Surgical Outcomes | Duclos[ | N/A | To evaluate the impact of a customized surgical coaching program, targeting surgeons’ physiological factors (sleep, stress, physical activity), on the occurrence of patient major adverse events. |
| 8 | Effect of Coaching on Surgeon Wellbeing, Job Satisfaction, & Fulfillment | Dyrbye[ | N/A | To determine if individualized professional coaching improves physicians’ sense of wellbeing and jobsatisfaction. |
Description of Coaching Programs
| Coaching Intervention Goals | Description of Coaches | Interaction | Coaching “Material” Used to Evaluate | Study ID | |||||
|---|---|---|---|---|---|---|---|---|---|
| Type of Intervention Goal | Overall Goals | Details About Coaching Goals | How Many? | Who Were the Coaches? | Were the Coaches Trained? | In Person? | Surgery Observed? | Metrics/Assessment Tools Used to Evaluate Coachees | |
| Technical skills | Investigators | Improve technical performance of cleft palate repair (decreasing fistula incidence) | 2 | Surgeons (peer) | No | In person | Video recording of operation in OR | Audit and feedback—collects standardized outcome measurements and reports individual and peer group results. | Sitzman et al[ |
| Investigators | Improve accuracy of self-assessment of laparoscopic operative skill using the OSATS scale | 2 | Surgeons (study investigators) | Specialists in rater training and coaching techniques | In person | Video recording of operation in OR | Coachees assessed the same peer surgery video at baseline, midway, and completion of the interventionto provide a point of reference for | Bull et al[ | |
| Nontechnical skills | Investigators | Leadership training to improve episodic team processes (transition, action, interpersonal) and team performance outcomes (surgical delays and distractions) | 1 | Surgeon (retired) | Certified in Crew Resource Management training | In person | Observation in OR | Coded assessment of surgical delays and distractions, and interpersonal, transition, and action processes | Maynard et al[ |
| Technical and/or nontechnical skills | Coachee | 6 coachees identified nontechnical goals (teaching, leadership, communication), 3 identified technical goals (new procedure, improving efficiency), and 2 did not identify a specific focus | 8 | Surgeons (peer) | 4-hour training session, training manual, and instructional videos | In person | Video recording of operation in OR | None | Greenberg et al[ |
| Coachee | Coachees could focus on any aspect of intraoperative performance, including technical skills, nontechnical skills (situational awareness, decision making, teamwork, leadership), or intraoperative teaching skills | 23 | Surgeons (peer) | 3-hour training session and weekly emails with coaching tips | In person | Observation in OR | OSATS and NOTTS | Pradarelli et al[ | |
| Coachee | No details about individual coachee’s goals were provided. | 15 | Surgeons (peer) | 4-hour training and “refresher” on coaching principles before each session | In person | Video recording of operation in OR | Operative time and 30-day patient complications | Greenberg et al[ | |
| Physiological factors | Coachee | Improve physiological factors to enhance surgeons’ performance and improve patient outcomes | N/A | Nonsurgeon | Certified professional coach | By videoconference | None | Charting system providing patient outcomes and profiling of individual surgeons as feedback | Duclos[ |
| Wellbeing | Coachee | Improve wellbeing (Burnout, job satisfaction, professional fulfillment) | N/A | Nonsurgeon | Certified professional coach | By phone | N/A | N/A | Dyrbye[ |
aThe goal the coachee was aiming to achieve over the course of the intervention. This goal was either defined by the study investigators or the coachees themselves.
OR, operating room; OSATS, objective structured assessment of technical skills; NOTTS, nontechnical skills for surgeons.
FIGURE 1.Visualization of the coaching goals and settings of the interventions.
Description of Study Designs, Coachees, and Study Setting
| Study Design | Coachee Population | ||||
|---|---|---|---|---|---|
| (GRADE Study Design) | Mixed Methods? | Multi- or Monospecialty | No. | Study Setting | |
| Cross-sectional | No control (Very low) | Yes | Multispecialty | 11 | WI, USA |
| Quasi-Experimental | Before-After without contemporaneous control | No | Multispecialty | 4 | Australia |
| Yes | Multispecialty | 23 | Boston, MA, USA | ||
| Before-After with contemporaneous control (Moderate) | No | Multispecialty | 20 | Rocky Mountain Region of CO, USA | |
| Yes | Monospecialty | 26 | MI, USA | ||
| Yes | Monospecialty | 2 | Across the U.S. and Canada | ||
| Experimental | Cluster-randomized trial | Yes | Multispecialty | 20 | Lyon, France |
| Crossover-randomized trial | No | Multispecialty | 80 | Mayoclinic, USA | |
aGRADE - simplified version of the Grading of Recommendations Assessment, Development, and Evaluation system - cross-sectional studies with no control group were classified as “very low”, quasi-experimental designs with no controls were classified as “low”, quasi-experimental designs with contemporaneous controls were classified as “moderate”, and experimental studies were classified as “high”. These ratings categorize studies based on their design, and unlike the comprehensive GRADE scale, do not fully reflect the validity or reliability of the research.
bStudy included 12 total coachees, but only 4 were attending surgeons, while 3 were fellows and 5 were residents.
Study Outcomes and Findings
| Quantitative Outcomes | Findings | Study ID | ||||
|---|---|---|---|---|---|---|
| Kirkpatrick 1 (Reaction) | Kirkpatrick 3 (Behavior) | Kirkpatrick 4 (Results) | ||||
| Surgeons’ Reactions About Being Coached | Surgeons’ Technical Skills | Surgeons’ Nontechnical Skills | Surgeons’ Wellness | Patient Outcomes | Positive Outcome(s) ? | |
| Coachees used 5-point likert scales to evaluate the effectiveness of the coaches and to evaluate the program following the intervention | Yes | Greenberg et al[ | ||||
| A likert type participant questionnaire was used to evaluate the surgeons’ opinions about the coaching intervention and coaching in general, after the program | Improvement of self-assessment accuracy (using OSATS scale) compared to expert rating and correlation of self-assessment with expert ratings. | Yes | Bull et al[ | |||
| Summary assessment of the coachee’s overall technical skill and ratings for each of five domains of technical skills using the OSATS framework. | The Nontechnical Skills for Surgeons assessment tool was used to rate coachees performance on each of four domains of nontechnical skill: situation awareness, decision making, communication and teamwork. | No | Pradarelli et al[ | |||
| Evaluated episodic team processes (interpersonal, transition, and action processes measured by SME) and subsequent team performance outcomes (delays and distractions) | Yes | Maynard et al[ | ||||
| Operative time | 30-day overall complications and 30-day surgical complications | Operative time: yes; | Greenberg et al[ | |||
| Ornonasal fistula incidence | No | Sitzman et al[ | ||||
| Operative time | Burnout (MBI), Psycho-social risk (self-reported 26 item Karasek questionnaire) | Occurrence of major adverse event, death, unplanned stay in intensive care unit, unplanned reoperation, and intraoperative and post-operative complications | Duclos[ | |||
| MBI, Job satisfaction (self-reported Physician Job Satisfaction Scale score) and professional fulfillment (change in Empowerment at Work Scale) | Dyrbye[ | |||||
We used an adapted Kirkpatrick model to categorize the studies’ outcomes. Outcomes related to surgeons’ reactions to the coaching intervention or coaching in general were classified as level 1. Outcomes measuring technical or nontechnical skills, knowledge, or attitudes in a simulated environment were considered level 2. Outcomes measuring technical or nontechnical skill or changes in behavior measured in the operating room or in real life were classified as level 3. Patient outcomes or surgeon-centered outcomes were classified as level 4.
OSATS, Objective Structured Assessment of Technical Skills; MBI, Maslach Burnout Inventory Scale; SME, subject matter expert.
FIGURE 2.Research gaps diagram classifying studies by coaching goal, outcome, design GRADE, and findings. Study findings are represented by colors. Green represents positive findings, yellow mixed findings, and red negative findings. Dark gray represents outcomes of trials that have not been published yet. Light gray represents outcomes that were not evaluated by any of the studies included in the review. We used an adapted Kirkpatrick model to categorize the studies’ outcomes. Outcomes related to surgeons’ reactions to the coaching intervention or coaching in general were classified as Kirkpatrick level 1. Outcomes measuring technical or nontechnical skills, knowledge, or attitudes in a simulated environment were considered Kirkpatrick level 2. Outcomes measuring technical or nontechnical skill or changes in behavior measured in the operating room or in real life were classified as Kirkpatrick level 3. Patient outcomes or surgeon-centered outcomes were classified as Kirkpatrick level 4. We used an approach inspired by the GRADE system to to classify evidence levels based on intervention study designs. Randomized trials were considered “high,” quasi-experimental studies with contemporaneous controls were categorized as “moderate” and without controls as “low.”