Mapping the robotic hysterectomy learning curve and re-establishing surgical training metrics.

OBJECTIVE: Common robotic training curricula in the US entail completion of an online module followed by lab training with standardized exercises, such as manipulating needles with robotic needle drivers. Assessments are generally limited to elapsed time and subjective proficiency. We sought to test the feasibility of a simulation-based robotic hysterectomy curriculum to collect objective measurements of trainee progress, map the trainee learning curve and provide a system for trainee-specific evaluation.
METHODS: An observational cohort study of a single institutions' residency members participating in a procedural hysterectomy simulation performed every 4 months. Each simulation episode had one-on-one teaching. The robotic platform was used to measure all movements within cartesian coordinates, the number of clutches, instrument collisions, time to complete the simulated hysterectomy, and unintended injuries during the procedure.
RESULTS: Voluntary participation was high. Objective metrics were successfully recorded at each session and improved nearly universally. More senior residents demonstrated superior capabilities compared to junior residents as expected. The majority of residents (29/31) were able to complete an entire simulated hysterectomy in the allotted 30-minute training session period by the end of the year.
CONCLUSIONS: This program establishes learning curves based on objective data points using a risk-free simulation platform. The curves can then be used to evaluate trainee skill level and tailor teaching to specific objective competencies. The pilot curriculum can be tailored to the unique needs of each surgical discipline's residency training.

“Medical instruction does not exist to provide individuals with an opportunity of learning how to make a living, but in order to make possible the protection of the health of the public.”

- Rudolf Virchow

INTRODUCTION

This statement holds even more truth when considering surgical instruction and training. The most prevalent methods of surgical training have always involved an apprenticeship model. The trainee follows the attending surgeon, watches procedures to be mastered, starts performing them, with the hope that they, too, will be able to teach these procedures, or as they the old (and outdated) adage goes, “see one, do one, teach one.” Over the course of surgical apprenticeship, there has never been any meaningful comparison of this ‘classic’ method of surgical training to other methods [1].

With the advent of robotic-assisted surgery, it is imperative that novel, efficient, and safe training paradigms are instituted. Since the introduction of computer and robotic-assisted surgeries, surgical tools have now become increasingly complex. As such, many postgraduate training programs have instituted resident training models to try to expose trainees to the robotic surgical system prior to live surgery. This training typically only occurs once a year. In addition, the majority of these curricula entail a an online module followed by dry lab training with standardized exercises, such as manipulating needles and cutting along set paths, that are evaluated on Likert scales [23]. Likert scales, while widely accepted and used for surgical training, may not represent the best evaluation for training of this nature [456]. Furthermore, those curricula that do incorporate actual steps or portions of an actual procedure generally do not have any significant or robust robotic systems-based training prior to procedural training [7]. The vast proportion of robotic training occurs during live surgery; the learner is made to climb two distinct learning curves: how to use the robot while simultaneously learning how to perform a specific surgical procedure, potentially hindering each curve. As healthcare continues towards quality metrics and outcomes data, we will need to adapt to more efficiently and effectively train our future surgeons with innovation and modernization of educational methods [8]. Obstetricians and gynecologists (OBGYN) training presents unique challenges as it has surgically oriented rotations others with little to no surgery. Unless a surgical rotation occurs shortly after the resident robotic training, the resident undergoes skill decay.

Surgical training methods are frequently studied, but these evaluations frequently focus on validating subjective evaluation systems, either directly observed or during simulated procedures [9101112]. Various models have been developed for gynecologic surgery, and while useful for demonstrating 3-dimensional anatomy they do not provide objective data on trainee skills [131415]. Various robotic simulators are also available, but again objective evaluation is typically limited to completion time, preventing truly focused feedback or training or practice [16]. This project evaluates the feasibility of a novel simulation- and motion-based robotic training curriculum held at regular intervals throughout the academic year. The construction and use of incremental learning curves for a robotic hysterectomy with respect to each technical aspect of the procedure at regular intervals was also evaluated. This quantitative metric-based training and assessment method and curriculum has never been evaluated before in the context of robotic surgical training; as such, was also assessed.

MATERIALS AND METHODS

Under an Institutional Review Board-approved protocol, a high-fidelity robotic hysterectomy simulator (software: 3DS Systems by Simbionix [now 3D Systems], Littleton, CO, USA and hardware: Intuitive, Sunnyvale, CA, USA) was employed to train and assess residents, with the ultimate goal of each learner performing a total robotic-assisted hysterectomy by the end of training. The novel simulator software on the Intuitive da Vinci Xi Simulator backpack.

The learners were all OBGYN residents at the University of Alabama at Birmingham; during the study, there were 8 residents per class year, for 32 total residents. The educational strategies for this robotic learning curriculum were learner-based with an emphasis on practice-based and systems-based learning. The teaching methods for the curriculum entailed individual simulation sessions scheduled every 4 months for each resident. Each of these sessions were designed with one-on-one coaching throughout the course of the academic year. As each learner progressed through the curriculum through the year, previously learned skills were built upon, and increasingly more portions of the simulation were able to be performed. Each of the 4-month sessions had direct one-on-one interaction with the same robotic faculty member, progressing through the standardized steps of the procedure (Table 1). Each session was limited to 30 minutes. This high-fidelity training occurred in regularly scheduled, low-stakes training sessions every four months for an entire year to maximize efficacy and decrease skill decay. Training between sessions was not standardized, beyond the clinical education of residency, through which all residents successfully progressed throughout the year.

Table 1

Standardized steps of a robotic hysterectomy used in this curriculum

Standardized steps
Identification of relevant anatomy
Ligation of the round ligament
Development of retroperitoneal spaces
Skeletonization and ligation if the infundibulopelvic ligament
Dissection of the posterior leaf of the broad ligament
Dissections of the anterior leaf of the broad ligament and mobilization of the bladder
Skeletonization and ligation of the uterine vessels
Repeated steps on the contralateral side
Performance of colpotomy

Vaginal cuff closure was not included in the simulation.

The simulator recorded all movements of the robotic arms, including the camera motion and clutching, in Cartesian coordinates. These metrics were used to assess economy and accuracy of motion as well as measuring procedure-specific safety and complication metrics (Table 2). Quantitative learning curves were constructed based on these objective metrics measured and recorded over time for each resident, as well as for each postgraduate training year (PGY) class.

Table 2

Efficiency and safety metrics measured by the 3DS simulator

Efficiency	Safety
Total time	Injury to bladder
Number of movements of left instrument	Injury to colon
Number of movements of right instrument	Injury to ureter (left or right)
Total path length of left instrument	Injury to uterine artery (left or right)
Total path length of right instrument	Injury to uterine artery was controlled (left or right)
Total distance by camera	Injury to the infundibulopelvic ligament (left or right)
Number of instrument collisions	Injury to the infundibulopelvic ligament was controlled (left or right)
Number of clutches/clutch usage	Injury to the utero-ovarian ligament (left or right)
Total path/number of times instruments are out of view of camera	Injury to the utero-ovarian ligament was controlled (left or right)
Total time of instruments out of view of camera	Injury to large vessels of the pelvis

RESULTS

Four one-on-one training sessions were held over the span of one year, scheduled after normal work hours. Sessions were held every four months. Resident participation was voluntary, and support was arranged to facilitate attendance by on-call residents. The first session was held in May so the incoming PGY1 class had not yet arrived and did not participate. Attendance rates were high, with 19/24 residents in the first session, and 30/32, 31/32, and 31/32 attending the second, third, and fourth sessions, respectively. Safety and complication data were compiled, including bowel or bladder injury, ureteral injury, uncontrolled bleeding, and critical vessel injury (Table 2). Complication rates were rare with the most common being uterine vessel bleeding (Table 3).

Table 3

Safety and complications

Complications	No. of events
Bladder Injury	1
Bowel Injury	0
Ureter injury	0
Ovarian vessel bleeding	11
Uterine vessel bleeding	39
Critical vessel Injury	0

Total number of safety events were summed across all participants and all sessions over the entire study period.

Surgical efficiency was measured via several metrics, including number of movements by each hand, total path length of each instrument, path length of the camera, number of instrument collisions, and number of clutches throughout the case (Table 2). Path lengths of instruments and the camera represent the total distance traveled in 3-dimensional space of the respective instrument or camera tip. All residents were given 30 minutes to complete the simulation; each resident was given a “complete” or “incomplete” designation for whether they were able to complete the entire hysterectomy procedure by the end of the 30-minute session. At the first session 3/19 (16%) participants completed the hysterectomy, and this increased to 14/30 (47%), 28/31 (90%), and 29/31 (94%) in sessions two, three, and four, respectively. Each resident's total camera distance was tracked and trended over the four sessions. The specific metric can be compared to equivalent PGY residents and to those more advanced. In addition, we can trend each metric relative to simulation completion, as we would expect more movements as surgical speed increases and they complete more of the procedure.

Trends in metrics were tracked over time throughout the course of the year, both individually and as PGY class groups. Instrument movements increased from session 1 to 2, but then either decreased or remained stable. The range between PGY classes also increased initially then decreased by session 4. Path length showed a similar trend with a greater decrease from sessions 2–4 (Fig. 1). Instrument collisions and clutches also initially increased from session one to two but decreased from sessions 2–4 (Fig. 2). The number of collisions decreased significantly more from session to session compared to the other metrics.

Fig. 1

Compiled data output.

Sample representation of the data output of our simulated hysterectomy. Camera distance traveled is shown for each participant over time, with averages for each PGY class shown below. This can be superimposed over hysterectomy completion rates as shown to the right.

PGY, postgraduate training year.

Fig. 2

Collisions and clutches.

Instrument collisions are shown on the upper graph, while number of instrument clutches is below. Values represent the mean for each PGY class. Error bars are the standard error of the mean.

PGY, postgraduate training year.

Residents completed the hysterectomy simulation at much higher rates at later sessions. At the first session, only three residents were able to complete the simulated hysterectomy, compared to 29 in the final session (Fig. 3A). When overlaid with the previous metrics, the increase in movements, path lengths, and clutches between sessions 1 and 2 correlates with more residents completing larger portions of the simulation, and therefore performing more of the surgical procedure (Fig. 3B and C). As the resident progressed through the year, increasing numbers of residents completed the entire hysterectomy, and with less movements, signifying increased levels of efficiency with decreased amounts of wasted movements.

Fig. 3

Simulation completion overlaid with class metrics.

Trainee completion totals are shown separately and superimposed on clutches and camera distance. Some increases in the latter metrics may be attributable to progressing further in the simulation.

PGY, postgraduate training year.

DISCUSSION

Here we demonstrate a novel quantitative method for surgical training and evaluation. Our training program had high levels of participation at four sessions carried out at 4-month intervals over 12 months. Participation levels were high for all PGY levels of training and the program was completed without undue burden on supervising or training physicians. Resident progress was tracked while performing a simulated hysterectomy and by the fourth training session, 94% (29/31) residents across all PGY levels were able to complete the procedure within the time limit. With this educational design, skill decay was minimized with regularly scheduled simulation sessions, and with instructional feedback by a single faculty surgeon mentor.

We demonstrate here the ability to track and evaluate quantitative measures regarding surgical performance including number of discrete movements, total path length, instrument collisions, and clutches. As expected, when more residents completed increasing portions of the hysterectomy within the allotted time, path length, collisions, and clutches initially increased; however, with additional sessions and despite more residents completing larger proportions of the procedure within the 30-minute time period, those same metrics showed a downward trend. This indicates increased efficiency and proficiency in completing the procedure with less wasted movements, as well as increased comfort and safety in using the surgical robotic system. This quantitatively represents an increase in efficiency and more effective economy of motion, a common goal in all surgical learning.

With any given procedure, there are two learning curves that the surgical learner will contend with: how to master the techniques with use of the instruments, and how to perform the steps of the procedure in the logical, correct manner to complete the procedure. The primary goal of the robotic hysterectomy curriculum was to train residents on how to use the robotic platform. However, the added benefit of the hysterectomy simulation afforded the additional advantage of also addressing the additional learning curve involving the procedural steps how to perform a hysterectomy. Of note, we did find that the simulator employed does appear to have a suboptimal simulation in the ligation of the uterine vessels, a limitation of the simulation technology that was available at the time of development. This was the complication that occurred most often. The majority of these uterine vessel injuries occurred after the resident had already performed the ligation of the uterine vessels, and the computer simulation did not recognize this consistently; indicating the technology can still be improved upon to continue to maximize surgical procedural learning. Regardless, a significant benefit of this curriculum allows for teaching how to address surgical complications in a safe, simulated environment and can allow for better preparation, and ultimately better care, for our patients.

The major limitations of the study include a small cohort at a single site, the use of a simulated surgery of uncertain correlation to actual surgery, and despite a single robotic surgical mentor, a lack of standardized teaching between sessions, which can influence trainee's skills at each time point. However, skill sets should normalize over the course of an entire academic year. In addition, the true utility of this curriculum and the data it produced has yet to be determined. While anecdotally, this has increased efficiency and surgical technique in the live operating room across various divisions and surgeons we inquired informally, we did not correlate skills in the simulation with those performed during live surgery. This was beyond the scope of this pilot study. The strengths include objective and quantitative measurement of trainee metrics, longitudinal data over time, and high participation rates.

This objective method of training can be likened to learning sports and playing in competitive games. Athletes will learn technique and skills prior to competing, and then at the time of competition, will apply their learned techniques and skills in a logical manner. In the current training method described, residents are trained repeatedly to learn the techniques and skills, as well as mastery of the use of the robotic surgical system as a tool, prior to engaging in the live operating room. Doing so allows for the residents to reserve the precious time in the operating room more specifically for procedural learning, rather than attempting to learn how to use the robotic surgical system as well as learn the procedure, essentially navigating two learning curves simultaneously while operating on a live patient.

As this is the first program incorporating quantitative and objective analysis of surgical skills at regular intervals over the course of a year, the baseline values or expectations are unknown. In fact, there may be no true baseline as skills and experience differ widely between individual trainees, and surgical instruction and coaching should likely focus on where the trainee currently stands with technique and skill, and building from that point. Regardless, the evaluation methods described here provide quantitative metrics to evaluate trainee progress through a standardized, simulated procedure. We would expect path length and clutches to increase and peak as trainees complete increasing proportions of a procedure, then gradually decrease as trainees become more efficient with economy of motion. Instrument collisions should also sharply decrease once trainees are move familiar with the robotic surgical platform, which was demonstrated. This is a similar phenomenon that occurs as individuals learn any coordinated technical movement, whether it be learning how to brush one's teeth, learning how to tie one's shoes, or learning how to play a sport or a musical instrument. Individuals will initially be inefficient and uncoordinated with increased proportions of wasted movements, and then at some point, peak with the number of movements before they become increasingly more efficient, thereby decreasing the total number of movements to accomplish a task. Decreasing skill decay with conducting regularly scheduled training sessions throughout the year with this curricular model was an additional benefit, which also increased the efficiency of their surgical learning throughout the year.

In conclusion, this metric-based training has allowed the construction of specific learning curves related to each of the technical components of performing a robotic-assisted procedure. The specific graphs of each component learning curve were constructed, such as left- and right-hand movement, camera movement, and clutching, which has enabled us to identify the stage of development at which a resident is throughout the year. This can also assist in measuring the phenomenon of skill decay, particularly pertinent as many of OBGYN residents progress through rotations that have little to no gynecologic or minimally invasive surgical experience. The metrics described here allow for individual progress to be tracked over time and can help train residents in a competency-based fashion, which can also allow for engagement of each resident to an appropriate level based on their proficiency. It also can help identify trainees who are not improving as expected and may need additional support or training, which can then be implemented earlier to get them to the appropriate competency level.

Most of all, this can be determined outside the high-stakes environment of live surgery, while also eliminating the bias of subjective observation. With the increasing focus on healthcare outcomes as well as maintenance of skills, it will be imperative for our surgical training programs to be able to implement and maintain surgical skills in an objective, competency-based manner.