A surgical virtual reality simulator distinguishes between expert gynecologic laparoscopic surgeons and perinatologists.

BACKGROUND: Concern regarding the quality of surgical training in obstetrics and gynecology residency programs is focusing attention on competency based education. Because open surgical skills cannot necessarily be translated into laparoscopic skills and with minimally invasive surgery becoming standard in operative gynecology, the discrepancy in training between obstetrics and gynecology will widen. Training on surgical simulators with virtual reality may improve surgical skills. However, before incorporation into training programs for gynecology residents the validity of such instruments needs to first be established. We sought to prove the construct validity of a virtual reality laparoscopic simulator, the SurgicalSim(TM), by showing its ability to distinguish between surgeons with different laparoscopic experience.
METHODS: Eleven gynecologic surgeons (experts) and 11 perinatologists (controls) completed 3 tasks on the simulator, and 10 performance parameters were compared.
RESULTS: The experts performed faster, more efficiently, and with fewer errors, proving the construct validity of the SurgicalSim.
CONCLUSIONS: Laparoscopic virtual reality simulators can measure relevant surgical skills and so distinguish between subjects having different skill levels. Hence, these simulators could be integrated into gynecology resident endoscopic training and utilized for objective assessment. Second, the skills required for competency in obstetrics cannot necessarily be utilized for better performance in laparoscopic gynecology.

INTRODUCTION

Obstetrics and gynecology is unique amongst the surgical specialties in that it covers a wide range of medicinal and surgical issues pertinent to women's health. This diversity, but particularly the combination of medicine and surgery, has been a major attraction for medical students applying for training in this specialty. However, it is not clear whether this should still be considered an advantage or if it is in fact a burden to train competent clinicians in this specialty. The quality of surgical training in obstetrics and gynecology residency programs has suffered in the past 2 decades. Indeed, 63% of experienced gynecologic oncologists on the faculty of North American residency training programs reported a decrease in the surgical skills of residents compared to those trained 5 years ago.[1] Similarly, in a European study, only 30% of residents and 78% of obstetrician and gynecological surgeons were satisfied with their training.[2]

Some contributing factors to the deterioration of surgical experience for obstetrics and gynecology residents include resident work-hour limitations, super subspecialization, increased interventional radiology procedures, and advances in conservative medical therapies. Together these tend to decrease the number of surgical cases available for resident training. Other factors include an increased number of highly technical procedures and the introduction of minimally invasive surgery into the specialty.[3] Indeed, laparoscopy is becoming the standard of care in many gynecologic surgeries including ectopic pregnancies,[4,5] benign ovarian cysts,[6] oophorectomies, and treatment of endometriosis.[7] In addition, with recent advances in laparoscopy, hysterectomies and myomectomies[8] are now increasingly being performed through minimally invasive surgery.

Obstetrics and gynecology residents are expected to achieve competency in a variety of abdominal, vaginal, and endoscopic surgeries while spending a considerable amount of their time in learning and providing service in obstetrics. Since acquiring minimally invasive surgical skills is an absolute necessity for contemporary and future practice in gynecology,[9] this mandates curricular reform to move forward efficiently and effectively to increase surgical competency in the graduates.

The learning curve for laparoscopy is inherently different from that of open surgery, and the skills used in obstetrics, or abdominal and vaginal surgeries, cannot be translated into laparoscopic skills. Laparoscopy requires complex hand-eye coordination, fine motor skills, ambidexterity, understanding of the fulcrum effect, and depth perception.[10] Other contributing factors to the significant learning curve of laparoscopy are lack of a 3-dimensional view, impaired tactile feedback, and using fixed port sites, which results in limited mobility of the long instruments that are only partly visible.[11] The complications are usually the consequence of the relative inexperience of the surgeon, and most open conversions occur in a surgeon's first few cases.[12,13] There is a trend of diminishing complications with increasing experience that further emphasizes the importance of comprehensive training during residency.

In teaching laparoscopy, the “see one, do one, teach one” approach cannot be safely applied, and there is a need to go beyond the Halstedian apprenticeship model.[3] The apprentice model, which is still the gold standard in surgical education, is inadequate as the sole training method for many reasons. It is limited by the availability of surgical mentors and their teaching abilities. It is also very subjective and through poor surgical mentors can perpetuate inadequate surgical skills and decision-making in the operating room. On the other hand, residents display metacognition as adult learners and are aware of their own learning strategies and preferences. The arena is inconsistent and often stressful, which makes the learning even more unpredictable. The Halstedian model is unstructured, unreproducible, incapable of being standardized, and has often been referred to as “education by random opportunity.” Finally, the operating room is expensive and may not be an ideal environment to teach and learn basic surgical skills. For these reasons many residency training programs are now using animate and inanimate modules as an integral part of their curricula.

Surgical simulators have been extensively validated as educational tools and are valuable in teaching and evaluating surgical skills.[14-16] Nearly all studies have shown that training on simulators improves the performance and that the improvement is transferable to the operating room.[17-19] Virtual reality simulators were introduced more recently and have been shown to have similar potential to box trainers.[20] Computer-generated virtual reality simulators present an on-screen environment that facilitates sensory interaction and gives the impression of actually being present. In doing so, they allow the operator to interact with 3-dimensional computerized images in real time.[17]

Before a simulator can be used as an educational tool, it must be first demonstrated to be valid, reliable, and feasible. To do this, a number of parameters have to be assessed. Reliability is the reproducibility and precision of the test or testing device, whereas validity measures whether the simulator actually is teaching or evaluating what it is intended to teach or measure. Face validity relates to the realism of the simulator and content validity is a judgment of the appropriateness of the simulator as a teaching modality. Finally, construct validity indicates whether the simulator can distinguish between groups with different levels of expertise.[14]

For competency assessment, performance on a simulator should predict, or at least correlate with, an individual's performance in the operating room. A large number of simulators have been developed for laparoscopic surgical skills, with only a few dedicated for gynecology. Several studies have been performed to prove the validity of the different virtual reality simulators such as the LapSim simulator,[21,22] LapMento,[23-25] and the MIST-VR.[26,27] However, no study has been reported to prove the validity of the more recently introduced virtual reality simulator called SurgicalSimTM. In this simulator, a virtual laparoscopic system is created using a computer, a video monitor, and a laparoscopic interface containing 2 pistol-grip instruments and a diathermy pedal. The novel features of the SurgicalSim include the real-time feedback and the ability to record trainees’ performance for later review by program directors. It also provides the trainees with a measure of their progress and an objective comparison to the performance of their peers.

Proving the construct validity of a simulator is one of the crucial initial steps in showing the value of that simulator. The purpose of this study was to prove the construct validity of the SurgicalSim virtual reality simulator by comparing expert surgeons with perinatologists. We hypothesized that the performance between these 2 groups would be statistically different and the simulator would be able to recognize this difference. Our study shows that virtual reality simulators can be used as tools for objective assessment of surgical skills.

MATERIALS AND METHODS

Participants

All participants in the study are staff in the Department of Obstetrics and Gynaecology at the University of British Columbia, Vancouver, Canada. Participation was voluntary, and the Global Resident Scoring system was used to select the expert group. The study was reviewed and approved by the Clinical Research Ethics Board of the University of British Columbia. A list of staff members in the Obstetrics and Gynaecology Department in all subspecialty areas was sent to the chief and the most recently graduated residents in the program. A group of 11 were selected out of 33, as the expert gynecologic surgeons. Because studies have shown that clinical background is required for optimum performance with a virtual reality laparoscopy simulator,[28] the control group was selected from the perinatologists in the Maternal Fetal Medicine division to reduce confounding factors and to secure homogeneity of the participants. Eleven staff perinatologists participated. There were 7 females and 4 males in each of the 2 groups.

Virtual Reality Simulator and Tasks

The simulator used in this study is called SurgicalSim™ (METI, Sarasota, FL). The SurgicalSim software contains a wide variety of modules ranging from basic to more complex laparoscopic tasks. This simulator does not provide haptic feedback and the modules are multi-specialty adaptable with advanced modules for laparoscopic suturing and knot tying. None of the participants had previous experience with this simulator. All received detailed instructions about the 3 tasks described below and were given 3 minutes of hands-on introduction for orientation with the equipment.

Task 1: Traverse Tube

The objective for this task is to assess bimanual object manipulation and depth perception. Performance was assessed based on the total time to complete the exercise, tip trajectory defined as the total tip movement path, and the number of errors, defined as a dropped tube or grabbing the tube outside target area (see Video 1, Supplementary Material online).

In Task 1, the operator uses a grasper to hold a flexible tube at a target zone shown as a blue band. Once the tube is grasped at the right area, the next target zone appears approximately 3cm away, and the operator must use the second grasper with the other hand to grasp and hold the tube (. The first grasper is then disengaged and used to grasp the tube at the next target zone (, with this being repeated until the end of the tube is reached. If the tube is grasped outside the blue target zone or if the tube is dropped, the tube turns red and this is recorded as an error (. The task is then performed in the reverse direction, and the tip trajectory, errors, and the elapsed time are recorded by the simulator (see Video 1, Supplementary Material online).

Task 1, Traverse tube. Two graspers are used to traverse a virtual tube. Target zones appear consecutively as the graspers move along the tube. If the grasper misses the target zone, the tube glows red and is recorded as an error.

Task 2: Place Arrow

The objective here is to assess coordinate 2-handed object manipulation for an optimal level of traction and counter-traction. The educational goal is to orient a 3-D object in space using a 2-dimensional view. Performance was assessed by the simulator based on the total time to complete the exercise, tip trajectory, and the number of errors, which were defined as a dropped arrow, slipped arrow, and closed entry (where the grasper touches the tube before opening of its tips (see Video 2, Supplementary Material online).

Task number 2 is more difficult and requires the operator to grasp an image of a 3-dimensional object, in this case an arrow, at either end ( and to then move it to overlay a target arrow placed in a different 3-dimensional orientation (. The arrow needs to be either stretched or compressed in order to complete the exact overlay and then be held steady for 3 seconds (. In addition, the graspers must maintain the length of the arrow by appropriate traction in an imaginary 3-dimensional environment during the transfer and alignment or else the arrow will slip out, which is recorded as an error. This is repeated 5 times with different placements of the target (see Video 2, Supplementary Material online).

Task 2, Place arrow. Two graspers are used to grasp and move a virtual arrow while applying the right amount of traction. The arrow must be aligned and kept stationary for 3 seconds over a target in a 3D environment.

Task 3: Retract and Dissect Tissue

The objective for Task 3 is to assess coordinate ambidextrous use of instruments. The surgical concept is the exposure and alignment of tissue for optimal access for the dissecting electrocautery hook. Performance was assessed by the simulator based on total time to complete the task, tip trajectory, and errors that were defined as excessive traction, thermal injury to the adjacent tissue, and electrocautery in the air (see Video 3, Supplementary Material online).

This most complex task begins by grasping a tongue of simulated tissue (, applying traction in the right amount and direction to expose the underlying attached surrogate tissue (, and then applying a simulated foot-activated electrocautery action to release the exposed attachment (. This was repeated by switching hands and using a different tongue of simulated tissue to assess ambidexterity (. The electrocautery action should only be initiated on the tissue and not in the air, which was also recorded as an additional parameter to compare the performance between 2 groups. Color bars indicated correct force applied versus under- and overtraction of the tissues. The thermal effect on the tissue was also shown with color bars: green indicated successful dissection, whereas red showed thermal injury (see Video 3, Supplementary Material online).

Task 3, Dissect tissue. The grasper holds a simulated tongue of tissue stationary under the right amount of traction while a foot-activated cautery releases the attachments. Upon release of the tongue, the hands switch and the task is repeated.

Statistical Analysis

The data were analyzed using SPSS 8.0 statistical software with the Kruskal-Wallis test to determine interindividual differences. Differences between groups were evaluated using the chi-square test for ordinal data, and the 2-tailed t test for continuous data. Statistical significance was defined as P<0.05.

RESULTS

Although this was a pilot study involving 22 participants, this sample size was large enough to provide highly significant data for most tasks. In particular, for Task 1 the study group performed faster (P<.001), more efficiently (P<.010), and with fewer errors (P<.016) compared to the control group (. For Task 2, the experienced operator group performed in less total time (P<.001) and with better economy of motion (P<.003) (. Again for the third task, the gynecologic surgeons performed faster (P<.009) and with fewer errors (P<.001) (.

Task 1 results showing a) time for task completion, b) tip trajectory, and c) number of errors. Expert gynecologic surgeons (study), perinatologists (control). Standard deviation bars are shown.

Task 2 results showing a) time for task completion, b) tip trajectory, and c) number of errors. Expert gynecologic surgeons (study), perinatologists (control). Standard deviation bars are shown.

Task 3 results showing a) time for task completion, b) tip trajectory, c) number of errors, and d) time that cautery was used in air instead of target tissue. Expert gynecologic surgeons (study), perinatologists (control). Standard deviation bars are shown.

Interestingly, there was no statistically significant difference between the 2 groups in the number of errors (P<.263) for Task 2. Likewise, there was no statistically significant difference between the 2 groups in tip trajectory (P<.293) and electrocautery in the air (P<.490) in Task 3 (. Potentially a larger group size might have shown significance in these 2 analyses.

DISCUSSION

Our study proves the construct validity of SurgicalSim by showing its ability to distinguish between the different levels of laparoscopic surgical skills among staff obstetricians and gynecologists. In so doing, we also show that the surgical skills pertinent to obstetrics differ from those required for laparoscopic surgery.

Because the practice of gynecology is rapidly changing so should the training. The need for competency based residency education is not new and was recognized as early as 1997. After graduation the problem is more acute—training for practicing surgeons is not standardized, because in most cases it is neither mentored nor monitored. Furthermore, industry sponsored weekend crash courses are not validated as an effective tool to achieve competency in laparoscopic surgical procedures. Therefore, it seems logical that the most effective time to learn laparoscopy is during residency. Obstetricians and gynecologists ought to remain competent in both obstetrics and in a variety of gynecological surgical procedures. Since these have inherently different learning curves, then improvements in training and implementing educational and objective assessment tools are a necessity both at the residency and continuing education levels.

By proving the validity of more sophisticated virtual reality simulators such as SurgicalSim, we would recommend the incorporation of this or similar educational tools in developing competency based endoscopy education programs specific for gynecology residents and also to use these as a tool for objective assessment of surgical skills among residents. Virtual reality simulators like SurgicalSim have the advantage of saving time and money for the instructors by automatic recording of results. This not only eliminates the observer bias in evaluating a trainee's skills but also saves considerable time and expense for the faculty members.

Three kinds of curriculum exist: the planned, the taught, and the learned. From the outcome perspective, the most important is the learned curriculum. Under the apprentice method, translating the planned to the learned is very much mentor dependent. However, with competency based education using a variety of tools including surgical simulators and with correctly implemented and valid assessment tools to drive learning, greater confluence of the planned and the learned curricula can occur. More competent surgeons will therefore emerge from such programs. Having a standardized endoscopic curriculum to teach and evaluate surgical skills pertinent to gynecology. After all, “To measure is to know. If you cannot measure it you cannot improve it” (William Thomson, Lord Kelvin).