Virtual reality uterine resectoscopic simulator: face and construct validation and comparative evaluation in an educational environment.

BACKGROUND AND OBJECTIVES: Recognizing that resectoscopic simulation may have an educational role, this pilot study was designed to evaluate the face validity and educational utility of a virtual reality uterine resectoscope training system.
METHODS: A pilot prospective comparative study of novice and expert hysteroscopists' performance on a targeting exercise and myomectomy with the virtual loop electrode. At baseline, expert and novice resectoscopists each performed both exercises. Following instruction, novices practiced each exercise a total of 9 times with the 10th recorded as the training outcome. Results were compared both to baseline and to those of the experts. Data were analyzed with the paired t and Wilcoxon rank sum tests as appropriate.
RESULTS: At baseline, all experts touched 4 targets in a mean of 33 seconds with no perforations, compared to a mean of 2 for the 11 novices in a mean of 57 seconds (P=0.0034) with one perforation. In 3 minutes, the experts removed a mean of 97.3% of the virtual myoma, compared to 66.1% for the novices (P=0.0153). On the 10th "run," novices touched a mean of 4 targets in a mean of 23 seconds, an improvement from baseline (P=0.0004) and improved to 89% on the myoma resection exercise (P=0.0515) 36.3% over baseline.
CONCLUSION: Although this pilot study has a relatively small sample size and represents the results at one institution, it demonstrates that virtual reality resectoscopic systems have the potential to measure and improve the technical skills of novices before they operate on human patients.

INTRODUCTION

Hysteroscopy is a surgical skill that provides the gynecologist and patient with the opportunity to experience improved diagnostic accuracy, and clinically effective, minimally invasive surgery for the treatment of a variety of disorders including symptomatic endometrial polyps, submucosal leiomyomas, and selected Müllerian fusion/absorption defects. Unfortunately, since its introduction into the medical literature by Pantaleone, from Italy, in 1869,[1] hysteroscopy and hysteroscopically directed procedures have been reluctantly incorporated into the practice of gynecology, despite the development of highly effective instruments for endoscopic visualization and performance of intrauterine surgery.

Training of residents, fellows, and practicing gynecologists has been hampered by a number of obstacles, including a lack of suitably trained mentors, limited access to appropriate equipment, restrictions in resident training hours, and the absence of readily available systems to support training outside of the operating theater. A number of systems and simulators have been proposed, but there still seems to be a relative paucity of postgraduate training programs with robust hysteroscopic education curricula. The existence of microprocessors capable of managing large volumes of data throughput has provided the opportunity for the design of virtual reality (VR) surgical simulators that create a near realistic operating environment. Such simulators have been demonstrated to be effective in other surgical disciplines and for other techniques, by reducing both the need for “in-OR” training time and activity associated with adverse events.[2-5] The use of VR simulation can provide a realistic operating environment that provides neither surgical risk to a patient nor the consumption of valuable operating room resources and time. If properly designed, such a system can provide the opportunity for objective measurement of selected surgical performance outcomes.

Recently, a prototypical Virtual Reality Hysteroscopic Simulator (VRHS) was developed by Immersion Medical (CAE Montreal PQ, Canada), which provides an opportunity to evaluate the utility of such a system for training in hysteroscopic surgery (. The device comprises a realistic resectoscope with an attachable “camera” and distal sensors, a pelvis with a mechanical external cervical os, and actual electrosurgical foot pedals, all attached to a personal computer loaded with proprietary software. Following assembly, the “resectoscopic system” is inserted through a mechanical external os, thereby allowing the distally based sensors to activate the software, in a fashion that provides a realistic, interactive screen image. Virtual inflow and outflow can be regulated, and the operator-controlled motions of the element and the distal optic are transmitted to the screen to simulate an actual intrauterine environment with a loop electrode that can be manipulated and activated by the “surgeon.” Activation of the foot pedals allows the surgeon to cut strips of virtual “tissue” from the target leiomyoma.

Virtual Reality Resectoscope Simulator. A realistic resectoscope and camera (upper left) is passed through a mechanical cervix (upper right) with the computer monitor displaying the virtual image (lower left). The operator (center) conducts the exercise using realistic foot pedals (lower right) to “activate” the electrosurgical generator.

The device has 2 integrated software-based skill development and testing exercises that include an introductory “Manipulation Module” (, which requires the operator to sequentially use the electrode to touch 4 randomly located targets within the endometrial cavity; and a “Myoma Resection Module” (, which allows the trainee to perform resectoscopic loop resection of a posterior submucosal leiomyoma. Uterine perforation is simulated if the trainee directs the hysteroscope or electrode in a fashion that could traverse the myometrium.

Targeting Exercise. The operator is randomly directed to touch 1 of 4 targets. The target stays red (left) until successfully touched (right).

Myoma Resection Exercise. The posterior myoma is displayed (A) with the extended loop electrode. The operator activates the loop electrode, which turns orange and resects myoma tissue by moving the operating element (B). This continues until either uterine perforation, 3 minutes have elapsed, or the surgeon has completed the resection (C).

The pilot study had 2 hypotheses:

The device will show face validity in that experts will complete the skill testing in less time than novice hysteroscopists.

Novice hysteroscopists will show significant improvement in their times to successfully complete both skills following a prescribed training program (construct validation).

MATERIALS AND METHODS

The Kaiser Permanente Regional Research Board approved the study. All procedures were performed in the Simulation Center of the Kaiser Foundation Hospital's Los Angeles Medical Center.

The study was designed in 2 phases. Phase 1 was the face validation component that was designed to compare a group of novice hysteroscopists with a group of surgeons with expertise in hysteroscopy and resectoscopic procedures. Phase 2 was designed to measure construct validity, by using the data from the novices enrolled in Phase 1 as a baseline for comparing outcomes following a prescribed training program using the VHRS. The post “training” data were also compared with the data from the 3 “experts” obtained in Phase 1.

The investigators identified 14 subjects from the attending staff and residents of a community Obstetrics & Gynecology residency education program, 3 “expert” hysteroscopists and 11 who were defined as “novices.” Each of the hysteroscopists had more than a decade of extensive experience with diagnostic and operative hysteroscopy, as well as resectoscopic surgery. All participated actively in resident surgical teaching, but none had any exposure to the VRHS. Novice hysteroscopists had no experience with hysteroscopy and comprised both medical students and first-year residents.

For Phase 1, all of the subjects were oriented to the VHRS in a single session that included a test “run” to familiarize themselves with each of the 2 exercises. Then, each subject performed a recorded “baseline” run through each exercise, first with the Manipulation Module and then the Myoma Resection Module. The exercise times, target outcomes, and measurable errors were recorded using the internal recording system and compared. A maximum time of one minute was allowed for the Manipulation Module, while 3 minutes were allotted for the Myoma Resection Module.

For Phase 2, the 11 novice hysterscopists were provided hysteroscopic education that included training in manipulation of the electrode, the lens, and other relevant aspects of resectoscopic surgery. Two “runs” were supervised and then the trainee was left alone, over a period of 2 weeks, to complete a total or 9 postbaseline runs through both tests. Following completion of the ninth session, a final proctored run through each of the 2 exercises was performed, recorded, and then compared to the baseline for the trainee and, collectively, to the expert scores obtained in Phase 1.

Statistical comparisons were performed using paired t tests for the Myoma Manipulation module, while the Wilcoxon rank sum test was used to compare percentage resection of myomas for the Myoma Resection Module. Because this was considered a pilot study, no formal power analysis was performed to determine sample size.

RESULTS

Phase 1

Manipulation Module (Baseline)

All subjects participated in the manipulation test. In the 60-second exercise, each of the experts successfully touched all 4 targets (, without perforation, with a mean exercise time of 33 seconds (. For the 11 novices, the median number of targets touched was 2 (range, 0 to 4) (, one perforation occurred, and the mean exercise time was 57 seconds (. These differences were significant.

Manipulation module results. Targets successfully touched.

Manipulation module results. Mean time to completion.

Myoma Resection Module (Baseline)

All 11 subjects completed the baseline of the myoma resection module, with the experts removing a mean of 97.3% (range, 97% to 98%) of the myoma within the 3-minute exercise time. The novices removed a mean of 66.1% of the virtual myoma (range, 12% to 92%) and one perforation occurred. The differences between “novices” and “experts” were significant (.

Myoma resection. First versus tenth exercises by novices.

Phase 2

Manipulation Module (Run 10)

Seven of the novices made it back within the allotted time for the recorded 10th performance; the other 4 did not complete the program in the prescribed time and, therefore were not included in the analysis. The mean number of targets successfully touched was 4 (. The mean time spent on the Manipulation module was 23 seconds (. The differences from baseline were significant and were similar to those of the “experts” at baseline.

Myoma Resection Module (Run 10)

For the 7 novices who completed the 10th run, baseline resection volume was 65.3% (range, 30 to 92), while at run 10 it was 89% (range, 64 to 99), with one perforation (. These differences approached significance, but there was a singular virtual perforation by one of the novices during run 10. The differences in percentage resection between “novices” and “experts” were still significant, but the differences were greatly reduced.

DISCUSSION

In this pilot study, the VHRS appears to have face validity in that it is able to distinguish between novices and experts, by our definition. Furthermore, the results of this study support the notion that this virtual reality simulation system has construct validity, in that it was able to quantitatively distinguish the experts from the novices, and novices were able, with practice, to improve scores so that they approached those of the experts.

Although the results of the study suggest that the manipulation module may be predictive of the performance of myomectomy, further studies would be necessary to confirm this impression. Furthermore, practice on such a system might be expected to reduce the operating room time spent by novices in the early part of the “learning curve,” a process that adds to the cost of training and/or the cost of care. However, this study was not designed to evaluate this outcome, and, consequently such a conclusion cannot be made at this time.

Simulator training using tissue simulations has been shown to be effective at improving objective structured assessing technique (OSAT) scores in residents at one month, which deteriorates by 6 months.[6] The process of setting up such time intensive laboratories is not conducive to repetitive training and practice, thereby undermining the value of such systems. Virtual reality systems, at least theoretically, can be available to busy residents on a daily or nightly basis, and, theoretically provide the opportunity for more sustained retention of learned skills.

Face validity simply implies that a given test appears to be able to measure a given skill. This VR system is 1 of 2 hysteroscopic VR systems that has been evaluated for face validity.[7] The other hysteroscopic VR simulator, HystSim (VirtaMed, Zurich, Switzerland), has also undergone construct validity evaluation and has been demonstrated to have value in diagnostic hysteroscopy and in manipulation of the instrument, but a surgical intervention such as resectoscopic myomectomy was not tested.[8] Although it is anticipated that training on the current system would reduce the additional time spent in the operating room involved with training residents at operative hysteroscopy and resectoscopy, a well-designed study involving operating room performance would be necessary to prove this hypothesis.

Several aspects of resectoscopic surgery are not assessed by this system. The competencies that were not tested included resectoscope assembly, use of foreoblique (angled) lenses, fluid management, and specimen removal. Many of these aspects could be added to the training provided by the VR system in the context of a comprehensive hysteroscopy and resectoscopic surgery education program. The impact of the training aspect of the study is limited somewhat by the number of subjects who did not return for the follow-up testing. While the sample size appeared to be adequate, it was difficult to organize residents to return within a predetermined period of time to complete the follow-up testing. Nonetheless, our data on the impact of training nearly reached significance, particularly since one of the residents who was performing well on the 7th through 9th sessions perforated the uterus on the 10th.

CONCLUSION

As a result of this pilot study, we feel that we have demonstrated potential value for this system for the early training of novices in hysteroscopic surgery. Further studies with larger sample sizes that include evaluation of impact on both resource and clinical outcomes, including risks and complications, will be necessary to determine the ultimate value of these systems.