Comparison of laparoscopy training using the box trainer versus the virtual trainer.

BACKGROUND AND OBJECTIVES: To evaluate whether training on a virtual reality laparoscopic simulator improves the performance on a laparoscopic box trainer.
METHODS: Twenty-six subjects were trained using a box trainer, and 17 participants were trained using a virtual simulator. Participants in the experimental group completed 1 session of 5 exercises on the box trainer, 4 sessions on the virtual simulator, and a final session on the box trainer. Participants in the control group completed 6 sessions of 5 exercises on the box trainer alone. Exercises were monitored and scored for time and accuracy. Participants completed a self-evaluation survey after each session and a user satisfaction questionnaire at the end of the training.
RESULTS: No significant difference existed between the 2 groups in improvement of accuracy. Pegboard time (P=0.0110) and pattern cutting time (P=0.0229) were the only exercise parameters that improved significantly more in the control group compared with the experimental group. The experimental group developed more interest in a surgical field as a result of their experience than the control group did (70.6% vs 53.8%, respectively).
CONCLUSION: The virtual simulator is a reasonable alternative to the box trainer for laparoscopic skills training.

RCT Entities:   Population Interventions Outcomes

INTRODUCTION

As a result of technological advances in medicine, minimally invasive surgeries are becoming more routine procedures. Due to this fact, the Department of Urology at the Indiana University School of Medicine developed the first laparoscopic elective course for third- and fourth-year medical students. According to Scott et al,[1] the third- and fourth-year medical students derive the most benefit from laparoscopic training. This elective allows medical students interested in general surgery, urology, and gynecology to gain more laparoscopic training using simulators. Laparoscopic surgery requires additional training compared with open surgery. The laparoscopic approach also requires a longer operative time and creates greater stress and fatigue in surgeons.[2]

Laparoscopy challenges surgeons skills on multiple grounds including: an inability to touch tissue, a lack of a 3-dimensional view, counterintuitive fulcrum lead, and the loss of finger dexterity.[3,4] For this reason, the course elective was designed to help medical students and residents develop and improve their laparoscopic skills before they participate in laparoscopic surgery on patients. Trainees may take advantage of unlimited practice time without placing a patient's safety at risk. According to Lucas et al,[5] because of patient safety constraints and financial considerations, achieving proficiency in an operating room through clinical experience has become more challenging. A study by Fried et al[6] regarding surgeons from 5 countries showed that even laparoscopy experts can improve their performance by practicing on simulators.

The purpose of this study was to determine whether training on a virtual reality laparoscopic simulator improves performance on a laparoscopic box trainer in comparison with training on the box trainer alone. The study compares the effectiveness of 2 laparoscopic simulators: the box trainer and the virtual trainer. This study compares the performance accuracy and performance times between 2 groups of participants: the experimental group who trained on the virtual trainer and the control group who trained on the box trainer.

METHODS

Forty-three medical students from the Indiana School of Medicine Department of Urology volunteered to participate in this study. The control group consisted of 26 medical students, and the experimental group included 17 medical students. Both the experimental group and the control group were asked to complete 6 exercise sessions. The control group completed 6 exercise sessions on the box trainer alone using 5 standardized exercises. The experimental group completed the first training session on the box trainer using standardized exercises, followed by 4 training sessions on the virtual simulator, and concluded with a final training session on the box trainer using the initial standardized exercises. The first and sixth box training sessions allowed for a direct comparison between the group that used the box trainer alone and the group that trained using the virtual simulator.

The control group's 6 training sessions involved utilization of a training lab that consisted of a box trainer equipped with a laparoscopic lens, camera, light source, and monitor. The box trainer is covered with an opaque membrane that prevents direct visualization of the working field by the operator. Two 12-mm trocars are placed through the membrane at convenient working angles on both sides of a 10-mm 0-degree laparoscope. The optical system includes a laparoscope, a video camera, a light source, and a video monitor. After watching a 15-minute instructional video, 5 standard exercises were performed using a fixed 0-degree lens. All exercises were monitored, timed, and scored for accuracy by a research assistant. All 5 exercises listed below (a-e) address basic laparoscopic skills: depth perception, manual dexterity, hand-eye coordination, and adaptation to the fulcrum effect. Images of these exercises developed by our laboratory were provided in an earlier publication by Gjertson et al.[4]

a. String running.

Intervals of 12-cm length were marked on a 4-foot long string (modified from Scott et al, 2000[8]). The string is traced from end to end by grasping on the designated marks, holding and stretching for 0.5 s using laparoscopic grasping forceps. If grasp is inaccurate it is noted by the monitor and must be repeated. The test is scored by time only.[4,6]

b. Peg board.

Four pegs are to be transferred from the left side of a peg (cribbage) board to the marked right side position using laparoscopic grasping forceps. During transfer pegs must be picked up by the left hand forceps, transferred to the right hand forceps, and then placed on the marked right side location. This exercise strengthens participants' ambidextrous coordination and spatial orientation skills. The test is scored by time and the number of pegs successfully transferred.[4,7]

c. Pattern cutting.

A 3 × 6-in piece of non-adherent gauze dressing onto which a 4 cm circle is drawn is used for the pattern cutting exercise. The gauze is elevated from the table and suspended with four scaffolds which provide some tension. The circle is to be cut using a grasper in one hand and laparoscopic scissors in the other. The test is scored by time and an accuracy score. The accuracy is measured by the amount of deviation from the pre-marked pattern of the circle on the gauze and reported as a percent error.[4,7]

d. Letter board.

A simple 3 × 3 checkerboard with painted outlines of numbers and letters (modified from Scott et al[8]) and wooden letters and numerals was placed in the box trainer. The letters and numbers were aligned on the perimeter of the board and are picked up and placed over the corresponding outlines as accurately as possible. The test is scored based on time and accuracy. Accuracy is measured based on the linear deviation from the painted outline in the x and y axes divided by length and width of the letter (expressed as a percentage).[4]

e. Intracorporeal knot tying.

A longitudinally slit Penrose drain is approximated using a 13-cm silk suture. The needle needs to pierce through the pre-marked points on the Penrose and then a surgeon's knot, followed by two additional throws, are tied. The exercise is timed. The point system for accuracy includes: 1 point for each millimeter that the needle pierces away from the marks, 10 points for an air knot, and 20 points for a knot that untangles.[4,7]

The experimental group performed the first training session on the box trainer in the same manner as the control group described above. The following 4 training exercises (sessions 2 through 5) were performed on the virtual simulator. The sixth, or final, training session was performed on the box trainer with the same parameters as the control group. The virtual simulator includes an electrocautery pedal, a pair of grasper handles, a pair of needle driver handles, and a camera head. The virtual simulator scored the performance of participants based on time, accuracy, tip trajectories, precision of electrocautery, and excessive traction. The virtual simulator utilized 9 exercises that closely resembled the box trainer exercises. The 9 Virtual Reality (VR) exercises included Zero Degree Target Visualization, Thirty Degree Target Visualization, Visualize and Grasp a Target, Retraction and Dissection, Trace a Transverse Tube, Grasp and Place an Arrow, Dissect a Gallbladder, Place a Knot, and Place a Realistic Free Knot. The Place a Knot exercise guides participants through knot tying while the Place a Realistic Free Knot exercise allows participants to choose the method of knot tying.

After the completion of a training session, participants in both groups evaluated their progress by filling out a survey reflecting their performance after each session. Participants rated their level of emotional comfort, physical comfort, and patience during the exercises. The scores measuring the change in time and accuracy and the surveys from the first and last training sessions were compared between the control group and the experimental group.

Data collection and analysis for the study was done using Microsoft Excel 2007 and SAS 9.1. The statistical tests used include paired t test, unpaired t test, means, and frequency analysis. The level of significance was α=0.05, with all P-values <0.05 considered “statistically significant.” The evaluator of time and accuracy was blinded to which arm the subject was in, to prevent bias.

RESULTS

All 43 participants (100%) completed their self-evaluation surveys. The medical interests of the participants are shown in . The experimental group had a greater interest in urology, laparoscopy, and endoscopy relative to the control group. The student response to the personalized didactic teaching and hands-on laparoscopic training was highly satisfactory.[4] shows the participants' response to survey questions. The control group had a higher perception of what surgery entails, a higher sense of improvement during the training, an increased understanding of live laparoscopic cases, and felt the scoring system more accurately measured their skills at the end of the training sessions. The experimental group had a higher desire to enter Urology and the surgical field.

Based on the changes in time and accuracy of the 5 exercises, the control group significantly improved in all exercises except Pattern cutting (P=0.0971) between the first and sixth sessions (). The experimental group improved significantly between the first and sixth training sessions in all parameters except the Pegboard accuracy (P=0.0557), Pegboard time (P=0.1166), and Checkerboard accuracy (P=0.0581) (). The greatest improvement in time was achieved in suturing in both the control group (-571.6 s) and the experimental group (-388.4 s). These values underestimate the true improvement demonstrated in both groups. Some participants in both groups were unable to complete the suturing task in their first session, but all participants were able to complete the suturing task in their last session.

When the changes in accuracy and time between the 2 groups were compared against each other, the only parameters that were significantly different between the experimental and the control group were the Pegboard time and Pattern cutting time ().

Although the box trainer improved the performance time of the control group in the Pegboard and Pattern cutting exercises significantly more than it did in the experimental group, the experimental group had comparable improvements in both timing and accuracy for the remaining exercises. Self-evaluation surveys showed no significant difference between the improvement levels of emotional comfort, physical comfort, and patience between the first and last sessions among the experimental and the control groups.

DISCUSSION

Our study was designed to compare the effectiveness of the box trainer versus the virtual simulator in helping participants improve their laparoscopic skills. Since simulator training for any surgical technique involves financial and time investments, it is important to prove the value of the simulator to justify the investment.[9] Our study did not find the virtual simulator to be a weaker training method than the box trainer except in regards to the time component of Pegboard and Pattern Cutting exercises in training novices in laparoscopy. This may be explained by the fact that we did not have a large, varied sample of participants to demonstrate its significance. Evaluations revealed that students were extremely satisfied with both training methods. demonstrates the greater influence of the VR simulator on the participants to follow a career in the surgical field (53.8% vs. 70.6%). Because the VR simulator mimics the surgical environment more realistically, it has a more profound effect on the participants. A greater percentage of the control group participants compared with the experimental group (73.1% vs. 64,7%) believed that they were scored fairly. Although the performance of participants on the box trainer was used for comparison, the experimental group was scored on their performance on both the VR and the box trainer exercises. This may have lead to their perception of scoring adequacy. As shown in , the control group did not significantly improve (P=0.0971) in Pattern Cutting accuracy performance, whereas the experimental group did (P=0.0319). However, comparison of their measured improvement, , did not prove be to significantly different (P=0.5086). The 2 groups are not significantly different due to the large enough overlap of their data. This phenomenon is also observed when comparing the Peg Board accuracy and Checker Board accuracy parameters. Both of these parameters show a significant improvement in the control group and an insignificant improvement in the experimental group, yet their difference is small and non-significant when the 2 groups are compared. In addition, both groups showed significant improvement in Pattern Cutting time, but their improvement was significantly different from one another (P=0.0229). This difference shows that even though both methods were effective in improving the participants' speed of Pattern Cutting, the box trainer significantly outperformed the VR.

The study compared improvement of performance in all 9 categories. This comparison looked at the difference between the amounts of improvement in the control group versus the experimental group. Shown in , the experimental group performance improvement was comparable to that of the control group in all categories except the Pegboard time and Pattern Cutting time parameters (respectively, 0.0110 and 0.0229). The improvement in the experimental group may not have measured up to improvement in the control group in the time component of these 2 exercises due to the complexity or difficulty of the task. While the control group had the opportunity to perform these exercises 6 times, the experimental group only dealt with them twice: once during the first session and once during the final session. The suturing exercise was also a challenging task. However, the experimental group had 2 particular VR exercises (Place a Knot and Place a Realistic Free Knot) that provided them with the needed experience to improve their time comparably (P=0.0522). All other parameters were not significantly different between the 2 groups. Overall, the experimental group improved comparatively to the control group in regard to accuracy across all exercises. Because of the extreme importance of accuracy in a surgical setting, our results reinforce the assertion that both training methods have the potential to successfully train medical students in laparoscopic performance. Training with the VR simulator improves laparoscopic performance and reduces errors during surgery, which translates into greater accuracy.[10]

According to our results, training on the virtual reality laparoscopic simulator improves performance on a laparoscopic box trainer enough to be considered a reasonable alternative training tool. Similar studies done by Seymour et al[10] showed VR training results in technical skills acquisition to be at least as good as, if not better than, programs that used conventional box trainers in improving the participants' laparsocopic skills. Many studies illustrate the validity of the box trainer and its significant improvement of laparoscopic skills. Derossis[11] illustrated that residents showed a significant improvement in their performance score with the box trainer involving similar exercises to those in our study. Interestingly, few reports have been published in the literature regarding the effects of training among medical students. Rather, most literature focuses on training residents. As an example, Seymour et al[10] showed that the virtual simulator improves the speed of gallbladder dissection by 1.39 times compared with nonvirtual reality trained residents. In addition, residents who were not trained with the virtual trainer were 9 times more likely to fail to progress during surgery and 5 times more likely to injure the tissues in the body. Although acquiring surgical skills is essential for all residents, this process can begin during medical school. In the recent years, laparoscopic training outside of operating room has been greatly encouraged due to patient safety concerns, resident work-hour restrictions, and an increasingly litigious medical legal environment.[4] Simulators have been used extensively for this purpose, and include cadavers, inanimate trainers, live animal models, and computerized virtual reality simulators.[12]

Although improvement in technical skills is important during laparoscopic training, it is not the only requirement for improved performance in actual surgery. Trainees also need to feel comfortable physically and emotionally with performing such tasks. Our study showed comparable physical and emotional comfort amongst both groups. A similar study by Hamilton et al[13] compared training on a box simulator with a virtual trainer similar to ours. They showed that psychomotor skills of residents improved in both groups with statistical significance (P<0.05). They concluded that the feedback of errors by the virtual simulator helped improve the residents' precision as well as the box trainer, reinforcing our study results.

The main advantage of the virtual reality simulator over the box trainer is its ability to provide objective measurement of training progress for each participant without requiring supervision.[14] The tasks are measured accurately and without bias. VR surgical simulators replicate critical laparoscopic skills and procedures through their advanced computer software. These simulators provide a more believable practice environment with higher face validity than box trainers. The VR simulators can be set up to accurately measure and save participants' performances and provide feedback to them. The software can also be modified to accommodate a pass/fail system for trainees. Another feature is the opportunity for participants to independently practice their skills as a structured curriculum on their own time.[15] Although the box trainer is inexpensive and allows for practice with real instruments, the VR simulators offer a more realistic environment that mimics actual surgery.[5] However, certain disadvantages of VR exist, such as the lack of force feedback, which moves the experience away from reality.[16] A VR simulator with haptics feedback would alleviate this disadvantage.[17] Seymour et al[10] showed that the mean errors of the virtually trained group were 6 times less than normal. Although Virtual Reality software represents the real environment to a great extent during basic tasks, it still can improve in complicated tasks, such as tissue manipulation and suturing to best resemble a real surgical setting.

This study was not without limitations. The heavy schedule of medical students made it difficult for them to finish the sessions on the scheduled optimal 2-week period. Some participants took up to 6 months to finish all of their sessions, which can negatively affect the accuracy of the data collected. Due to the different heights of participants, instruments were adjusted at each session, which may have introduced errors and bias into the data. Selection of participants by faculty members in the Department of Urology may have introduced a selection bias that could reduce the internal validity of the study. The small sample size of the experimental group (n=17) was another limiting factor. A larger experimental group may have demonstrated statistical significance in both the Pegboard and Checkerboard times rather than a trend towards significance. We were not able to investigate whether VR training is the most effective training method for laparoscopic naïve participants who have greater potential to improve compared with experts.[1,5] Also, due to the small sample size, some variables did not possess a normal data distribution. Recommendations for further research include a similar cohort study with a larger sample size used to assess and compare the improvement of medical students and residents. This comparison will assess the notion that laparoscopic training is more effective when introduced at earlier stages of medical education.

CONCLUSION

Our study shows that participants were satisfied with the educational role of the virtual trainer. Training on the Virtual Reality simulator was comparable to training on the box trainer in our study in all exercise parameters except two. No statistically significant difference between the self-evaluation progress of the experimental group and control group existed. The virtual simulator is therefore a reasonable alternative to the box trainer for laparoscopic skills training. ()

Laparoscopy training laboratory.

Peg board.

Pattern cutting exercise.

Letter board.

Virtual laparoscopy simulator.

Table 1.

Interests of Participants

Interests	Control Group	Experimental Group
Surgery	88.5% (n=23)	88.2% (n=15)
Urology	57.7% (n=15)	70.6% (n=12)
Laparoscopy	69.2% (n=18)	88.2% (n=15)
Endoscopy	46.2% (n=12)	76.5% (n=13)
Minimal Invasive Radiologic/Vascular Procedure	65.4% (n=17)	64.7% (n=11)

Table 2.

Final Survey Response Comparison Between the Groups

Survey Question	Control Group	Experimental Group	P Value
1. Laparoscopy training course added to my perception of what surgery entails.	92.3% (n=24)	88.2% (n=15)	0<0.0001
2. The laparoscopy training altered my desire to enter a surgical filed.	53.8% (n=14)	70.6% (n=12)	0<0.0001
3. The laparoscopy training altered my desire to enter Urology.	46.2% (n=12)	47.1% (n=8)	0<0.0001
5. After the last training session, I felt that I improved.	96.2% (n=25)	94.1% (n=16)	0<0.0001
6. The laparoscopy training increased my understanding of actual laparoscopic cases that I participated in?	84.6% (n=22)	82.4% (n=14)	0<0.0001
9. I feel that the scoring system adequately measured my skills.	73.1% (n=19)	64.7% (n=11)	0<0.0001

Table 3.

Accuracy and Time Improvement Within the Control Group

Variable	Accuracy–Mean Percent (SD)	P Value	Time–Mean Seconds (SD)	P Value
Pegboard	0.9615 (1.0763)	<0.0001	−160.4 (128.89)	<0.0001
Pattern Cutting	0.0731 (0.2162)	0.0971	−397.7 (206.36)	<0.0001
Checker Board	0.0627 (0.1288)	0.0202	−152.6 (101.71)	<0.0001
String Running	×	×	−140.8 (101.09)	<0.0001
Suturing	6.6154 (7.7927)	0.0002	−571.6 (343.2)	<0.0001

Table 4.

Accuracy and Time Improvement Within the Experimental Group

Variable	Accuracy–Mean Percent (SD)	P Value	Time–Mean Seconds (SD)	P Value
Pegboard	0.5882 (1.1757)	0.0557	−52.65 (130.87)	0.1166
Pattern Cutting	0.0449 (0.0787)	0.0319	−239.1 (228)	0.0005
Checker Board	0.0456 (0.0922)	0.0581	−105.6 (83.135)	<0.0001
String Running	×	×	−142.1 (80.271)	<0.0001
Suturing	5.2667 (6.933)	0.0107	−388.4 (406.26)	0.0024

Table 5.

Difference in Accuracy and Time Between the Experimental and Control Groups

Variable	Accuracy (SD)	P Value	Time (SD)	P Value
Pegboard	0.3733 (1.1162)	0.2899	107.74 (129.66)	0.0110
Pattern Cutting	0.0364 (0.1752)	0.5086	158.54 (215.06)	0.0229
Checker Board	0.0052 (0.1167)	0.8869	46.93 (94.892)	0.1205
String Running	×	×	−1.29 (93.519)	0.9650
Suturing	1.9683 (7.3916)	0.3982	228.91 (366.97)	0.0522