Effect of virtual reality training on laparoscopic surgery: randomised controlled trial.

OBJECTIVE: To assess the effect of virtual reality training on an actual laparoscopic operation.
DESIGN: Prospective randomised controlled and blinded trial.
SETTING: Seven gynaecological departments in the Zeeland region of Denmark. PARTICIPANTS: 24 first and second year registrars specialising in gynaecology and obstetrics.
INTERVENTIONS: Proficiency based virtual reality simulator training in laparoscopic salpingectomy and standard clinical education (controls). MAIN OUTCOME MEASURE: The main outcome measure was technical performance assessed by two independent observers blinded to trainee and training status using a previously validated general and task specific rating scale. The secondary outcome measure was operation time in minutes.
RESULTS: The simulator trained group (n=11) reached a median total score of 33 points (interquartile range 32-36 points), equivalent to the experience gained after 20-50 laparoscopic procedures, whereas the control group (n=10) reached a median total score of 23 (22-27) points, equivalent to the experience gained from fewer than five procedures (P<0.001). The median total operation time in the simulator trained group was 12 minutes (interquartile range 10-14 minutes) and in the control group was 24 (20-29) minutes (P<0.001). The observers' inter-rater agreement was 0.79.
CONCLUSION: Skills in laparoscopic surgery can be increased in a clinically relevant manner using proficiency based virtual reality simulator training. The performance level of novices was increased to that of intermediately experienced laparoscopists and operation time was halved. Simulator training should be considered before trainees carry out laparoscopic procedures. TRIAL REGISTRATION: ClinicalTrials.gov NCT00311792.

RCT Entities:   Population Interventions Outcomes

Introduction

Laparoscopy has become the standard approach for many conditions in most surgical specialties.1 2 3 This development has been driven by the desire for less surgical trauma, faster postoperative recovery, shorter hospital stay, and better cosmetic results, and a sales drive by the medical industry.4 It is evident, however, that laparoscopy is associated with a longer operation time and a higher rate of surgical complications during the learning curve of the surgeons. This has been verified in many different specalties, including general5 6 urological,7 8 paediatric,9 and gynaecological surgery.10 The possibility of overcoming these problems during the learning curve by appropriate training and ensuring that surgeons perform a sufficient number of procedures has also been documented.11 The technical skills needed for laparoscopic surgery are fundamentally different from those for traditional open surgery, leading to a prolonged learning curve. The primary obstacles in learning laparoscopy are psychomotor and perceptual. The unique nature of laparoscopic surgery combined with an increasing focus on patients’ safety and rights, the present decrease in working hours, and concern over costs of operating theatre time are factors that challenge the traditional surgical approach and contribute to a growing need for novel methods in the training of laparoscopic surgeons.12 Although virtual reality simulation has the potential to offer important advantages in the area of training for new skills and procedures, evidence on the transfer of skills from the simulated environment to the operating theatre is still limited.13 14

We investigated the impact of training using a virtual reality simulator on the quality of skills acquired for a key gynaecological procedure. The investigation was carried out as a prospective, randomised, observer blinded, controlled trial, according to the guidelines of the consolidated standards of reporting trials (www.consort-statement.org).

Methods

From September 2006 to June 2007, 32 trainees in gynaecological specialty training years 1 and 2 (postgraduate years 3-8; see box), with no experience of advanced laparoscopy (defined as all laparoscopic procedures involving coordination of more than one instrument), were included in the study. Of a total cohort of 42 (38 women and four men) trainees in the region, eight were not eligible, as they were too experienced and four came from the two departments that did not participate in the trial. Of the remaining 30 eligible trainees, the first 24 who volunteered were enrolled. They came from seven of nine gynaecology departments in the Zeeland region of Denmark (population 2.3 million): Gentofte hospital (five trainees), Herlev hospital (n=4), Roskilde hospital (n=4), Hilleroed hospital (n=1), Holbaek hospital (n=1), Hvidovre hospital (n=2), and Rigshospitalet hospital (n=7).

Preregistration house year—one year

Introduction before specialist training—one year

Optional additional training to qualify for further specialisation (possibility for writing PhD thesis)—1-3 years

Specialist training—four years

Randomisation and blinding

To ensure that the trainees’ baseline characteristics were similar within and between each group, we chose a stratified randomisation based on previous experience of simple laparoscopy (defined as laparoscopic procedures performed using a single instrument, such as diagnostic laparoscopic sterilisation (clips) or diagnostic laparoscopy). The Clinical Trial Unit at Copenhagen University independently randomised the trainees by computer to intervention or control groups. The randomisation procedure was concealed and achieved by using the trainees’ unique personal identification number (central personal register number).

Trainees in the intervention group were given an oral introduction to the simulator and the rating scale used for outcome measure. Any operations done during the study were recorded. Owing to the nature of the trial it was not possible to blind the trainees to their allocated group, but all involved departments, supervisors, and staff in the operating theatres were blinded to the trainee’s group, and the assessors of outcome were blinded to both the trainee and their allocated group. The primary investigator saw the data only after completion of all assessments and once data had been loaded in a database.

The control group was to continue standard clinical education. During the study no trainee in either group was allowed to perform advanced laparoscopic surgery, only simple laparoscopy or to assist senior colleagues. To check that randomisation had been successful, the control group were trained in the simulator after the trial. Their baseline performances were indistinguishable from those of the intervention group.

Equipment

The virtual reality laparoscopy simulator program (LapSim Gyn v 3.0.1; Surgical Science, Gothenburg, Sweden) was run on an IBM T42 computer in a docking station (Pentium M 1.8 GHz/512 MB RAM; IBM, Armonk, NY, USA) using an interface with a diathermy pedal (Virtual Laparoscopic Interface; Immersion, San Jose, CA, USA). The operations took place in the operating theatres of the participating departments and were recorded on DVD using a camera attached to the laparoscope for later blinded evaluation. During the operation one of the authors (CRL or designated TD) observed the procedure to record the handling of the surgical instruments, any involvement of the supervisor, whether the standard procedure for the operation was followed, and whether the recording was done correctly, finalised, and assessed.

Simulator training

The intervention group undertook a specific training programme in the simulator. The programme comprised two parts: firstly, training in the two basic skills of “lifting and grasping” and “cutting” during which the trainees were introduced to the simulator environment and the different instruments; secondly, one procedure specific task in which the trainee had to carry out a complete right sided salpingectomy while preserving the ovary. The training in basic skills was done once in each training cycle of 45-60 minutes and the salpingectomy repeated continually during the remainder of the cycle. The simulator provided the trainees with instant feedback on time, path length and angular path of the instruments’ movements, bleeding, cutting of uncoagulated arteries, and use of diathermy on non-target tissue. The training sessions were repeated until the expert criterion level was reached in two consecutive and independent simulations. The proficiency criteria were established by experts in previous studies of construct validity and learning curves.15 16 The requirements and settings of the simulator are available at www.skopisimulator.rh.dk.

Surgical procedure

The trainees performed their first salpingectomy at their local gynaecological department and were supervised by a senior colleague who was told about the purpose of the trial. To make comparison of performance easier, the trainees all carried out procedures on the right side. The patients were admitted for elective salpingectomy before treatment for infertility or for prophylactic removal of fallopian tubes and ovaries owing to a positive test result for breast cancer gene 1 (BRCA1). The trainees were not allowed to operate on patients who had undergone previous open or laparoscopic surgery below umbilicus, had possible abdominal malignant disease, had an American Surgical Association score ≥3 (patients with severe systemic disease), had a body mass index less than 18 or more than 27, had haemophilia, or had other factors of potential influence on the surgical procedure. The operations followed a modified standard procedure on the basis of expert consensus.17 18 The supervisors were allowed to give oral instructions only, and one researcher was present to observe the procedure and to record who handled the instruments.

Outcome measures

The primary outcome measure was technical performance, measured as total score (10-50 points) using the objective structured assessment of laparoscopic salpingectomy, which comprises a five item general rating scale and five item task specific rating scale.19 Two independent observers blinded to trainee and allocated group assessed the recorded operations. The secondary outcome measure was operating time in minutes. The reliability of the assessment was determined by calculating the inter-rater agreement (number of agreements for each of the assessed items divided by total number of assessed items) and the γ coefficient. We present outcomes as medians and interquartile ranges.

Power calculation

The power calculation was based on a previous validation study on the procedure specific scale of the objective structured assessment of laparoscopic salpingectomy.19 This study showed a difference of six points between novice laparoscopists (0-5 procedures) and intermediately experienced laparoscopists (30-50 procedures). An improvement of skills to the level of 30 or more points was considered acceptable. On the basis of these findings we chose the minimal relevant difference to be six points. We determined that with an α of 0.05 (two sided) and a power of 80% (β=0.2 giving Zα=1.96 and Zβ=0.84, largest SD=4.40) we required 18 or more trainees. To compensate for possible drop outs, we added an additional third to the 18, totalling 24 trainees.

Statistical analysis

We present cumulated scores as medians (average score of two observers), compared using non-parametrical analysis (Mann-Whitney U test). We considered a two tailed P value of 0.05 or less to be statistically significant and an inter-rater agreement and γ coefficient of 0.8 or more for each to be acceptable. Analysis was done using SPSS 13.0 for Windows. Graphics were made using Graph pad Prism (Graph Pad, San Diego, CA, USA).

Results

Eight of the total cohort of 42 trainees (38 women, four men) were ineligible for the study as they were too experienced and four came from the two departments not participating in the trial. The remaining 30 trainees agreed to participate. The first 24 were enrolled; 22 (90%) were women, representing the current sex distribution among trainees in obstetrics and gynaecology in Denmark (figure ). The average age of the trainees was 32.8 years (range 26-42 years), and 23 were right handed. Eleven trainees were randomised to virtual reality training in laparoscopic salpingectomy and 10 were randomised to traditional clinical education. Table 1 shows the baseline characteristics of the trainees. Two trainees were subsequently excluded from the simulator trained group because one failed to complete the training programme and the other was involved in an operation that was cancelled because of anatomical abnormality and suspected malignant disease in the patient. One trainee was excluded from the control group because of a technical fault in the DVD recorder used to record the operation.

Flow of trainees through trial

Table 1

Baseline characteristics of gynaecology trainees randomised to virtual reality simulator training in laparoscopic salpingectomy or to standard clinical education (controls). Values are numbers of trainees unless stated otherwise

Characteristics	Simulator trained group (n=13)	Control group (n=11)
Men	1	1
Women	12	10
Mean (range) age (years)	33.3 (30-42)	32.4 (26-38)
Experience of simple laparoscopy	6	5
No experience of simple laparoscopy	7	6

The median total score on the general and task specific rating scale reached 33 points (interquartile range 32-36 points) in the simulator trained group and 23 (22-27 points) in the control group (P<0.001, table 2).

Table 2

Impact of virtual reality simulator training on surgical performance and operation time. Values are medians (ranges; interquartile ranges) unless stated otherwise

Outcome measure	Simulator trained group (n=11)	Control group (n=10)	P value*
Surgical performance:
Total score (points)	33 (25-39; 32-36)	23 (21-28; 22-27)	<0.001
% reaching ≥30 points	82	0
Operation time:
Total time (minutes)	12 (6-24; 10-14)	24 (14-38; 20-29)	<0.001

Inter-rater agreement 0.79. γ-coefficient 0.83 (95% confidence interval 0.68 to 0.98).

*Mann-Whitney U test.

The median total time to complete the procedure was 12 minutes (interquartile range 10-14 minutes) in the simulator trained group compared with 24 (20-29 minutes) in the control group (P<0.001, table 2). Twenty one operations were assessed.

The median number of simulated salpingectomies needed to reach the proficiency level in the simulator trained group was 28 (24-32 salpingectomies). The control group was offered simulator training after the study operation; nine of the 11 trainees in this group volunteered and a median of 26 (23-32) simulated operations were needed to reach the proficiency level (P=0.70). The mean time spent on training using the simulator was 7 hours and 15 minutes (5h 30 min-8h 0 min) in the intervention group and 7 hours and 0 min (5h 15 min-7h 45 min) in the control group (P=0.65; table 3). The baseline score (first attempt) was 8 (5-15) in the simulator trained group and 9 (7-19) in the control group after training (P=0.70; table 3). All trends of differences in baseline characteristics were not statistically significant.

Table 3

Number of sessions and duration of training in virtual reality simulator training programme in intervention group before training and in control group after surgery

Variable	Simulator trained group (n=11)	Control group (n=9)*	P value†
No (range) of training sessions	28 (16-39)	6 (19-43)	0.76
Duration (range) of training	7h 15m (4h 15m-9h 30m)	7h 0m (4h 0m-9h 15m)	0.70
Median (range) score on first attempt (%)	8 (5-15)	9 (7-19)	—

*Voluntary simulator training after surgery.

†Mann-Whitney U test.

The time used by the assessors to fill in the rating chart was the mean total operation time plus five minutes for each DVD recording. The inter-rater agreement was 0.79 (166/210). The γ coefficient used to investigate strength of correlations among the observers at single subject level reached 0.83 (95% confidence interval 0.69 to 0.99).

Discussion

Proficiency based virtual reality training in laparoscopic salpingectomy compared with standard clinical education was associated with a clinically important improvement of operative skills during the actual procedure. The learning curve in the operating theatre was also shorter. On the rating scale used in this study, which had previously been validated in a separate investigation, novices (fewer than five procedures) scored a median 24 points, and intermediately experienced trainees (20-50 procedures) a median 33 points compared with a median 39 points for experts.19 The clinical implications of the present findings are thus extensive. After training in a specific procedure to a predefined (proficiency based) level inexperienced trainees progressed from the performance level of a novice to that of an intermediately experienced gynaecologist, assessed in their first complex laparoscopic procedure. By using simulator training it might be possible to bypass the early learning curve, which is known to be associated with an increased rate of complications.20 This study was not designed to investigate complication rates, and conclusions in this area must be drawn cautiously. In general it is difficult to use patient outcomes to evaluate a medical training course. Firstly, in contrast with trials of a single intervention (for example, a new drug) medical education is a complex intervention involving many interconnecting parts and different layers.21 Secondly, assessment of surgical technical skills of individual trainees will need to be based on surrogate end points rather than outcomes such as morbidity or mortality because it is an ethical imperative that an operation performed by a supervised novice ought to have the same outcome as that of the supervisor. Training may cost time and some inconvenience for the patient but should never jeopardise safety or outcome. Thirdly, to show differences in outcome, based on a training course, the numbers of trainees should by far outnumber the total number of trainees available, thus making such a trial unfeasible.

Operating time

Although operating time might be greater with novice surgeons, the outcomes of a supervised operation ought to be the same. The time to complete the laparoscopic salpingectomy was reduced by half. As the operating theatre serves both productivity and educational purposes, shorter operation times are of benefit.

The present results emphasise that by using virtual reality simulator training the surgical community can meet the need for proficiency based basic training in laparoscopy. These results also show that criterion based procedural training using a virtual reality simulator can help compensate for reduced working hours by bringing trainees to a higher level of performance more quickly. Traditional training depends on the supply of suitable procedures for training purposes, whereas simulator training can be used according to demand. To achieve an average of 28 salpingectomies can take a year or more in clinical practice, compared with eight hours of intensive training using the simulator.

Finally, reducing the operating time by half, from 24 minutes in the control group to 12 minutes in the simulator trained group, might not seem important. Nevertheless it has been estimated that during residency the additional costs of longer operating time is about (1997) $48 000 (£31 841.00; €35 907.00) per graduate.22 For more extensive surgery or just a bilateral procedure, increased use of simulator training could reduce novice operating time substantially, easing the pressure on the often limited capacity for training in the operating theatre.

Transfer of skills

Despite virtual reality simulators being introduced more than a decade ago, evidence on the effect of simulators on performance in the operating theatre is still sparse.13 To date no published studies on the transfer of technical surgical skills from simulator to real operations had exceeded grade 2a evidence; in our study the level of evidence is 1b. These findings are supported in a systematic review,14 which also showed that the randomised controlled trials on effect of simulator training were generally of poor scientific quality. The major problems were randomisation with inadequate concealment of allocation, lack of power calculations or insufficient sample sizes, lack of observer blinding, insufficient training time or goals, non-standardised comparators in control groups, and disparate interventions in intervention groups. Most studies also used surrogate end points such as bench station or simulator tests to establish the transferability of skills, instead of assessing skills in real operations. The conclusion of the meta-analysis was that only few studies possess the necessary quality, that two studies showed a positive effect23 24 (real operation) and one study no effect25 (simulated operation) of simulator based training. Another common feature of the previous studies is that they were all carried out using basic skills rather than procedure specific simulation. We used a procedural simulator, which provides training in both psychomotor and cognitive skills, thereby also improving knowledge of the surgical procedure and its potential pitfalls. There are probably several reasons for the significant impact on both performance and time in our study compared with previously published studies. Firstly, the simulator provides a realistic graphic presentation of anatomy in the surgical field and a good feedback system enabling the trainees to adjust their performance. Secondly, using predefined training goals (the expert proficiency level) instead of fixed time or numbers, encouraged the trainees to rehearse until they reached the maximal effect of the simulator training. Thirdly, instead of selecting medical students who might not choose a surgical career, we studied highly motivated trainees who needed to learn laparoscopic skills for the future.

We measured the impact of simulator training on a salpingectomy, which is an important gynaecological laparoscopic procedure and can be viewed as a key operation possessing all the core skills needed for most laparoscopic procedures. We did not test external validity and reproducibility beyond the specialty of gynaecology. However the skills that were improved can probably be considered as genuine core skills, which would be beneficial across the surgical specialties.

The flow of an operation is based on both technical skills and knowledge of the procedure, and therefore we expect the impact of simulator training would apply to other laparoscopic procedures, although the effect would be more visible on the technical side than in the knowledge of procedural skills. These suppositions are supported by a contemporary Swedish study on procedural virtual reality simulator training of cholecystectomy, which reached conclusions similar to those of the present study.26

Observer reliability

In the present study the γ coefficient showed that there was no systematic discrepancy among the raters. The inter-rater agreement also reached a sufficient level, which is evidence of a valid and reliable assessment.

The investigation was carried out in the same way that a curriculum integrated training course most likely would be implemented. The internal consistency of the trial could have been higher if all the trainees had operated in the same theatre, using the same technical equipment, and with the same supervisor and staff. However, by showing the effects of simulator training in settings closely resembling a regional simulator training course the external validity was improved. The primary investigator helped the trainee to use the simulator and introduced the different training modules but did not teach laparoscopic techniques. The feedback on performance was based on assessment in the simulator. A designated supporter at the training session could, however, be a source of bias. A setting where trainees practise by themselves could eliminate this potential source of bias. Finally, performing laparoscopic surgery also consists of identifying diseased anatomy, communication, teamwork, decision making,27 and leadership, alternative plans, and conversion to open surgery if needed.28 29 These non-technical skills are taught in the currently existing virtual reality systems to a limited degree only. In this study we did not provide training in these non-technical skills or assess them. Simulator training should probably be considered only as a supplement or preoperative training; further education and practice in the operating theatre as well as further development of more complex virtual surgical environments (hybrid simulation)30 is still required.

Conclusion

It is possible to transfer skills acquired during proficiency based training using a virtual reality simulator to a real operation. Training in proficiency based skills should be incorporated in a comprehensive surgical training and assessment curriculum for all residents before they operate on real patients. This can potentially improve patients’ safety and improve efficiency in the operating theatre.

The European Working Time Directive has put extra pressure on surgical training programmes

Virtual reality simulators could contribute to the training of core skills for laparoscopy

High grade evidence of the effect of virtual reality simulator training on real operations is sparse

Training using a virtual reality simulator improved performance in a laparoscopic procedure