Evaluation of applied laparoscopic urology course using validated checklist.

BACKGROUND AND OBJECTIVES: The objectives of this study were to investigate the effectiveness of the applied laparoscopic urology course using a validated checklist and to determine any differences in laparoscopic skills achieved by the participants at the end of the course period based on whether they began their training in a dry or wet laboratory.
METHODS: To facilitate the mastering of challenging laparoscopic skills by urologists, a unique 3-day mini-training program was established at the Gulhane Military Academy of Medicine, Surgical Research Center, Ankara, Turkey. Only 30 trainees were accepted in each course, and they were divided into 3 subgroups. The primary outcome of the study was the changes in the performance and task accomplishment duration of the trainees at the beginning compared with the end of the course. The secondary outcome was any differences in the basic skills of the trainees based on whether they started their training in the dry or wet laboratory.
RESULTS: The overall laparoscopic skills, which were evaluated by use of a standardized laparoscopic suturing task score, significantly improved (18.8 to 26.0, P < .001), and the time needed for task accomplishment decreased throughout the course (9.5 minutes to 5.25 minutes, P = .002). With respect to the course design, laparoscopic skills scores and the times needed for task accomplishment showed no statistically significant changes at the end of the course despite the fact that the trainees had started their training at different stages.
CONCLUSION: The applied short-term laparoscopy course was shown to be an effective format particularly for achieving laparoscopic skills in which suturing and knotting are essential. This is mainly achieved through close cooperation in dry and wet laboratories.

INTRODUCTION

The modern operating room is recognized as not being the ideal learning environment for laparoscopic surgeons because of increasing time constraints, costs, stress, and ethical considerations because they need to acquire very different skills compared with the surgeons conducting open surgery. Many of the laparoscopic procedures required have a significant learning curve.[1] Hence, different institutions offer different urologic laparoscopic training programs.[2-4] These programs, aiming to train urologists to apply laparoscopy effectively, can be for extended or short periods.

For the past 5 years, our institution has offered applied laparoscopic urology courses in our research laboratory with practice in both box trainers (dry-laboratory training) and live-animal models (wet-laboratory training). In the applied laparoscopic urology courses, course participants are generally trained in the dry laboratory before they can proceed to the wet laboratory; however, some participants start in the wet laboratory because of the course concept.

Short applied laparoscopy courses are known to be effective in the development of laparoscopic skills and performances of their participants.[5-7] Nevertheless, their role and efficacy have not been comprehensively assessed by use of a validated checklist.

The primary objective of this study was to investigate to what degree the applied short-term laparoscopy course conduces advancement in laparoscopic skills by use of a validated checklist. The secondary objective was to determine the differences in laparoscopic skills achieved at the end of the course period among participants based on whether they started their training in the dry or wet laboratory.

METHODS

Course Design

To facilitate the mastering of challenging laparoscopic skills by urologists, the biannual Applied Laparoscopic Urology Courses and Symposia have been organized at the Gulhane Military Academy of Medicine Department of Urology, in collaboration with the Gazi University School of Medicine Department of Urology and the EAU Section of Urotechnologies in Ankara, Turkey, since 2005.

Overall, each course has an intensive graduate training program comprising real dry-laboratory application of laparoscopic instruments, hands-on training, development of stereotactic abilities, a video-based theoretical session, and real wet-laboratory training on live porcine tissue. There are 3 major structural components: a video-based theoretical session, a dry laboratory, and a wet laboratory. Only 30 trainees are accepted in each course, and they are divided into 3 subgroups. The trainees were divided into groups according to course application order. The first group participates in a video-based theoretical session, the second group trains in the dry laboratory, and the third group trains in the wet laboratory. After 4 hours, the groups rotate, and thus the first group proceeds to the dry laboratory after the video session, the second group to the wet laboratory where live-animal surgery is conducted on pigs, and the third to the video session. Each group has the opportunity to participate in the same field twice.

Study

At the beginning of the study, the study protocol was explained to all trainees and their written consent was taken. The demographic data of all trainees were obtained by use of the Resident Experience Questionnaire (European Urological Residents Education Program 2007).[1] In light of the information provided, those who had participated in other laparoscopy courses or who were practicing laparoscopy in their clinics were excluded from the study. The remaining trainees were included in groups according to the course application sequence.

The evaluation of suturing and knot-tying skills on the training box was conducted with a 29-point scoring checklist for assessing the performance of the standardized laparoscopic suturing task. This checklist was also used in the assessment of laparoscopic suturing skills of urology residents in the Pan-European Study and prepared according to a laparoscopic knot that was defined and described point by point on a previously constructed, validated checklist[1,8] (). Technical features of the task such as needle position and driving, pulling through of the suture, and the technique and quality of the knots were scored on this checklist.

Twenty-nine-point scoring checklist for assessing performance of standardized laparoscopic suturing task.

Qualitative analysis of the trainee's suturing performance was performed by 3 observers (S.G., C.G., and F.S.) who were blinded to participants' subgroup assignment. The performance and time required for task accomplishment of the trainees were determined at the beginning of the course using the 29-point scoring checklist. The second and last evaluation was conducted at the end of the course. The initial and final mean checklist scores were compared for each group. The change in mean checklist scores for each group was also compared between groups who had started with dry- and wet-laboratory training.

The primary outcome of the study was changes in the performances and required times for task accomplishment of the trainees at the beginning compared with the end of the course. The secondary outcome was any differences in the basic skills of the trainees based on whether they started the program in the dry or wet laboratory. Hence, the trainer in the dry laboratory recorded the scores and the duration required for completion of standardized laparoscopic suturing tasks after the initial training and at the end of the course.

Statistical Analysis

In this study all statistical analyses were calculated with the SPSS statistical package program (version 15.0; IBM, Armonk, NY, USA), and the results were evaluated according to a significance level at .05. Because of the small number of observations and the non-normally distributed data, nonparametric statistical methods were used for all analyses. The χ2 test was used in cross tables to evaluate the dependency between categorical variables. The Mann-Whitney U test was used for comparing two independent samples on continuous variables, and the Wilcoxon test was used for comparing two dependent samples on continuous variables. The related descriptive statistics, test values, and corresponding P values are presented in . For comparisons of the laparoscopic scores achieved at the end of the training according to the group in which participants started their training, a two independent-samples nonparametric Mann-Whitney U test was used. Cronbach α reliability analysis was used to evaluate the concordance among the observers.

RESULTS

Of 30 trainees, 23 were included in the study; 7 were excluded because of previous experience in laparoscopy, either by participating in a training course or by practicing laparoscopy in their clinics.

The overall laparoscopic skills, which were evaluated by use of a standardized laparoscopic suturing task, significantly improved throughout the short-term applied laparoscopy course (). In the group starting in the dry laboratory (group I), the laparoscopic suturing task scores rose significantly from 18.8 to 26.0 (P < .001) and the time needed for task accomplishment decreased from 9.5 minutes to 5.25 minutes (P = .002). According to the course design, in the group beginning in the wet laboratory and proceeding to the training box (group II), the laparoscopic suturing task scores rose significantly from 19.1 to 26.8 (P = .017) and the required time decreased from 8 minutes to 5.29 minutes (P = .027) ( and ).

Changes in laparoscopic skills scores of group I and II trainees at beginning and end of course.

Changes in laparoscopic task accomplishment time of group I and II trainees at beginning and end of course.

Because of the course concept, trainees were divided into dry- and wet-laboratory groups: some began their training in dry laboratories, whereas others began training directly on living animals. A comparison of the trainees' laparoscopic skills scores and the times needed for task accomplishment at the end of the course showed no statistically significant changes despite the fact that they had started their training at different stages (). For groups I and II, the initial task scores were 18.88 and 19.14, respectively, and the final task scores were 26.0 and 26.86, respectively. The comparison of the laparoscopic skills scores was not different between groups. For groups I and II, the initial task accomplishment times were 9.50 and 8.00 minutes, respectively, and the final accomplishment times were 5.25 and 5.29 minutes, respectively. There was no significant task accomplishment time difference between groups.

There were no statistically significant differences among the observers' scores (Cronbach α reliability analysis, 0.99 before course and 0.99 after course).

DISCUSSION

Laparoscopic techniques are difficult to master, especially for surgeons who did not receive such training during residency. Recently, Stolzenburg et al.[3] provided an algorithm on how urologic laparoscopy is to be conducted. In vitro and in vivo training models help urologists acquire basic laparoscopic skills such as hand-eye coordination, depth perception, and knot tying. For this reason, many laparoscopic training options have been developed over the years. These vary from bench-top training courses with box trainers to virtual-reality simulators (“dry-laboratory” training) or live-animal models (“wet-laboratory” training).[1] According to this algorithm, short-term courses are the supporting stages in the acquisition of urologic laparoscopy. Training would start with simulated models and clinical situations in the skills laboratory until predefined proficiency criteria are met. Only then would the trainee be able to progress to clinical situations on the ward and in the operating room.[4,9]

In this study we evaluated how much the course participants benefited from the short-term laparoscopy course that we have been conducting for the past 5 years by means of a validated checklist. The study showed that the trainees showed great improvements in their basic laparoscopic skills. In line with the course concept, trainees were divided into different training groups. Thus, whereas some began their training in dry laboratories, others began training directly on living animals. In the relevant literature, it was suggested that the trainees continue with wet-laboratory training after some time spent in dry laboratories.[5] Our study has shown that no differences were found in the acquisition of laparoscopic skills by the trainees who were divided into different study groups. The dry- and wet-laboratory trainings are important steps in laparoscopic surgery education; however, there is a need for a validated standardized training program about how to provide this education in short-term courses.

Van Sickle et al.[8] described and validated a novel assessment methodology applied to laparoscopic intracorporeal suturing and knot tying. This method was modified and adopted in various studies including the laparoscopic suturing skills of urology residents in the Pan-European Study.[1] In the study reported by Van Sickle et al., for the construct validity of the objective assessment method, they used 4 subject groups (experts, trained residents, untrained residents, and medical students). The demographic characteristics of untrained residents and medical students were similar to those of the trainees in our study. The mean suturing time was 445 seconds (untrained residents) and 558 seconds (medical students). In our study the task accomplishment time was 9.5 minutes and 8 minutes in group I and group II, respectively, at the beginning of the course. These results were comparable. Kroeze et al.[1] used a 29-point checklist to determine whether there were performance differences for the completion of a simulated laparoscopic suturing task among urology residents based on their postgraduate year of training. The time allowance for suturing task performance was 10 minutes to limit the frustrations of laparoscopically inexperienced junior residents. With regard to the time needed to complete the laparoscopic suturing task, 21% of the participants were unable to complete this task within the allotted 10 minutes. The checklist scores were 13, 14, and 10.5 in junior residents (postgraduate years 1 and 2), intermediate residents (postgraduate years 3 and 4), and senior residents (postgraduate years 5 and 6), respectively. In our study the mean laparoscopic suturing task scores were 18.8 and 19.1 in group I and group II, respectively, at the beginning of the course. The differences between these two studies might stem from the demographic variety of the participants.

Vlaovic et al.[10] reported about their 5-day mini-residency program that was held at the University of California to help urologists master challenging laparoscopic skills. The mini-residency program consisted of (1) ureteroscopy/percutaneous renal access, (2) laparoscopic ablative renal surgery, (3) laparoscopic reconstructive renal surgery, and (4) robot-assisted prostatectomy. To test whether basic laparoscopic skills were improved by a 1-week laparoscopy training program, they evaluated 101 participants on their laparoscopic skills acquisition at the end of the program. Laparoscopic and robotic skill testing scores, but not open scores, improved significantly during their 5-day mini-residency program.

The limitation of this study is the small number of the participants and the lack of a control group. This is because of the course concept. Because there were 30 trainees, 23 of them volunteered to take part in the study. Given the small number of observations and the non–normally distributed data, nonparametric statistical methods were used for all analyses and the results were made according to these results. Because the video sessions are based on theoretical instructions and the observation of laparoscopic surgeries, we considered in our study design the first group of the trainees. Trainees who started with the video session and had their first laparoscopic practice with live animals were included into the wet-laboratory group.

It has been recognized that because of increasing time constraints, cost, stress, and ethical considerations, the modern operating room is not the ideal learning environment. The lack of training procedures needs to be overcome by structured training programs, and the experience gained needs to be transferred into daily clinical practice. The applied short-term laparoscopy course has proven to be an effective format as a part of this education. To this end, the applied short-term laparoscopy course is essential in our country for all urology residency programs.

CONCLUSION

Our study showed that the applied laparoscopic urology course is an effective format, especially for acquiring laparoscopic skills in which suturing and knotting are essential. The trainees have benefited from the course and showed great improvements in their basic laparoscopic skills. The different schedules of the training program do not affect the end results regarding the laparoscopic skills of the trainees. The predictive validity of the applied laparoscopic urology course is still unknown. How these advanced skills are reflected in the operating room remains to be proven.

Table 1.

Comparison of Laparoscopic Skills Scores and Task Accomplishment Times of Trainees at End of Course and According to Training Groups Beginning in Wet Versus Dry Laboratory

	Group I		Group II		Z	P Value
	Mean	SD	Mean	SD
Initial task score	18.88	6.34	19.14	6.26	–0.034	.974
Final task score	26.00	4.80	26.86	2.54	–0.106	.922
P value for initial vs final task score	<.001		.017
Initial task accomplishment time	9.50	3.43	8.00	3.42	–1.007	.341
Final task accomplishment time	5.25	2.02	5.29	1.70	–0.238	.820
P value for initial vs final task accomplishment time	.002		.027