Retention of basic laparoscopic skills after a structured training program.

The purpose of this study was to test the retention of basic laparoscopic skills on a box trainer 1 year after a short training program. For a prior study, eight medical students without prior experience (novices) underwent baseline testing, followed by five weekly training sessions and a final test. During each of seven sessions, they performed five tasks on an inanimate box trainer. Scores were calculated by adding up the time to completion of the task with penalty points, consequently rewarding speed and precision. The sum score was the sum of the five scores. One year later, seven of them underwent retention testing for the current study. The final test results were compared with retention test results as a measure of durability of acquired skills. Novices' scores did not worsen significantly for four out of five tasks (i.e., placing a pipe cleaner p = 0.46, placing beads p = 0.24, cutting a circle p = 0.31, and knot tying p = 0.13). However, deterioration was observed in the performance on stretching a rubber band (p < 0.05), as well as in the sum score (p < 0.05). Nevertheless, all retention scores remained better than the baseline results. In conclusion, basic laparoscopic skills acquired during a short training program merely sustain over time. However, ongoing practice is advisable, especially to preserve tissue-handling skills, since these may be the first to deteriorate.

Introduction

Laparoscopic surgery requires skills that are different from those required for open surgery. Simulators were developed to train these skills in a pressure-free environment with or without supervision [1]. They can roughly be divided into box trainers (also video trainers) and virtual reality trainers. The interest in training facilities outside the operating room (OR) was further enhanced by issues like quality control, patient safety, and cost-effectiveness [1, 2]. Simulator training is shown to be effective in providing skills that are transferable to the OR [3-7] and to decrease procedural complications [8, 9]. Besides, simulators have the potential for objectively assessing laparoscopic skills [10].

In the Leiden University Medical Centre (LUMC), we developed a skills laboratory with an inanimate five-task laparoscopic simulation model (box trainer) for basic training and evaluation. In a recent study, our group has established the construct validity of these five tasks (i.e., the ability to discriminate between different skills levels) [11]. Additionally, the median score of five laparoscopic experts was set as performance standard for training and evaluation purposes [11]. This standard was based on one trial for all exercises. Finally, it was found that novices’ skills improved quantifiably and met the performance standard within seven trials (after a 5-week training course) [11].

In order to establish an efficient laparoscopic training program, the retention of the acquired skills is highly important. Additionally, insight into retention is important to judge the necessity and frequency of continued training needed to maintain the acquired skills level.

As yet, a couple of studies have described skill retention after simulator-based laparoscopic training [12-16]. The retention test results varied considerably, as well as the study designs. On one hand, a 25% skills deterioration was observed 3 months after a 1-day hands-on training course on a box trainer [16], and on the other hand, excellent skills retention was revealed 6 months after validated proficiency-based training sessions [15].

The objective of the current study is to investigate the retention of skills 1 year after the start of our laparoscopic training program and to enhance the insight into the retention process.

Materials and methods

The study was performed in the skills laboratory located in the Department of Gynecology at the LUMC in The Netherlands. The simulator was designed (F.W.J.) and fabricated at this tertiary teaching hospital. It consisted of an inanimate five-task box trainer with a nontransparent cover, measuring 45 × 30 × 25 cm using a 0° scope.

Outcome measures

The individual’s performance on the box trainer was measured using a scoring system that rewarded precision and speed. During each task, the time to completion (seconds) and penalty points were measured. Scores were calculated by the addition of completion time and penalty points, thus rewarding both speed and precision (score = time + penalty points). Consequently, faster and more accurate performance was rewarded with a lower score. Additional to separately scoring each task, a sum score was calculated (the sum of scores of all five tasks).

Tasks

The tasks in this study, as well as the scoring system, were based on the studies of Derossis et al. [17] and are shown in Fig. 1. The tasks vary from simple placing object tasks to more complicated maneuvers such as cutting and knot tying.

Fig. 1

Laparoscopic training tasks

Pipe cleaner

This task involved the placement of a pipe cleaner though four small rings. A penalty was calculated when a ring was missed. Score = time in seconds + (the number of missed rings × 10).

Placing rubber band

This task required the participant to stretch a rubber band around 16 nails on a wooden board. A penalty was calculated when the rubber band was not stretched around a nail at the end of the task. Score = time in seconds + (the number of missed nails × 10).

Placing beads

This task involved the individual’s placing 13 beads to form a letter “B”. A penalty was calculated when a bead was dropped next to the pegboard. Score = time in seconds + (the number of dropped beads × 10).

Cutting circle

This task required the participant to cut a circle from a rubber glove stretched over 16 nails in a wooden board. Penalty points were calculated when the individual deviated from cutting on the line. Score = time in seconds + surface of glove in milligrams deviated from circle.

Intracorporeal knot tying

This task involved the tying of an intracorporeal knot (two turn, square knots) in a foam uterus. A penalty was calculated to reflect the security (slipping or too loose) of the knot. Score = time in seconds + 10 when knot was slipping or loose.

Participants and measurements

The same eight medical students (novices) who had volunteered to participate and had been trained in our previous study [11] were asked to participate in the current study for a retention of skills test 1 year after the start of the training program. At the time of training, they were in the second to fourth years of their medical study at the LUMC and had no prior experience with simulator training or clinical laparoscopy. A precondition for current participation was that they had not further been training or practicing their laparoscopic skills during the consecutive 1-year period.

As described previously [11], the novices had underwent baseline testing on the simulator followed by five weekly training sessions and had been measured again the week afterwards for final testing, as shown in Fig. 2. Novices performed all five tasks once during baseline testing, the training sessions, and the final testing. Consequently, the novices had completed all tasks a total of seven times by the end of the study (one baseline test, five training sessions, one final test). Therefore, it will be referred to as seven trials. One year after the start of the training, they were asked to volunteer to perform the five tasks once more (retention test) for the current study.

Fig. 2

Performance during acquisition and retention

The primary outcome measure for the durability of the acquired skills was the comparison of the novices’ retention test scores to their final test scores. Secondarily, the retention test scores were compared with the baseline test, and retention test scores were also compared with the performance standard. This standard was established by the median scores of five “expert” gynecologists (having performed more than 100 advanced laparoscopic procedures) obtained from one single performance on the five simulator tasks once [11]. The flowchart considering participants is shown in Fig. 3.

Fig. 3

Flowchart of the study

Statistical analyses

Collected data were analyzed by SPSS 16.0 software package (SPSS, Chicago, IL, USA). Statistical analyses were performed using Mann–Whitney and Wilcoxon signed-rank test. Probability below 0.05 was considered statistically significant.

Results

Among the eight students who participated in our initial study, one was unable to participate in the current study due to absence during the retention test. As a result, the performances of seven students who participated in our initial 7-week training program were measured on the box trainer for assessment of retention of basic laparoscopic skills. The participants were considered novices since they had no surgical, laparoscopic, or simulator experience prior to the training program. Novices’ demographics are outlined in Table 1. During follow-up until the retention test, none of them had additional surgical, laparoscopic, or simulator experience.

Table 1

Novices’ demographics at retention test

	Novices (n = 7)
Mean age (range) in years	22.7 (21–24)
Male, n (%)	2 (29)
Median year of study	5 (3–5)
Laparoscopic experience	None (except from box training in preceding study)

Table 2 presents the median scores of the seven novices on the baseline test, the final test, and the retention test. It has to be emphasized that a better performance is represented by a lower score. Table 3 compares the novices’ median retention test scores with the median experts’ scores (performance standard, set as the training goal).

Table 2

Novices’ median scores

Table 3

Novices compared to performance standard

	Retention test (novice n = 7)		Performance standard (expert n = 5)		Difference
Task	Median	Range	Median	Range	P valuea
Pipe cleaner	42	28–63	62	49–100	0.06	N.S.
Rubber band	60	53–99	62	35–195	0.94	N.S.
Beads	283	159–417	271	111–318	0.29	N.S.
Cutting Circle	123	87–525	189	76–240	1.00	N.S.
Knot tying	182	60–343	118	50–177	0.18	N.S.
Sum score	800	515–1,219	705	351–878	0.34	N.S.

Score = time + penalty points. A lower score represents a better performance

N.S. nonsignificant

aMann–Whitney test

Durability of acquired skills as primary outcome measure

The retention test score did not worsen significantly compared with the final test score for four out of five tasks: pipe cleaner, placing beads, cutting a circle, and knot tying (Wilcoxon signed-rank test). However, deterioration was observed in the score for stretching a rubber band (p < 0.05) as well as the composed sum score (p < 0.05) (Table 2).

Retentions test results compared with baseline testing and with the performance standard

Comparison of the retention test with the baseline test reveals a significant improvement on all five tasks as well as the sum score (Wilcoxon signed-rank test: pipe cleaner, rubber band, placing beads, cutting a circle, knot tying, and sum score all p < 0.02) These differences are not displayed in a tabular form. This means that novices perform all tasks faster and more accurately at retention than at baseline testing.

No statistical differences were found in any task or the sum score between the novices’ retention test and performance standard (Mann–Whitney test; Table 3).

Discussion

The previously quantified improvement in laparoscopic skills remained at the same level for four out of five laparoscopic box trainer tasks 1 year after a basic laparoscopic skills training program. This long-lasting retention of skills is encouraging and supports the implementation of laparoscopic simulator training program at the beginning of residency.

However, deterioration in performance was observed in the task to stretch a rubber band around 16 nails. This finding is remarkable because, in that particular task, haptic feedback and force transmission play an important role. The ability to adapt to diminished haptic feedback, considered as a substitute for tissue handling, is one of the difficulties of laparoscopic surgery. The current finding may be interpreted as an argument that tissue-handling skills are the first to deteriorate in the absence of any practice.

Additionally, the sum score at the retention measurement was worse than the score immediately after the training. It is well possible that the performance in all five tasks slightly decays with time, but that significance was not revealed due to the small number of participants (n = 7) who completed the study. Though, this small decay trend is significant for the composite sum score.

Factors considered to be accountable for the longevity of the acquired skills are the retention interval, the quality of the original training, and trainees’ individual differences [15, 18]. Our 11 months retention interval is the longest studied so far, as others studied intervals varying between 3 weeks and 7 months. The quality of training is influenced by the type of trainer used for skills acquisition and the duration, intensity, and goals of the training course. In some studies, box trainers were used; in others, virtual reality trainers. Stefanidis and colleagues compared both devices and found better skill retention for box trainers [14]. In that study, the bean drop task used as in the box trainer mainly requires eye–hand coordination and can be compared to our beads placing task. In general, practice interspersed with periods of rest (distributed practice) leads to better acquisition and retention of endoscopic skills than continuous practice (massed practice) with little or no rest in between [19, 20]. Additionally, goal-oriented training leads to consistency of the final results, since all residents are expected to reach the performance standard [11]. In summary, these data are supportive for the quality of our basic laparoscopic skills course, since we used box trainers, training was held in distributed sessions (1 h weekly for 7 weeks), and an expert’s performance standard was set as training goal.

Two striking differences were revealed between our findings and other study results. First, a recent study with a virtual reality trainer revealed that the skills required to perform more difficult tasks deteriorated more than skills needed for the easier tasks 6 months after training [13]. That finding contradicts our finding that the more complex knot tying and cutting skills did not decay significantly, while the placing rubber band task did. Maybe the (relative) resistance to decay of our complex tasks can be explained by a better durability of skills acquired due to the quality of our training, or the small number (seven) of participants failed to show significant deterioration. Second, the study of Vossen and colleagues revealed that most training effect was achieved after 20–30 square knots in a box trainer [21]. This finding contrasts with the small number (seven) of trials needed in our study to achieve the performance standard. However, it has to be noted that the experts in our study only performed one trial of each task for the establishment of this standard. They would probably have shown better performances after they have familiarized with the box trainer and the tasks. The resulting “lower expert level” might be marked as a shortcoming of the study, and the value of the result that novices still met the performance standard 1 year after training may be doubted. On the other hand, this standard revealed to lead to achievable learning goals for skills improvement that sustain over time.

It is of significance to gain insight into the retention of skills in order to realize optimal frequency and efficiency of laparoscopic simulator training. Especially, this is important since students and residents may have a long interval in rotations or residency training before returning to a department in which they can train their laparoscopic skills. Individual differences in retention—and in innate dexterity—among trainees stress the importance of reassessment. Not all subjects may be able to maintain the acquired skill and some require extra training in addition to the training program. To identify these subjects, objective skills assessment on the simulator should be performed regularly.

Previously, we have shown that a voluntary training program on a box trainer during residency has a substantial risk to fail [22]. Therefore, a goal-orientated structured training needs to be implemented into practice in a mandatory fashion, preferably early in residency [22]. Specifically, 1 h of box trainer training a week fits more easily into an already busy residency training schedules than a less efficient training course compressed into 2 or 3 days. In order to maintain the acquired skills optimally, it is our opinion that simulator assessment (eventually followed by training) should be repeated at least annually.

In conclusion, our short training program on the box trainer is shown to result in measurable skills improvement that is and merely durable over time. In order to maintain tissue-handling skills and to reassure that the skills level for each individual maintains at the performance standard, continuous hands-on practice has to be facilitated and promoted. In order to reach optimal benefit, we recommend the implementation of laparoscopic simulator training program at the beginning of residency training and biannual or annual simulator training and reassessment.