Feedback in laparoscopic skills acquisition: an observational study during a basic skills training course.

This study aimed to obtain insight in the effect of expert feedback during a basic laparoscopic skills training course for residents. A questionnaire was held among participants regarding provided feedback and the self-perceived laparoscopic skills improvement. The participants (n = 24) who completed the questionnaire were in their first to fifth postgraduate year. Most feedback was directed at intracorporeal knot tying (47% reported extensive feedback), while camera navigation and body positioning received the least feedback (40% and 43%, respectively, responded to have received no feedback at all). After the course, the self-perceived competence in intracorporeal knot tying and cutting had improved significantly, while camera navigation, body positioning, pointing, and grasping tasks did not improve. In conclusion, most benefit from expert feedback can be obtained at the start of the learning curve. Therefore, the basic laparoscopic skills course should be attended early in residency. Additionally, it is crucial that training objectives are clear prior to a course for both the expert and the trainee, in order to focus the feedback on all training objectives.

Introduction

Nowadays, several training devices and methods are available for laparoscopic skills acquisition. Most studies about laparoscopic skills acquisition are focused on the validation of simulators, i.e., devices that recreate operative conditions. However, other important issues concerning the learning process, like feedback, are seldom discussed. This often leads to the absence of validated feedback during training [1].

In the historical apprenticeship system of “see one, do one, teach one”, feedback was directly provided during surgery in the operating room (OR). Due to ethical and financial constraints and because of limitation of resident work hours, simulators were introduced in the last decade [2]. This enables repeated skills training in a stress-free environment without causing harm to patients.

Simulators can roughly be subdivided into video box trainers and computerized virtual reality (VR) trainers. Box trainers need the assistance of expert tutors during training sessions and courses to provide feedback. Experts can provide feedback on basic laparoscopic skills (BLS) or can guide the trainee step by step through difficult tasks, e.g., intracorporeal knot tying, while giving tips and tricks. They can also increase motivation, which enhances the learning process [3], though objective assessment of the laparoscopic skills is not directly present in a traditional box trainer [1]. On the other hand, VR trainers are programmed to give objective feedback about time, errors, and movement parameters. However, the value of some of these movement parameters remains disputable [4]. Furthermore, there the feedback is limited to what has been programmed. This implicates that several aspects of BLS, for example the position of the hands and arms, will not be learned without the presence of a tutor.

Expert feedback seems to be indispensable in the acquisition of motor skills. Videotape mentoring by an expert even improves the laparoscopic skills of experienced urologists [5]. However, due to scarce time and relative high costs, the expert should be exploited optimally. Therefore, we conducted an observational study of the role of expert feedback in laparoscopic skills acquisition during a mandatory course in residency training.

Materials and methods

In 2007, an observational study was performed during a basic laparoscopic skills course in The Netherlands. This course, named the “Cobra-alpha”, is a basic laparoscopic skills course, which is mandatory for Dutch Obstetrics and Gynecology (Ob/Gyn) residents since 1997, and should be attended in the first or second postgraduate year (PGY) [6]. The goal of this course is to train the participants in BLS and provide knowledge necessary for continued training. In summary, the BLS include camera navigation, body positioning, pointing and grasping tasks, and the use of the scissors. Although it is an advanced task, laparoscopic knot tying is also trained as it incorporates all other BLS [7].

Outline of the course

For the laparoscopic skills training, five validated laparoscopic box trainer exercises were used [7]. These exercises were placing a pipe cleaner through four small circles, placing beads in the form of a B on a board, stretching out a rubber band around 16 nails on a wooden board, cutting a circle from a rubber glove and intracorporeal knot tying. The level of performance on each task was established by adding the time to completion of a task with penalty points, consequently rewarding precision and speed. Additionally, the resulting score could be compared to a pre-established expert level, used as performance standard [7]. Three hours of hands-on training started after a 10-min introduction video in which the exercises were explained. For each two participants, one box trainer was available. Consequently, one resident could hold the camera, while the other performed the exercises. Approximately, half an hour was reserved for each participant per exercise. Six experts in minimally invasive gynecology supervised the training as tutors. Although not specifically trained to teach basic laparoscopic skills, they work as consultants in teaching hospitals and have at least 5 years of experience in these skills courses. They had been instructed to guide the participants through the exercises and to provide feedback and instructions on the BLS.

Measurements

A questionnaire was developed by the authors on the role of expert feedback in laparoscopic skills acquisition. The questionnaire consisted of a part that was handed out prior to and a part immediately after the course. In the first part, residents’ characteristics were collected, as well as their self-perceived level of competence on their BLS. The second part concerned the expert feedback and instructions; additionally, the self-perceived level of competence was asked again.

As primary outcome measure, the expert feedback was studied. Participants were asked to which extent they had perceived feedback on 13 items (1 = no feedback, 2 = little feedback, or 3 = extensive feedback). For analysis, these items were categorized in five topics: body position, camera navigation, use of instruments, laparoscopic knot tying, and other tips and tricks. Furthermore, participants were asked what feedback they judged to be helpful in acquiring BLS and on which topic they had desired more feedback (multiple options were possible). Finally, ideas or comment on the course could be noted.

Secondarily, the self-perceived competence was determined before and after the course in order to obtain a measure of a participant’s learning effect. The self-perceived level of competence had to be rated on a Visual Analog Scale (VAS). The VAS ranked from unskilled to very skilled and was interpreted in the range 1 to 5. For data analysis, a VAS of 3 was chosen as cutoff point, and a score of 3 or higher was considered to be skilled.

Statistics

Statistical analyses were carried out using SPSS 14.0 (SPSS Inc., Chicago, IL, USA). Percentage of perceived feedback was calculated. The self-perceived skills levels prior to and after the course were compared, using Wilcoxon signed-rank test. The correlation coefficient between resident’s experience (expressed in number of postgraduate years) and self-perceived skill level prior to the course was analyzed by a nonparametric Spearman’s rank correlation test. A p value below 0.05 was considered statistically significant.

Results

Out of the 28 participants who attended the course, 24 (86%) completed the entire questionnaire. The remaining four participants had incompletely filled out the questionnaire and were excluded from further analysis. Residents ranged between their first and fifth PGY (median: PGY 3). Among the participants, 83% reported prior experience on a box trainer or VR trainer. A minority of 29% had practiced laparoscopic knot tying before.

The highest amount of experienced feedback was focused on laparoscopic knot tying and in some lesser extent on instrument handling (Table 1). Additionally, participants appraised feedback on these two topics as most valuable. Less than 30% of the participants indicated to have received extensive feedback on the other topics. Least attention was paid to body positioning; 43% claimed to have received no feedback on that topic. In general, 66% of the participants thought that more feedback would have resulted in more improvement of their skills, except from feedback on camera navigation.

Table 1

Amount of experienced feedback

Topic	Feedback (%)
	No	Little	Extensive
Body positioning	43	36	21
Camera navigation	40	32	28
Instrument handling	31	39	31
Intracorporeal knot tying	11	42	47
Other tips and skills	46	25	29

Prior to the course, the median self-perceived skills assessment varied between 1.2 and 3.4 (Table 2). Participants already rated themselves to be skilled in camera navigation and pointing and grasping tasks prior to the course. Residents who were in an earlier phase of residency, expressed in smaller number of PGY, rated their skillfulness prior to the course lower than the ones who were more experienced (Spearman’s rank correlation test, ρ = 0.19, p = 0.04). In general, the self-perceived competence improved significant for laparoscopic knot tying and the use of the scissors (Wilcoxon signed-rank test, p < 0.001). No significant improvement was seen for the other BLS.

Table 2

Self-perceived skill improvement

Skill	Median VAS score		Difference before and aftera
	Before	After	p value
Body positioning	2.8	2.9	0.67 N.S.
Camera navigation	3.4	3.2	0.18 N.S
Pointing en grasping	3.0	3.1	0.13 N.S
Use of the scissors	2.7	3.1	<0.001
Intracorporeal knot tying	1.2	2.4	<0.001

VAS: self-perceived level of competence, range 1 (unskilled) to 5 (very skilled)

VAS Visual Analog Scale, N.S not significant

aWilcoxon’s signed-rank test

The improvement in relationship to the skill level prior to the course is shown in Fig. 1. This figure shows that participants, who rated themselves not as skilled, improved more than the already skilled participants, except for body positioning. This is best shown in the camera navigation. Prior to the course, 16 residents scored over 3.0. Overall, no improvement was made. However, six of the eight residents that scored less than 3.0 prior to the course showed improvement (Wilcoxon signed-rank test, p = 0.06).

Fig. 1

Skills improvement during a basic laparoscopic skills course. X-axis represents VAS score of participant’s self-perceived skills level prior to the course; Y-axis represents the skills improvement (VAS score after minus VAS score before the course), n = 24 participants. VAS Visual Analog Scale, in which 1 = very unskilled, 5 = very skilled

Discussion

This study revealed that residents judge feedback as a factor that positively influences their skills acquisition during a basic laparoscopic skills course. Most feedback seems to have been directed toward intracorporeal knot tying. This skill also showed the greatest improvement. Alternatively, body positioning seems to have got the least feedback, and this skill did not show improvement. Admittedly, the self-perceived competence is not the most objective and accurate measure for assessment of skills level. However, in one study, residents showed to be capable to rate their own laparoscopic skills with good reliability and validity [8]. Of course, no firm conclusions can be drawn from this observational study. However, a possible relation between feedback and learning can be hypothesized. This fits the results of the study of Mahmood et al., who showed that without feedback there can be no learning [9]. Albeit, the objectives of a course need to be predefined in order to facilitate expert feedback on all training goals.

Though it is worrisome that in the presence of six tutors, who were informed about the learning objectives, more than 70% of the participants denied to have received extensive feedback on some basic skills (e.g., camera navigation and body positioning), a possible explanation is that the more complex tasks distract the experts’ attention from the more basic tasks. Furthermore, the experts in the current study might be more eager to teach knot tying and cutting because they are more used to practicing and training these skills in clinical practice. This information may provide new leads for improving of the course. In the first place, tutors have to ascertain that their feedback has actually reached the trainee. Secondly, it may be helpful to make the learning objectives clear to the trainees as well, to enable them to ask for more specific feedback on their skills.

The improvement of self-perceived competence due to the BLS course was the highest for complex tasks, i.e., the use of scissors and intracorporeal knot tying, while residents indicated no or little improvement in the more basic skills. That reciprocates the finding that participants rated themselves better skilled in the basic than in the more complex skills. This phenomenon can be explained by the influence of feedback on a learning curve. In the beginning, the learning curve is steep and trainees experience most benefit from cognitive feedback [10]. The cognitive feedback, which explains how and why something is performed as it should, can best be provided by experts. The positive effect that experts can generate in the start of the acquiring of motor skills was already revealed by Rogers et al. [11]. This group showed that novices have a significantly better performance on a knot tying task if expert feedback is added to computer-assisted learning.

Particularly, in the beginning of the process of acquiring of new skills, training courses in presence of an expert are useful. Unfortunately, the residents who attended this course were not all in the beginning of residency. Most participants had already trained laparoscopic skills in dry laboratories or in the OR during residency. Because participants obtain most benefit from the BLS course in the beginning of their learning curve, residents should attend the training course early in residency.

After progressing along a learning curve, improvement becomes slower resulting in the need for another type of feedback. Skills have to develop into automatisms. Consequently, trainees are able to pay more attention to new elements, for example the anatomical structures during an operation [12]. During this stage, learning relies more on intrinsic feedback, provided by sense organs and previous cognitive knowledge [3, 10]. Additionally, trainees have to become independent from external feedback [13]. In this learning stage, feedback in the form of knowledge of results is important, instead of cognitive feedback [14]. Therefore, little skill improvement is expected during a BLS course later in residency.

It can be argued whether basic skills like camera navigation, instrument handling, and body positioning require the same attention as more advanced skills like cutting and knot tying. In the aforementioned theory, every skill should be taught in an equivalent way, keeping the competence level of a trainee in mind to tailor the training to individual needs. Therefore, an expert tutor with good didactic skills is highly important. Additionally, an expert can draw parallels to OR practice and, last but not least, strongly enhances the motivation of a trainee [3].

Further research needs to be done on the effect of expert feedback, in which more objective outcome measures might be used, though, with current knowledge, we can conclude that feedback plays an important role in skills acquisition and therefore the effectiveness of a training course. Training objectives need to be predefined and to be clear to both tutor and trainee, in order to guide and structure the feedback. In a BLS course, most benefit can be obtained at the start of the learning curve. Therefore, residents have to attend the BLS course early in residency.