Training on an inexpensive tablet-based device is equally effective as on a standard laparoscopic box trainer: A randomized controlled trial.

BACKGROUND: The aim of the study was to assess whether an inexpensive tablet-based box trainer (TBT) is at least equally effective compared with a standard box trainer (SBT) to learn basic laparoscopic skills (BLS). BLS training outside the operating room has been shown to be beneficial for surgical residency. However, simulation trainers are expensive and are not consistently available in all training centers. Therefore, TBT and other homemade box trainers were developed.
METHODS: Medical students were randomized to either a TBT or an SBT and trained 4 fundamentals of laparoscopic surgery (FLS) tasks for 1 hour twice a week for 4 weeks. A baseline test before the training period and a posttraining test were performed. All students then completed a questionnaire to assess their assigned box trainer. The primary outcome measure was the improvement in total test scores. Improvement in the scores for the 4 individual FLS tasks was chosen as a secondary outcome measure.
RESULTS: Thirty-two medical students were recruited. Baseline test scores did not differ significantly between the groups. BLS improved significantly in both groups for the total score and for all 4 tasks separately. Participants in the TBT group showed a greater improvement of total scores than those in the SBT group, although this did not reach statistical significance; noninferiority of the TBT compared with the SBT concerning the improvement of total scores could be demonstrated. Regarding the individual FLS tasks, noninferiority of the TBT could be shown for the pattern cutting and the suturing with intracorporeal knot-tying task. The acceptance of the TBT by the trainees was very good.
CONCLUSION: Learning BLS on a homemade TBT is at least equally effective as on an SBT, with the advantage of being very cost saving. Therefore, this readily available box trainer may be used as an effective, flexible training device outside the operating room to improve accessibility to simulation training.

RCT Entities:   Population Interventions Outcomes

Introduction

Laparoscopic surgery has become the gold standard for many surgical procedures.[ In comparison with open surgery, laparoscopic surgery requires a specific psychomotor skill set that may be more difficult to acquire, resulting in a prolonged learning curve.[ Due to residents spending less time in the operating room (OR), mainly as a result of work hour restrictions, difficulties have arisen in gaining enough surgical experience during training to reliably acquire sufficient laparoscopic skills.[ In order to overcome this problem, training of basic laparoscopic skills (BLS) outside the OR has gained increasing importance, and various simulation models have been developed to meet this need.[

Numerous studies have shown that skills acquired in the simulation-environment transfer to the OR, and that simulation training programs significantly decrease the clinical learning curve.[ Although contemporary simulation training cannot replace clinical experience entirely, it allows trainees to make mistakes while practicing on a training model without compromising patient safety. Furthermore, it contributes to shorter operative times and reduces the risk of intra- and postoperative complications when trainees transition to the real OR.[

Although the importance of simulation training is widely recognized and surgical bodies emphasize that training in a simulated environment should constitute an integral part of surgical residency,[ surgical skills laboratories are not consistently available across all training centers. For this reason, many different types of low-cost homemade trainers have been developed.[ A particularly promising version consists of a cardboard box with an iPad serving as the camera and display[ (Fig. 1). Nowadays, tablets are owned by a considerable part of health care providers, especially residents.[ Therefore, nothing else but a cardboard box is necessary to construct this box trainer, which is particularly convincing since it is cheap and readily available everywhere in the world.

Figure 1

Tablet-based cardboard box trainer.

The aim of the present study was to investigate whether practicing on a homemade, inexpensive, tablet-based box trainer (TBT) is at least equally effective compared with the use of a traditional box trainer in improving BLS.

Materials and methods

Study design

This randomized controlled trial (ClinicalTrials.gov ID: NCT02491710) was conducted at the Surgical Skills Training Center in the Department of Obstetrics and Gynecology at the Medical University of Vienna.[ The study started August 2015 and ended September 2015. Participants were randomized to 1 of the following 2 groups: training on a homemade tablet-based cardboard box trainer (TBT) or training on a standard box trainer (SBT).

Participants

Medical students were recruited from the Medical University of Vienna. Medical students with limited experience in laparoscopic surgery were eligible. Students were excluded if they had performed any laparoscopic operation as the primary surgeon or had regular (e.g., once per month) practice on a box trainer for the last 12 months.

Written informed consent was obtained from all participants prior to the start of the study. A unique study identification number was assigned to each participant and baseline demographic data were collected using a paper questionnaire at the time of consent. Approval for this study was obtained from the Ethics Committee of the Medical University of Vienna (IRB approval number: 1322/2015).

Interventions

Training sessions took place twice a week for 4 weeks with each training session lasting a maximum of 1 hour. Participants performed the following 4 fundamentals of laparoscopic surgery (FLS) tasks: peg transfer, pattern cutting, suturing with extracorporeal knot tying, and suturing with intracorporeal knot tying.[ At the beginning of the study, prior to the first training session, an instruction video demonstrating the 4 FLS tasks was shown to all participants, followed by a baseline test for each of the 4 tasks. During the training sessions, surgical instructors were present to provide feedback and supervise the participants. Participants were asked to complete at least 10 repetitions of the peg transfer task and the pattern cutting task, whereas for both suturing tasks participants were instructed to practice until a self-perceived improvement was noted, with a maximum overall time limit of 1 hour for each training session. Every training session consisted of 2 of the 4 tasks, of which one was a suturing task. At the end of the study, each participant performed a posttraining test, identical to the baseline test.

Training on an inexpensive TBT

During the study period, participants randomized to the TBT group performed the 4 FLS tasks on a box trainer consisting of an Apple iPad 6 (Apple Inc., Cupertino, CA) placed over a hole cut into the upper part of a cardboard box (Fig. 1), which was constructed as previously described.[ The iPad was used on the video recording function with the auto-snooze and auto-focus functions turned off during the exercises.

Training on an SBT

Participants randomized to the SBT group practiced on the LaproTrain box trainer (Endosim, Belfast, Northern Ireland) consisting of a box with a self-holding camera connected to a separate color video monitor (Grundig, Nürnberg, Germany) (Fig. 2).

Figure 2

Standard box trainer.

Outcome measures

Baseline and posttraining assessment

The baseline and the posttraining assessment were completed by each participant performing each of the 4 FLS tasks once. A WISAP Simulation Trainer (WISAP Medical Technology, Brunnthal, Germany) (Fig. 3) was used together with a camera head (Karl Storz, Tuttlingen, Germany), a 300 W Xenon light source (Karl Storz), a 10 mm 0° laparoscope (Karl Storz) and a color video monitor (Sony, Tokyo, Japan). For both the baseline and the posttraining test, participants were allowed to become familiar with this type of box trainer by performing half of the first task (peg transfer) once prior to the assessment.

Figure 3

Box trainer used for baseline and post training assessment.

Primary outcome measure

Performance of the 4 FLS tasks was assessed according to the time needed to complete the task as well as the accuracy of task performance using the FLS scoring system as previously described.[ A staff surgeon, proficient in laparoscopy and with significant experience in surgical training using the FLS tasks, carried out the assessment. The improvement of performance for the total score as well as for each FLS task was calculated by subtracting the baseline test scores from the posttraining test scores. The primary outcome measure was the improvement in total test scores.

Secondary outcome measures

In addition to the total test scores, improvement in the scores for the 4 individual FLS tasks was chosen as a secondary outcome measure. Furthermore, at the end of the study, participants were asked to complete a questionnaire (see Supplemental Content) regarding the quality and usefulness of the box trainer they had practiced on during the study. Lastly, after completion of the study participants performed each of the 4 FLS tasks on the other type of box trainer once (i.e., participants randomized to the SBT group used the TBT and vice versa), and were asked to indicate whether they would have preferred to practice on this type of box trainer instead of the box trainer they were randomized to.

Sample size calculation

Sample size calculation was performed for a noninferiority trial with a continuous outcome. We considered a noninferiority margin of 10% of the mean improvement in total scores after training of the control group (i.e., the SBT group) to be the minimum clinically significant difference and calculated the required sample size applying a single-tailed α of 0.025 and a power of 0.90. Calculation was based on the previous results for the FLS tasks of novice trainees showing a mean improvement in total scores of 342 points with a standard deviation (SD) of 24 points.[ These are the best available data given the absence of available SD of mean in other trials, or sufficient published data to calculate a correlation coefficient of absolute scores and SDs that could be used to impute the SD of change improvement. According to the considerations made above, 12 participants were required in each training arm. Anticipating a drop out rate of 20%, a minimum of 15 participants had to be included per study group.

Randomization and masking

For randomization, an allocation ratio to each treatment of 1:1 was used. Participants were randomized according to the concealed sequence of a computer-generated randomization plan by one of the research team members. Participants were consecutively randomized into 1 of the 2 groups by means of sealed envelope technique. No masking was used, because the FLS scoring system is standardized and objective.

Statistical analysis

Data was analyzed according to the intention-to-treat principle. Diagrams were used to assess normality of the distributions. For data not following a normal distribution, nonparametric tests were performed, using the Mann–Whitney U test for comparisons between groups and the Wilcoxon signed-rank test for comparisons within groups. For normally distributed data, Welch's t test for independent samples was used for comparisons between groups. Medians and interquartile ranges (IQRs) or means and SDs, respectively, are shown for continuous and ordinal data. Nominal data were analyzed by the Fisher exact test. Two-sided P < 0.05 were considered statistically significant. For the analysis of the primary outcome (i.e., improvement in total test scores) as well as of the improvement in the scores for the individual tasks, noninferiority was determined if the lower bound of the 2-sided 95% confidence interval (according to the Welch's t test) of the mean difference in improvements did not exceed the noninferiority margin (i.e., 10% of the mean improvement in the SBT group). IBM SPSS version 21.0 (IBM Corp., Armonk, NY) was used for statistical analysis.

Results

Thirty-two medical students were recruited and randomized to either the SBT (n = 16) or the TBT (n = 16) group. All participants completed the study and were included in the final analysis. Baseline demographic data of the participants are shown in Table 1. There were no significant differences between groups regarding age, year of medical school, sex, handedness, having assisted in laparoscopic operations, simulation training experience, video game experience, or playing a musical instrument.

Table 1

Baseline demographic data of SBT and TBT groups.

Primary outcome measure: improvement in total scores

There were no statistically significant differences in the baseline test scores between the SBT and the TBT group for the total scores (Table 2). Both in the SBT and in the TBT group, a significant improvement between baseline and posttraining total test scores was demonstrated (Table 2). Total score improvement was found to be greater in the TBT group than in the SBT group, although this difference did not reach statistical significance (Table 3). Noninferiority regarding the total score improvement in the TBT group could be shown (Table 3). For the posttraining total test scores, there was again a trend toward higher posttraining scores in the TBT group compared with the SBT group, although this did not reach statistical significance (Table 2).

Table 2

Comparison of the baseline and post training test scores for the total scores as well as for the different FLS tasks within and between groups.

Table 3

Comparison of the score improvements for the total scores and the different FLS tasks between groups.

Secondary outcome measure: improvement in the scores for the individual FLS tasks

There were no statistically significant differences in the baseline test scores between the SBT and the TBT group for the 4 individual FLS tasks (peg transfer, pattern cutting, suturing with extracorporeal knot tying, and suturing with intracorporeal knot tying) (Table 2). Both in the SBT and in the TBT group, a significant improvement between baseline test scores and posttraining test scores was demonstrated for all 4 FLS tasks (Table 2).

Noninferiority of the TBT compared with the SBT regarding score improvement was shown for the pattern cutting and the suturing with intracorporeal knot-tying task, whereas it could not be observed for the peg transfer and the suturing with extracorporeal knot-tying task (Table 3). The posttraining test scores in the TBT group were significantly higher than those in the SBT group for the suturing with intracorporeal knot-tying task (Table 2). There was no difference regarding post training test scores for the other 3 FLS tasks (Table 2).

Secondary outcome measure: comfort and satisfaction with the assigned box trainer

There were no statistically significant differences in the evaluation of visibility, posture comfort, effectiveness as a laparoscopic box trainer, and overall satisfaction with the assigned box trainer on a 10-point visual analog scale between the SBT and the TBT group. The lighting conditions were perceived to be better in the SBT group than in the TBT group (SBT group: median 9, IQR 8–9.75; TBT group: median 7, IQR 6–8.75; P = 0.015, Mann–Whitney U test), whereas the image sharpness was rated better in the TBT than in the SBT group (SBT group: median 6.5, IQR 4.5–7; TBT group: median 8.5, IQR 8–9; P = 0.005, Mann–Whitney U test). There was no statistically significant difference in the proportion of participants who would have preferred practicing on the type of box trainer of the other group instead of the box trainer they were assigned to (2/16 in the SBT group versus 6/16 in the TBT group, P = 0.22, Fisher exact test).

Discussion

Improvement of BLS

We performed a randomized controlled trial to assess whether training of BLS on a homemade TBT is at least equally effective compared with an SBT. After a training period of 4 weeks, a significant improvement of skills was observed in both study groups, irrespective of the type of box trainer that was used. Noninferiority of the TBT compared with the SBT could be shown for the improvement in total test scores, as well as for the improvement in the scores of the pattern cutting and the suturing with intracorporeal knot-tying task. For the suturing with intracorporeal knot-tying task, participants in the TBT group showed statistically significantly higher posttraining scores than those in the SBT group. No significant difference in posttraining test scores was observed between the 2 groups for the other 3 tasks studied. For all tasks including total scores, a trend toward a greater improvement in the TBT group than in the SBT group could be seen.

These results suggest that an inexpensive, homemade TBT can be considered to be at least equally effective compared with an SBT in terms of training of BLS, while having the great advantage of being readily available for everyone and very cost saving. The SBTs used in this study were purchased for 4890 € each (without monitor), while the homemade TBT, which can be built by simply “recycling” a cardboard box, was free of cost except for the iPad or tablet. Beyond the great cost savings, the TBT is also very space saving and offers the advantage of considerable flexibility concerning training location, as it is very light and has no need for an external light source or a monitor to be connected to. Furthermore, as opposed to the SBT, no power outlets or cables are necessary for the TBT, adding to the flexibility. Therefore, it represents an easily accessible, flexible, cost saving, and effective training alternative outside the OR.

Laparoscopic skills training outside the OR

It has been repeatedly demonstrated that the skills acquired by simulation training transfer to the OR and decrease operative times, intra- and postoperative complications.[ Despite the widely recognized importance of simulation training, adequate training opportunities are not consistently available across training centers. For example, only about one-third of the respondents of a survey among gynecology, urology, and general surgery residents in Belgium had the facilities to implement deliberate practice in simulation skills laboratories.[ Interestingly, even residents in programs with access to a skills laboratory often made limited use of this resource due to restrictive opening hours or inconvenient location of the training facilities.[ In the Netherlands, a survey found that 55% of all gynecological teaching hospitals did not offer simulation training at all,[ and a survey encompassing 32 different European countries reported that 42% of the residents did not have access to a simulation training facility.[

In order to overcome these difficulties, trainees may consider practicing on a self-made box trainer at home or in the hospital (e.g., on-call room). Homemade box trainers allow training without any constraints regarding availability, location or opening hours of surgical skills laboratories. To date different low-cost box trainers have been developed.[

Previous randomized controlled trials comparing low-cost box trainers to SBT reported similar results to those found in this study. However, the available trials had flaws regarding the study design and methodology, therefore limiting their reliability and generalizability. Some of these limitations include not making use of a neutral box trainer for the assessment of task performance,[ carrying out no baseline testing, having only short training periods or using a camera-less homemade trainer with direct insight into the box,[ with or without having 1 eye obscured to reduce stereoscopic vision. In some trials, a comparison between different types of box trainers was made solely based on subjective evaluation of the participants without objective evaluation of skill improvement after a training period.[ The present study addresses many of these issues by using subjective and objective assessments, a rigorous distributive training curriculum and pre- and postcurriculum testing using a neutral trainer.

In addition to methodological limitations of previous studies comparing different homemade box trainers, these simulators were reported to have structural disadvantages, which were improved upon in the TBT used in the current study. These disadvantages include the need of camera and monitor for visualization, which limits the portability of the box trainer, or the need of an external light source inside the box, which increases construction complexity and costs. Therefore, in this study we decided to use a tablet-based solution consisting of nothing else but a cardboard box and a commonly available tablet.[ With the cardboard box being very light and relatively small, this box trainer is both portable and space saving.

Quality, comfort, and satisfaction with the assigned box trainer

The homemade TBT was well-accepted by the participants of this study. The overall quality and comfort was equivalent for the 2 different types of box trainers. Lighting in the TBT was rated by the participants to be inferior to the SBT. As mentioned above, the TBT has no specific light source but depends entirely on indirect lighting. The study took place in a room without daylight and no additional light sources (e.g., a desk lamp) were used, which is the most likely reason for this finding. However, this problem can be overcome when the box trainer is used in rooms with daylight or if the interior of the box is directly illuminated by a desk lamp, which both greatly improve the lighting conditions while still promoting portability. Image sharpness was rated superior in the TBT compared with the SBT. This may be the reason why participants who trained on the TBT outperformed participants in the SBT group in the suturing with intracorporeal knot-tying task. High-definition cameras and displays of new-generation tablets offer excellent visualization, which greatly eases training of intracorporal knot tying.

Generalizability of the results

This trial involved medical students without significant laparoscopic experience. Therefore, the results may be generalized to trainees with no or little experience in laparoscopic surgery, which applies to both students and residents in their early stages of training.

Tasks and training design

FLS tasks were chosen for the study because an objective scoring system for the FLS tasks has been previously published, the results can be compared with existing literature and the validity of these tasks has already been thoroughly evaluated.[ We used the 2 easier tasks, peg transfer and pattern cutting, and the 2 more complex tasks, suturing with extracorporeal knot tying and suturing with intracorporeal knot tying, to evaluate different levels of task difficulty. The fifth FLS task, the ligating loop task, was recently found to have poor discriminatory ability and was therefore not included.[

Based on the knowledge from previous studies that demonstrated skill retention is superior using a distributed training model (i.e., practice interspersed with periods of rest) compared with mass training model (continuous practice with little or no rest in between practice sessions),[ the training sessions in this trial were limited to 1 hour each and offered twice a week for a 1 month time period.

In order to avoid a possible advantage of 1 group over the other at the time of the posttraining test due to habituation to a particular type of box trainer, both the baseline and the posttraining test were performed on a third, neutral box trainer.

Limitations

This trial has some limitations. First, the use of a tablet for visualization does not allow for training on camera navigation, which is an essential skill in laparoscopic surgery. However, in addition to the availability of simple and low-cost simulation methods,[ teaching camera navigation in the OR is less problematic, as it does not represent a direct threat to the patient.

Furthermore, it could be argued that tablets are expensive and not everyone has one. However, to over the last decade the prevalence of tablets in medicine has sharply increased. Nowadays, iPads or other tablets are owned by a considerable proportion of medical institutions and health care providers,[ especially residents. In addition, tablets are becoming more integrated into clinical practice and medical education, with a continuous development of new applications.[ Furthermore, low-cost tablets have become available in recent years, which will likely further augment this trend.

Conclusion

An inexpensive, homemade, tablet-based laparoscopic box trainer appears to be equally effective compared with an SBT in terms of BLS acquisition and is well accepted by trainees. The type of box trainer tested offers an effective, very cost saving and flexible opportunity for students and residents to train and improve their skills outside the OR without compromising patients’ safety. Considering the very low cost compared with a SBT, this concept represents a promising method to promote acquisition of technical skills outside the OR in surgical training programs with limited availability and access to simulation centers.

Acknowledgments

The authors thank Olympus for providing the needle holders and knot pushers used in this study.