Does sleep deprivation alter virtual reality-based robotic surgical skills?

INTRODUCTION: Robotic surgery is widely used in many surgical specialities, and there has been no study to assess the impact of sleep deprivation on the complex environment of robotic surgery. AIM: To compare specific metrics of selected robotic simulator exercises on sleep-deprived and non-sleep-deprived surgical residents.
MATERIAL AND METHODS: We enrolled 20 volunteers, residents in surgery, evaluated before and after an 18-hour overnight shift, regarding their results on virtual robotic surgery simulator - the sleep deprivation (SD) group. As a control group, the same subjects were evaluated 5-7 days after the post-shift evaluation, without having a shift overnight and at least 7 h of sleep the previous night - the non-sleep-deprivation (nSD) group.
RESULTS: A statistically significant difference between the pre-shift and post-shift overall results for all exercises in the SD group and no statistical differences for the nSD group were observed. As the difficulty of the exercises increased, statistical differences were observed on specific metrics for all exercises between the pre-shift and post-shift as well as between the post-shift and the morning after a normal sleep period overnight. In a subgroup analysis, the overall results revealed a stronger statistical difference between pre-shift and post-shift for residents with more intense sleep deprivation (< 3 h of sleep vs. > 3 h of sleep).
CONCLUSIONS: Sleep deprivation leads to impairment of surgical skills assessed by robotic virtual simulator. The more complex and skill demanding the exercise, the higher the difference between sleep deprived and non-deprived residents. Copyright:

Introduction

The issue of the impact of sleep deprivation on surgeons’ skills and the quality of their surgery is widely debated and controversially reported in the literature [1-4].

Medical activities requiring more than 80 h per week have been shown to be associated with significantly increased attentional failures during the subsequent night work hours [5]. A study published in 2014 revealed a 24% lower rate of detection of colonic adenoma during colonoscopy among experienced physicians who were an on-call the previous night [6]. Extended work hours, duration shifts, and higher frequency of shifts among residents were associated with an increased risk of medical errors, patient fatalities, and attentional failures [7]. In line with the trend to reduce the average number of working hours in industrialised countries, new regulations have emerged, limiting the number of consecutive hours on duty and the weekly work hours, differing from country to country. On the other hand, and especially for surgical specialties, the working-hours regulations have raised concerns related to the lack of time for residents to complete their curricula, to learn and to achieve satisfactory skills at the end of residency programs.

The amount that medical personnel work differs between disciplines, and surgical specialties seem to be particularly associated with an exhausting workload [8]. Surgeons are susceptible to sleep deprivation because their activity sometimes involves long surgical procedures during shifts, frequent shifts, or longer-than-usual consecutive working hours. The surgical procedures are cognitive and technical skills demanding activities and the effect of sleep deprivation could be easily seen during a surgical procedure. A sleep-deprived surgeon’s bad surgery could be more easily detected than that of an internal medicine physician who usually treats the patient within a team, thus the effect of a bad decision or treatment could not be detected so easily.

However, it is hard to objectively assess surgical dexterity because it involves many skills. It can be done by direct or indirect observation of a certain surgical procedure, by performing an experimental procedure on animals (both of them are subjective methods without clear, pre-defined parameters) or by virtual simulators, which are able to measure predefined parameters and compare them with averages and/or previous results. Virtual reality simulator exercises seem to be the best tool to objectively assess surgical skills [8, 9].

Within recent years, robotic surgery has widely expanded in many specialties, offering the advantages of a 3D view, precise and fine movement, intuitive motion, Endo WristTM instrumentation, and articulated instruments. In order to ensure proper training for console surgeons, high-quality, realistic robotic surgery virtual simulators were developed, facilitating an objective evaluation of proficiency and learning curve on specific surgical skills by exercises of different levels of difficulty (e.g. camera control, instrument manipulation, dissection, coagulation, suture, dissection, multiple instruments use).

Until now, all the studies objectively evaluating the impact of sleep deprivation on surgery were done on virtual laparoscopic simulators, in which the skills used are not fully superposable to those of robotic or open surgery.

Aim

The aim of this study was to evaluate whether, and how much, sleep deprivation influences the surgeon’s performance on robotic virtual reality simulator exercises as a good predictor of their robotic surgery quality, and to compare the results with those of studies performed on laparoscopic virtual simulators.

Material and methods

In this observational, cross-sectional study, we enrolled 20 volunteers, residents in surgery, who were evaluated before and after an 18-hour overnight shift, regarding their results on a virtual robotic surgery simulator using the da Vinci Xi platform (dV-Trainer, Mimic Technologies, Seattle, WA, USA) (sleep deprivation group – SD). All the enrolled residents had previous experience with the virtual reality robotic simulator (including the three exercises selected for this study) and had no experience as console surgeons. As a control group, to avoid biases and as an internal validation of the test, the same subjects were evaluated 5–7 days after the post-shift evaluation, without having a shift overnight and being asked to have had at least 7 h of sleep the night before (non-sleep-deprivation group – nSD). The residents included in the study did not enter more than once in each study group (once in the SD group and once in the nSD group), in order to avoid biases.

The study was conducted according to the national and hospital ethical requirements, and consent was obtained from each of the participants of the study after an explanation of the study objectives and methods. The study was carried out in accordance with the Good Clinical Practice Guidelines implemented in the World Medical Association Declaration of Helsinki.

One day before the evaluation, all participants completed the selected exercises on the simulator, in order to be familiarised with the tasks.

Three simulation exercises of increasing difficulty were selected from the simulation program, having being used previously to evaluate the influence of earlier open surgical experience in robotic surgery: Peg Board level 1 (the easiest task); Energy Dissection level 2 (intermediate task; and Suture Sponge level 3 – exercise 3 (the most complex), all of them being cited as the most reliable exercises to differentiate between beginners, intermediate level, and experienced robotic surgeons [10]. The simulator’s software calculated and displayed the individual results, which were recorded and analysed by the investigators.

The first exercise – Peg Board level 1 – requires the subjects, by using two needle drivers, to place highlighted rings onto highlighted pegs on the floor. The second exercise – Energy Dissection level 2 – requires the participants to cauterise small vessels at two points by using bipolar energy, and then to precisely cut between the points. The third exercise – Suture Sponge – level 3 – asks them to pass a curved needle through specific dots on a sponge, from different angles, using the backhand or forehand technique. For every exercise an overall score as well as metrics specific to a certain exercise were evaluated.

The participants were asked to complete the exercises before beginning the shift and after the end of the shift. Before starting the exercises, a five-minute warm-up time on the console was offered to every participant. At the end of the shift, before starting the exercises, participants were asked to disclose the number of hours they slept during the shift. Their personal data were blinded to the investigators and to the head of the department.

Because there are a paucity of data in the literature, an analysis of study power could not be done beforehand. However, the most similar studies included 20 participants and were considered to have been appropriately powered [8]. Null hypothesis (H0) was defined as: there is no difference between the simulator results before and after the shift (for the SD group), or there is no difference between the simulator results the morning before and the morning after a normal sleep period (for the nSD group). An alternative hypothesis (Ha) was defined as follows: there is a difference between the simulator result before and after the shift (for the SD group), or there is no difference between the simulator result the morning before and the morning after a normal sleep period (for the nSD group).

Statistical analysis

Data were displayed as mean values ± standard deviation. Statistical analysis was performed using the Kruskal-Wallis test. A p-value < 0.05 was considered significant.

Results

The group male/female ratio was 4/1 (16 males and 4 females). The mean age of the group was 29 years (range: 27–32 years), most of them (90%) being right-handed (18 out of 20).

The mean number of sleep hours during the overnight shift was 2.75, ranging from 1.5 to 4 (Table I). The overall results for the three exercises are presented in Table II.

Table I

The mean number of sleep hours during the overnight shift for every study participant

Subject no.	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20
Hours of sleep	1.5	3	3	3	2	3	1.5	1.5	2	3	2	3	2.5	4	3	4	4	3	2	4

Table II

The overall results for the three exercises (bold values for statistical significance, ns – not statistically significant)

Exercise	Sleep deprivation (SD) group Mean ± standard deviation		No sleep deprivation (nSD) group Mean ± standard deviation		Statistical significance
	Pre-shift	Post- shift	Morning	Next morning
Exercise 1 (Peg board – level 1)	82 ±21	79 ±23	81 ±22	82 ±24	Pre-shift vs. post-shiftp = 0.038Pre-shift vs. morningp > 0.05 – nsPost-shift vs. next morning p =0.036
Exercise 2 (Energy dissection – level 2)	71 ±19	66 ±22	72 ±21	70 ±23	Pre-shift vs. post-shift p = 0.032Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.028
Exercise 3 (Suture Sponge – level 3)	63 ±21	52 ±24	62 ±22	61 ±25	Pre-shift vs. post-shift p = 0.031 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.029

The results show that there is a statistically significant difference between the pre-shift and post-shift overall results for all exercises in the SD group and no statistical differences for the nSD group. Moreover, there are no statistical differences comparing the initial overall results for each exercise in the SD and nSD groups, validating the reliability and the reproducibility of the exercises included in the study.

The detailed specific metrics for the three exercises and the statistical significance of the results are presented below (Tables III–V).

Table III

Specific metrics for Exercise 1 – Peg board – level 1 (bold values for statistical significance, ns – not statistically significant)

Variable	Sleep deprivation (SD) group Mean ± standard deviation		No sleep deprivation (nSD) group Mean ± standard deviation		Statistical significance
	Pre-shift	Post- shift	Morning	Next morning
Time to complete [s]	94 ±28	96 ±26	95 ±25	96 ±29	Pre-shift vs. post-shift p > 0.05 – nsPre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p > 0.5 – ns
Instrument collision	0.91 ±31	0.95 ±32	0.90 ±30	0.89 ±33	Pre-shift vs. post-shift p = 0.037 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.031
Excessive force applied to instruments	0.06 ±0.02	0.12 ±0.1	0.07 ±0.01	0.06 ±0.01	Pre-shift vs. post-shift p = 0.028 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.030
Instruments out of view	0.34 ±0.07	0.43 ±0.08	0.35 ±0.1	0.34 ±0.09	Pre-shift vs. post-shift p = 0.028 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.028
Economy of motion [cm]	147 ±42	165 ±51	144 ±45	146 ±50	Pre-shift vs. post-shift p = 0.034 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.027
Drops	0.21 ±0.06	0.59 ±0.07	0.23 ±0.05	0.22 ±0.07	Pre-shift vs. post-shift p = 0.035 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.041
Master workspace	10 ±3	14 ±3	9 ±2	10 ±3	Pre-shift vs. post-shift p = 0.032 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.034

Table V

Specific metrics for exercise 3 – Suture Sponge – level 3 (bold values for statistical significance, ns – not statistically significant)

Variable	Sleep deprivation (SD) group Mean ± standard deviation		No sleep deprivation (nSD) group Mean ± standard deviation		Statistical significance
	Pre-shift	Post- shift	Morning	Next morning
Time to complete [s]	254 ±32	299 ±134	261 ±38	265 ±36	Pre-shift vs. post-shiftp = 0.037Pre-shift vs. morningp > 0.05 – nsPost-shift vs. next morningp = 0.042
Economy of motion [cm]	355 ±101	528 ±88	367 ±98	370 ±94	Pre-shift vs. post-shift p = 0.027 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.034
Instrument collision	2.4 ±0.8	8.4 ±1.2	2.5 ±0.7	2.6 ±0.6	Pre-shift vs. post-shift p = 0.032 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.039
Excessive force	0.2 ±0.03	0.7 ±0.07	0.3 ±0.05	0.2 ±0.06	Pre-shift vs. post-shift p = 0.030 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.030
Instruments out of view	0.78 ±0.19	1.42 ±0.4	0.71 ±0.5	0.75 ±0.4	Pre-shift vs. post-shift p = 0.030 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.024
Drops	0.1 ±0.02	0.9 ±0.03	0.2 ±0.02	0.2 ±0.01	Pre-shift vs. post-shift p = 0.027 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.03
Master workspace	7.6 ±1.4	9.1 ±1.6	7.5 ±1.3	7.4 ±1.2	Pre-shift vs. post-shift p = 0.037 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.032

For the easiest exercise, all the specific metrics evaluating the performance revealed statistically significant differences between pre-shift and post-shift, as well as between the post-shift and the morning after a normal sleep period overnight. The statistical difference in the SD group was observed for the economy of motion (p = 0.034 and p = 0.027 for pre-shift vs. post-shift and for post-shift vs. next morning, respectively). The only parameter without significant differences in the groups was the time to complete the task. The results from Table IV show that as the difficulty of the exercises increases, the statistical differences were observed between the pre-shift and post-shift as well as between the post-shift and the morning after to a normal overnight sleep period. Applying excessive force to the instruments was the most sensitive parameter in this exercise, which revealed sleep deprivation (pre-shift vs. post-shift, p = 0.031; post-shift vs. next morning, p = 0.032).

Table IV

Specific metrics for Exercise 2 – Energy dissection – level 2 (bold values for statistical significance, ns – not statistically significant)

Variable	Sleep deprivation (SD) group Mean ± standard deviation		No sleep deprivation (nSD) group Mean ± standard deviation		Statistical significance
	Pre-shift	Post- shift	Morning	Next morning
Time to complete [s]	105 ±24	117 ±23	106 ±25	105 ±27	Pre-shift vs. post-shift p = 0.036 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.037
Economy of motion [cm]	159 ±28	173 ±34	158 ±32	159 ±31	Pre-shift vs. post-shift p = 0.039 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.037
Instrument collision	0.5 ±0.1	0.9 ±0.1	0.4 ±0.08	0.5 ±0.09	Pre-shift vs. post-shift p = 0.041 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.040
Excessive force	0.29 ±0.06	0.56 ±0.1	0.31 ±0.08	0.29 ±0.09	Pre-shift vs. post-shift p = 0.031 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.032
Instruments out of view	0.2 ±0.03	0.6 ±0.04	0.3 ±0.03	0.2 ±0.01	Pre-shift vs. post-shift p = 0.042 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.042
Misapplied energy time	6.3 ±1.3	7.9 ±1.5	6.2 ±1.4	6.4 ±28	Pre-shift vs. post-shift p = 0.047 Pre-shift vs. morning p > 0.05 – nsPost-shift vs. next morning p = 0.047
Master workspace	15.2 ±3.4	19.4 ±4.3	14.7 ±4.2	15.1 ±3.4	Pre-shift vs. post-shiftp = 0.041Pre-shift vs. morningp > 0.05 – nsPost-shift vs. next morningp = 0.037

For the toughest exercise (Exercise 3 – results presented in Table V), statistically significant differences were noted between the pre-shift and post-shift as well as between the post-shift and the morning after a normal overnight sleep period, the economy of motion and drop faults being strongly correlated with the sleep deprivation.

When we subdivided the group with sleep deprivation into subjects (n = 8) with less than 3 h of sleep during the shift and with three or 4 h of sleep (n = 12), the overall results revealed a stronger statistical difference between pre-shift and post-shift (Table VI) for residents with longer sleep deprivation.

Table VI

Overall results on subgroups of less than 3 h of sleep vs. equal or more than 3 h of sleep (bold values for statistical significance, ns – not statistically significant)

Exercise	Less than 3 h of sleep N = 8 Mean ± standard deviation		Equal or more than 3 h of sleep N = 12 Mean ± standard deviation		Statistical significance
	Pre-shift	Post- shift	Pre-shift	Post- shift
Exercise 1 (Peg board – level 1)	82 ±21	75 ±20	83 ±22	80 ±23	Pre-shift vs. post-shift < 3 h p = 0.032 Pre-shift vs. post-shift < 3 h p = 0.046
Exercise 2 (Energy dissection –level 2)	72 ±18	63 ±15	73 ±17	68 ±20	Pre-shift vs. post-shift < 3 h p = 0.027 Pre-shift vs. post-shift < 3 h p = 0.033
Exercise 3 (Suture Sponge – level 3)	65 ±24	50 ±18	64 ±22	55 ±19	Pre-shift vs. post-shift < 3 hp = 0.029Pre-shift vs. post-shift < 3 hp = 0.036

Discussion

Published articles have demonstrated by functional cerebral magnetic resonance imaging (MRI) studies that sleep deprivation is related to a decreased signal from the dorsolateral prefrontal cortex, the same area that plays an important role in performing tasks requiring sustained attention [11]. Because surgical procedures require sustained attention throughout, it can be expected that the quality of different surgical steps of the operation will be altered. Moreover, slight chronic sleep deprivation/suboptimal sleep duration is as bad as an acute whole night without sleeping [12]. Although there is no consensus related to the optimal sleep duration in adults, there are studies showing that continuous sleep restriction to 6 h per day for a week leads to the same neurobehavioural performance as in subjects deprived of sleep for a whole night or alcohol intoxicated with blood alcohol levels of 0.04–0.05‰ [2, 13].

To our knowledge, this is the first study to assess the effect of sleep deprivation on a robotic virtual simulator. Up to now, most studies were done on laparoscopic simulators, which are unable to evaluate surgical skills in a 3D environment, with articulated instruments and intuitive motions.

The group was homogenous, with subjects having similar age and experience, avoiding biases related to this and reported in the literature [2]. A study reported that senior surgeons performed better than residents put in same sleep deprivation conditions, hypothesising that they had acquired a special ability regarding subjective alertness during their residency [8]. However, there are authors suggesting that even sleep deprivation/jet lag for experienced surgeons operating abroad during congresses with live surgery sessions can impair their surgical outcomes [14].

The overall results for the three exercises revealed no statistical difference between the results on pre-shift and the morning after a normal sleeping period (p > 0.05). At the same time, for all the three exercises, there were statistically significant differences between pre- and post-shift overall results. The post-shift overall results were statistically significantly worse than the overall results for the morning after a normal sleep period.

The time to complete the first exercise revealed no difference between pre- and post-shift, pre-shift vs. morning, and post-shift vs. next morning results (p > 0.05). This can be explained by the easy level of difficulty for this exercise and by the fact that the tasks can be accomplished by working memory and basic skills, which are not necessarily affected by sleep deprivation. Working memory is described as keeping task-relevant information for a few seconds to be used by other ongoing activities, and it seems not to be always affected by sleep deprivation [15]. Specific metrics for the first exercise, those related to skills and fine movements, show statistically significant differences between pre- and post-shift and between post-shift and morning after a normal sleep period. For the first exercise, the economy of motion was the most sensitive specific parameter to reveal a sleepy surgeon.

As the exercise difficulty increases, requiring more attention and dexterity, differences are obvious and statistically significant for all the specific metrics, including the time to complete the exercise. Active conscious concentration and working memory, which worked for the first exercise to obtain a similar time to complete, was unable to surpass a more complex exercise. These data are similar to those from other studies performed on a MIST-VR virtual laparoscopy simulator, whilst other authors found no differences or reported better results post-call, thought to be the result of a highly motivated cohort, being able to hold a high level of concentration for a predictable amount of time [1, 8, 16].

For the toughest exercise (Exercise 3) statistically significant differences were obtained between pre- and post-shift as well as between post-shift and morning after a normal sleep period. However, for all the specific metrics of this exercise, statistical significance was noted in comparison with previous exercises, showing that for more complex and longer tasks sleep deprivation leads to poorer outcomes. Interestingly, the overall results were statistically significant between pre- and post-shift irrespective of the period of overnight sleep (Table V).

This strengths of this study included the homogeneity of the study group regarding age and surgical experience, the fact that the control group was formed from the same subjects put in normal sleep conditions, and the objective assessment of the surgical performance. The limitations of this study are due to the relatively small group, the great variability of individual sleep patterns, the lack of an objective assessment of sleep quality, and different individual sleep propensity [17].

Although data from the literature are still debatable, it is clear that sleep deprivation leads at least to suboptimal surgical performance, and countermeasures should be found. Data from the literature suggest that napping (5–15 min) can be an effective measure to combat sleep deprivation for the next 1–3 h, without the risk of sleep inertia, which usually occurs following longer naps (30 min or more) [18, 19]. Sleep inertia after longer naps can be harmful for surgical activities within the first 20 min after waking, being at least as dangerous as sleep deprivation itself [19-21].

It is reasonable to believe that working time regulations will reduce sleep deprivation-related events for both the patients and the medical staff. On the other hand, measures to optimise and to adapt the training programs should be found in order to surpass the decreasing number of hours spent in hospitals during the residency program because, as Professor Aviva Katz from the Department of Bioethics an Health Law at the University of Pittsburgh, Pennsylvania stated in an academic debate related to the working hours of residents, “We can do better than hoping that residents will learn what they need if we just keep them captive long enough”.

Conclusions

Sleep deprivation leads to impairment of surgical skills assessed by a robotic virtual simulator. The more complex and skill demanding the exercise, the greater the difference between sleep deprived and non-deprived residents. Further, larger, and more complex studies could find the optimal measures to counteract sleep deprivation, besides reducing working hours.

Conflict of interest

The authors declare no conflict of interest.