Tangible effects of the COVID-19 pandemic: A fall in dexterity amongst surgical trainees?

BACKGROUND: The coronavirus pandemic has had an indisputable impact on surgical training. The qualitative effect on the perceived confidence and capability of trainees has been widely reported. In order to fully delineate the scope of the problem, quantitative data is also required.
METHODS: This cross-sectional study collected compared data on the laparoscopic skills (pre- and post-pandemic) of first-year urology residents in the United Kingdom, who attended the annual Urology Simulation Boot Camp (USBC) in 2018 and 2019 (pre-pandemic), and 2021 (post-pandemic).
RESULTS: Pre-pandemic group performance was significantly better in 2 out of 4 tasks (Task 3 p < 0.001) (Task 4 p = 0.003) during the practice session. During the assessment, pre-pandemic group performance was better (p = 0.017) for Task 2 and significantly faster (p = 003) for Task 4.
CONCLUSIONS: Our results provide evidence to support the notion that the coronavirus pandemic has had a tangible and detrimental effect on the technical skills of surgical trainees.

Introduction

The novel coronavirus (COVID-19) outbreak was declared as a global pandemic by the World Health Organisation (WHO) in March 2020, and continues to have a significant impact on healthcare systems across the world.[1] Reprioritization of workforce resources to meet medical demands and increase hospital capacity has resulted in the cancellation of elective operating lists, and redeployment of trainees to non-surgical specialties.[2] Not to mention the effects of personal illness or mandatory isolation during this time.

While commendable efforts have been made to deliver necessary education through the use of simulation models, remote or socially distanced teaching, and an increased focus on non-technical skills, there has nonetheless been a substantial reduction in the hands-on experience that forms the bedrock of classical surgical training and mastery learning.[3-5] The implications of this unparalleled global phenomenon on the perceived confidence and capability of surgeons in training have been reported in several qualitative studies.[4,6] Quantitative effects on manual dexterity and technical skills are alluded to but are harder to qualify. As a result they are less commonly reported in the literature, or are assumed through the use of surrogate measures such as operative logbook numbers.[7]

Simulation has long been used as a method of improving surgical capabilities, but has become more commonplace with the advent of increasingly sophisticated feedback models, safeguarding restrictions on trainee working-hours, and a drive to optimise surgical skills ex-vivo prior to operating on patients.[8] Performance in simulation has been proven to beneficially translate into clinical practice, and correlate with operating speed and quality.[9-11] Task-based measures in simulation may therefore provide a more accurate quantitative indicator of surgical performance.

The aim of this study is to provide an indirect measure of the impact of the pandemic on surgical trainees’ technical skills, by comparing pre- and post-pandemic outcomes on surgical simulators using standardised laparoscopic tasks. We hope that the findings will inform our understanding of training requirements and suggest avenues for future resource allocation.

Methods

The Urology Simulation Boot Camp (USBC) is a 5-day hands-on training course for newly appointed residents in the United Kingdom (UK). There are 6 technical skills and 2 nontechnical skills modules in the course. Basic laparoscopic skills training is one of the technical skills modules and the European-Basic Laparoscopic Urological Skills (E-BLUS)[12] exercises are part of the laparoscopic module. Our sample consisted of newly appointed Urology registrars in the UK who performed the E-BLUS exercise during the annually delivered USBC in 2018, 2019 and 2021.[13] The USBC was not conducted in 2020 due to COVID-19 pandemic restrictions.

The E-BLUS training consists of 4 exercises of increasing complexity performed on a laparoscopic skills simulator: Task 1 “peg transfer” – transfer of pegs one by one from one side of poles on the board to the other using two graspers and transfer back to the original position, Task 2 “cutting a circle” – cutting a circle within two lines on a double layer of gauze using a grasper and endoscopic scissors, Task 3 “Needle guidance” – guiding a suture needle through metal rings across the board using two needle drivers, Task 4 “Knot tying” – performing a single interrupted laparoscopic knot with one double-throw and two single-throws using a needle driver, grasper and endoscopic scissors.[12]

Each candidate was asked to perform all of the Tasks in succession, supervised by a consultant surgeon experienced in laparoscopic procedures. A validated assessment tool was used to assess technical skills.[12] Performance in each Task was timed as part of a practice round. On the final day of the course, candidates re-performed Tasks 2 and 4, which were timed as part of the “assessment” component of the course. In order to be successful, candidates must complete each exercise within a time limit determined by the E-BLUS exam, while meeting basic quality criteria. Time to complete each of the practice Tasks 1–4 and assessment Tasks 2 and 4 was recorded, in addition to the allocation of a pass/fail mark for each station. Candidates who did not have their Task completion time recorded were excluded from analysis.

Data from 2018 and 2019 were combined to measure “pre-pandemic’ performance, and data from the 2021 cohort was used for “post-pandemic”. Independent samples T-test was used to compare data on the speed of task completion, and Chi-squared test was used to compare categorical pass//fail data in assessment tasks, p < 0.05 was used as a significance threshold in all statistical analysis.

Results

A total of 47, 56 and 48 trainees attended the UBC in 2018, 2019 and 2021 respectively. Almost 90% of the same trainers attended all 3 courses. The numbers of participants that had their performance timed for each exercise per year are displayed in Table 1. Outcomes from the 2018 and 2019 cohorts were combined for comparison with the 2021 cohort. Table 2 displays the average time per Task for each cohort. Within the practice run, there was no significant difference between pre- and post-pandemic performance in Task 1 for either time taken or percentage pass rate. In Task 2, the pre-pandemic group were on average 26 s faster (267 vs. 293 s) and obtained a higher average pass rate of 8.8% compared to 2.4% in the post-pandemic group, although this was not statistically significant. Pre-pandemic performance was significantly better in Tasks 3 and 4. In Task 3 the average time to completion in the pre-pandemic group was 175 s less (399 vs. 574 s), which was statistically significant (p < 0.001). The pass rate was also higher at 16.7% compared to only 5.7% post-pandemic. Similarly, for Task 4 the pre-pandemic group was on average 107 s faster (417 vs. 524 s) (p = 0.003), with a significantly better pass rate of 50.5% vs. 23% (p = 0.004) (Figure 1).

Table 1.

Number of participants for each task, per year.

	2018	2019	2021
Task 1	46	56	42
Task 2	47	56	41
Task 3	47	55	35
Task 4	45	50	39
Assessment task 2	48	37	46
Assessment task 4	47	39	42

Table 2.

Average time per task in seconds, standard deviation (SD), and p-value.

	Pre-pandemic (2018 + 2019)	Post-pandemic (2021)	P value (*<0.05)
Task 1	191 (SD 82)	197 (SD 94)	0.693
Task 2	267 (SD 127)	293 (SD 136)	0.142
Task 3	399 (SD 140)	574 (SD 287)	< 0.001*
Task 4	417 (SD 176)	524 (SD 198)	0.003*
Assessment task 2	206 (SD 65)	238 (SD 88)	0.017*
Assessment task 4	329 (SD 129)	426 (SD 141)	< 0.001*

Figure 1.

Average time and pass rate for practice (baseline data) tasks 1–4.

In both assessment rounds, the pre-pandemic group performance was significantly faster. In Task 2 the average time was 32 s less (206 vs. 238 s; p = 0.017), with a higher pass rate of 22.4% compared to 15.2% in the post-pandemic group. Assessment round completion of Task 4 took an average of 97 s less in the pre-pandemic cohort (329 vs. 426 s; p < 0.001), and the pass rate was also significantly higher at 57% compared to 38% (p = 0.005) (Figure 2).

Figure 2.

Average time and pass rate for assessment tasks 2 and 4.

Informal feedback from examiners and course facilitators also indicated that on average candidate technical ability was poorer in the post-pandemic cohort, supporting the quantitative differences in the performance described above. Although global performance ratings were obtained for each candidate, these were not included in the analysis due to likely heterogeneity in reporting between different assessors.

Discussion

The landscape of surgical training has changed significantly over the past two decades. Increased emphasis on community-based medicine within medical schools and the UK foundation curriculum has resulted in a decrease in early surgical exposure.[14] This has been further compounded in higher training due to policies such as the European Working Time Directive (EWTD) and Modernizing Medical Careers (MMC), which have created a transition from firm-based to shift-based working, and a move away from the apprenticeship-model for learning in surgery.[15] The COVID-19 pandemic had had a catalytic effect on this trend, with many centres seeing a near standstill in surgical activity, and therefore traditional learning opportunities.

Our results indicate that the pandemic has had a significant negative impact on technical skills amongst urology trainees. In all but practice Tasks 1 and 2, trainees in the post-pandemic cohort performed significantly worse than their predecessors in either time taken or pass rate. Interestingly, a greater disparity between speed and pass rates was seen in higher-order tasks such as laparoscopic suturing. One possible explanation for this is that while basic laparoscopic skills can be acquired in earlier training or with previous simulation alone, progression to higher-order skills may be motivated by real-life exposure. The purpose of simulation is to provide an arena for deliberate practice to allow refinement of skills prior to operating on patients, and to ameliorate the steep learning curve that many new surgical trainees face. The validity of a simulation model may be asserted by its ability to predict actual performance.[16] During the pandemic, face-to-face skills training sessions were almost halted, and trainees were encouraged to substitute real-life experience for self-directed simulation practice if possible, with remote assistance; therefore it is surprising that our data suggest that even in a simulated setting, post-pandemic cohorts fall short in technical ability. This suggests that there is a bidirectional relationship between actual and simulated exposures, with real-life experiences positively contributing towards the retention of skills learnt in simulation.

A challenge facing current curriculum setters and surgical trainees is how to acquire the same level of technical ability as their predecessors within the confines of the post-pandemic landscape. While minimum requirements for operative logbooks and workplace-based assessments were temporarily lowered in some places to accommodate reduced training opportunities,[7] this is not a long-term solution as ultimately surgeon quality must not be compromised. Increasing individual working hours in order to meet training demands must not come at the expense of patient safety or surgeon burnout. Furthermore, stringent competency-based progression without regard for time spent in training may result in delayed progression and ensure a shortage of senior surgeons. Focusing solely on practical skill records as measures of competency may also be detrimental to training attitudes and result in the negation of learning in other valuable environments such as the ward and outpatient clinics in favour of more time spent in the operating theatre.

Although our study does demonstrate a quantitative reduction in technical ability in post-pandemic trainees, it is limited in that it is a single-specialty sample, and additionally only assesses laparoscopic competencies. Trainees from specialties that typically receive a greater emergency and trauma caseload may have seen preserved operative opportunities. Emergency cases are more likely to be open procedures, and laparoscopy was specifically prohibited in COVID-19 positive patients,[17] as a result it may be that laparoscopic skills were particularly compromised by the pandemic and not indicative of overall surgical capability. Nevertheless, as laparoscopic techniques are an increasingly fundamental component of many surgical fields,[18] our data does show a gap in an essential technical area which will need to be compensated. It is likely the impact of the pandemic on training may take longer to recover, therefore, training committees and programme directors should consider innovative methods to support skills training (e.g. regular remote assistance, immersive technology.[19] Future work in the form of cross-specialty studies with the incorporation of tactile feedback models may help to further elucidate the impact of the pandemic on the surgeons of today.

Conclusion

Our work provides evidence to support the theory that the COVID-19 pandemic has had a tangible and detrimental effect on the technical skills of surgical trainees. While technical proficiency is not the only attribute of a successful surgeon, it is undoubtedly essential. Therefore, it is important that going forwards more resources are dedicated to ‘catching up’ trainees who have had a compromised experience during this time. This may be through local interventions or widespread curriculum change.