Veterinary students' views on surgical entrustable professional activities and the impact of COVID-19 on clinical competence development.

BACKGROUND: The primary purpose of this study was to understand veterinary students' views on the described key surgical entrustable professional activities (EPAs) and to understand how COVID-19 restrictions have impacted their clinical skill and competence development.
METHODS: Final-year veterinary students at a single institute completed a web-based survey distributed by email. The survey aimed to characterise five constructs regarding EPAs, and a specific five-point Likert-like scale was created asking explicitly worded questions for each construct.
RESULTS: One hundred and ten students responded. The cohort agreed that the previously described key surgical EPAs were clinically important and relevant, but over 50% of the respondents felt that they had no substantial experience with them and were not confident or comfortable performing them. Additionally, most students (95%) felt their clinical development was negatively impacted by COVID-19.
CONCLUSIONS: The results of this study show that the key EPAs proposed are considered important skills by the undergraduate cohort described and that experience levels when entering the final year are lacking, potentially due to reduced exposure to clinical cases influenced by the COVID-19 pandemic.

INTRODUCTION

Undergraduate veterinary students in the UK are required to achieve a number of essential ‘Day 1 Competences’ prior to graduation and registration, as set by the RCVS. These competencies attempt to ensure patient and veterinarian safety, but despite a paradigm shift to outcome‐based training within the profession, it is not common practice for these competencies to be officially assessed prior to graduation. A contributing factor is that assessment of competence can be challenging in medical and veterinary education, especially in situations where the specific clinical task is conceptual or abstract, for example, determining competence in clinical reasoning or self‐awareness. , Entrustable professional activities (EPAs) have been described and implemented in various professions in an attempt to avert this concern by working to clearly and simply link competence to clinical practice. An EPA is a self‐contained unit of clinical practice consisting of multiple integrated competencies that can be entrusted, as a task, to a trainee or student to execute unsupervised. Tasks are easily trainable, easily assessable and vary in complexity, which makes them valuable tools in the development of competency‐based assessment frameworks within medical programmes and other curricula. , , , , In 2014, the Association of American Medical Colleges defined the 13 core EPAs used in medical education, which represent physicians’ tasks in daily clinical practice. Examples included gathering a history, performing a physical exam and prioritising a differential diagnosis list. Thereafter, a veterinary education working group described eight core EPAs specific for veterinary clinical practice, and in 2019, Favier et al. took a further step in defining surgery‐specific EPAs within veterinary practice. The group initially drafted 37 surgical EPAs, which were narrowed down to a final list of 13 key surgical EPAs, or tasks that veterinarians were expected to perform competently at the time of graduation. Due to the lack of exhaustive surgical exposure within most veterinary curricula in the UK, this expectation may be daunting for students, and associated stress could be detrimental to students’ learning experience and mental health. ,

There is an expectation for final‐year veterinary students to develop and refine their basic surgical skills in their clinical years of training prior to graduation. However, providing real‐life surgical learning opportunities in the latter stages of the undergraduate course is increasingly challenging due to focuses on the ethical use of cadavers, the associated cost of using cadavers and recent COVID‐19 restrictions. Historically, students would observe and assist surgeons during their final‐year clinical rotations at the teaching hospital, but with the rising numbers of undergraduate students and focuses on referral level services within university hospitals, hands‐on experience is often limited. It is a mandatory requirement for UK undergraduate veterinary students to complete 26 weeks of in‐clinic practice extra‐mural studies (EMS). EMS placements allow students to grow professionally and provide invaluable experience with clinical case management and first‐opinion level surgical experience in preparation for graduation. Unfortunately, the COVID‐19 travel and social distancing restrictions in the UK massively impacted the ready access to and completion of clinical placements, and as a result, core EMS requirements were revised by the RCVS for 2021, 2022 and 2023 undergraduates. The amended requirements state that 13 weeks of clinical EMS placements are now required to qualify for graduation. This will inevitably impact students’ clinical abilities, practical knowledge and development of interpersonal skills. More importantly, reduced surgical exposure is likely to impact graduates’ surgical competence and confidence, which could have wider ongoing consequences.

The aim of this study is to explore the views of UK final‐year veterinary students (2022 graduates) on the 13 key surgical EPAs published and to evaluate their views on the impact of COVID‐19 on clinical skill development.

MATERIALS AND METHODS

Participants and recruitment

Participants were all final‐year veterinary students at The Royal (Dick) School of Veterinary Studies in the academic year 2021–2022. The school runs a 5‐year undergraduate veterinary degree programme (UG‐P) for students entering directly from high school and a 4‐year graduate entry programme (GEP) for graduate students who have previously obtained a relevant degree. The two cohorts are blended from the third year of UG‐P and the second year of GEP. Participants were recruited via email, and participation was voluntary.

Survey design

The survey aimed to characterise five constructs regarding EPAs: student perspectives on the importance of each skill (skill importance), their current clinical experience with each skill (skill experience), their comfort level with each skill (skill comfort), perceived difficulty of the skill (skill difficulty) and whether students felt a model would help them develop their skill (skill model) (Appendix 1). For each of these constructs, a specific five‐point Likert‐like scale was created asking explicitly worded questions. For example, to assess ‘skill importance’, the question read ‘Please read the following surgical skills and decide for each one: How important is this skill for a newly graduated practicing vet?’ with five potential responses: ‘Very unimportant’, ‘Unimportant’, ‘Neither important nor unimportant’, ‘Important’ or ‘Very important’. There was also a free‐text response question asking students if there were any further comments they would like to contribute in relation to their surgical skills.

Analysis

All quantitative analyses were conducted in R Version 4.1.1. Broad characterisations of each skill construct were performed using the Likert package to establish the percentage of respondents with positive or negative perspectives. These variables were characterised across all participants, but also between the GEP and UG‐P groups, and across different COVID‐19 impact responses. There may have been relationships between the programme and COVID‐19 impact on participants’ responses, for example, those who felt COVID‐19 had a negative impact on their skill development may have felt more negatively about their skills in general. To explore this relationship, the validity of each construct was assessed via Cronbach's alpha using the psych package. While Cronbach's alpha estimates can be impacted by the number of responses and items, we wanted to retain as many items as possible to remain comparable with the existing literature. All constructs returned an alpha estimate greater than 0.7 (Table 1), which is often cited as a ‘minimum threshold’, so all items within each construct were retained. A summed construct score was then generated for each construct across all items within it. Differences in summed construct scores between programmes were explored using Welch's one‐way ANOVA to characterise the difference between programmes in the ggstatsplot package. Relationships between summed constructs across the assessment of COVID‐19 impact were explored using the Spearman rank correlation coefficient in ggstatsplot. A correlation matrix of each summed construct score was calculated using corrplot to explore relationships between the constructs.

TABLE 1

Crohnbach alpha estimates and 95% confidence intervals (CIs) for skill constructs

Skill	Alpha estimate	Lower 95% CI	Upper 95% CI
Skill importance	0.95	0.94	0.97
Skill experience	0.85	0.81	0.89
Skill difficulty	0.78	0.72	0.84
Skill comfort	0.83	0.78	0.83
Skill model	0.88	0.85	0.92

A reflexive thematic analysis was used to minimally organise free text comments into themes that categorised the data in an inductive or bottom–up approach. For example, themes were categorised based on their presentation within the comments as opposed to on a priori assumptions. Free text data were also tokenised into distinct lemmas (root words) that were explored for frequency alongside the thematic analysis. Thematic analysis was undertaken using NVivo Plus Version 12 software. Lemmatisation and tokenisation of text was conducted using the textstem and tidytext packages in R Version 4.1.1. ,

RESULTS

In total, 110/182 students responded: 62/109 students were from the UG‐P programme and 48/73 students were from the GEP programme. No responses required removal. The overall response rate for the entire graduating cohort was 61%.

Impact of COVID‐19 on skill development

The majority of students (95%) were concerned that the COVID‐19 pandemic and resulting UK lockdowns had a negative impact on their surgical skill development, and there was little difference between the programmes (Figure 1).

FIGURE 1

Students’ (n = 110) perspective on the impact of COVID‐19 on their practical skill development. The percentages of students with positive, neutral and negative impacts are shown from left to right. GEP, graduate entry programme; UG‐P, undergraduate programme

Student EPA perspectives

All skills were considered to be important for newly graduated veterinary surgeons by the majority (>50%) of all participants. As seen in Figure 2, repositioning of an ocular proptosis was considered to be unimportant or very unimportant by 21% of respondents. Gloving and gowning, and basic knot tying were considered to be the most important skills, with 95% of participants rating these as important or very important.

FIGURE 2

Student (n = 110) ratings of entrustable professional activity (EPA) skill importance. The percentages of students rating EPA skills as important, not important or unimportant, or unimportant are shown from left to right

As shown in Figure 3, there was no skill in which more than 50% of participants felt they had gained considerable or extensive practical experience while on EMS, but preparing patients for surgery, gloving and gowning and basic knot tying were the most practiced skills.

FIGURE 3

Student (n = 110) ratings of their experience with entrustable professional activity skills. The percentages of students rating their experience levels as extensive/considerable, a little or very little/no practical experience are shown from left to right

The difficulty of skills was more variably rated by students, with 90% of students considering repositioning an ocular proptosis to be complicated, while more than 60% of students considering basic knot tying, surgery patient preparation and gloving and gowning to be simple, as shown in Figure 4.

FIGURE 4

Student (n = 110) ratings of entrustable professional activity (EPA) skill difficulty. The percentages of students rating EPA skill difficulty as extremely simple or simple, neither simple nor complicated, and complicated or very complicated are shown from left to right

Students were generally not comfortable in their ability to perform EPA surgical skills, with the exceptions of surgery preparation, knot tying and suturing, and gloving and gowning–skills in which over 60% of students felt comfortable performing these skills. The results are shown in Figure 5.

FIGURE 5

Student (n = 110) ratings of their comfort levels with performing entrustable professional activity (EPA) skills. The percentages of students rating their comfort levels as very comfortable or a little comfortable, neither comfortable nor uncomfortable, and a little uncomfortable or very uncomfortable with performing EPA skills are shown from left to right

Finally, students considered surgical models to be a potentially useful tool for mastering all skills except for gloving and gowning, where 37% of students thought a model would not be useful in helping to practice these skills (Figure 6).

FIGURE 6

Student (n = 110) ratings of how a model would support their learning. The percentages of students rating their views as to whether a model would be very useful or useful, neither useful nor not useful, or not useful or not at all useful are shown from left to right

The total skill scores were independent from one another in a correlation matrix (Figure 7), except for comfort and experience. Students with higher overall comfort with EPAs also had higher total experience scores (r = 0.67, 95% confidence interval [CI] [0.55, 0.76], p < 0.001).

FIGURE 7

Correlation matrix of summed entrustable professional activity skill scores (total model, total difficulty, total experience and total comfort). Each box represents the strength of the association through the correlation coefficient and colour. Nonsignificant correlations (alpha greater than 0.05) are indicated with an ‘×’, n = 110 final year students

EPA skills by programme

There were significant associations between programme and student perspectives on skills (Table 2). GEP students were more likely to have a lower cumulative skill importance score than UG‐P students (mean scales of 19.81 and 25.27, respectively). This appeared to be mainly driven by a small number of UG‐P students who had very high skill importance scores, rating all skills equally important (Figure 8). There were no significant differences between the programmes in their cumulative ratings of their experience, the difficulty of a skill, their comfort in performing a skill or whether students felt a model would improve their skill use.

TABLE 2

Differences between student perspectives on entrustable professional activities by programme (undergraduate programme [UG‐P, n = 62] vs. graduate entry programme [GEP, n = 48])

Response variable	Mean score	Model parameters
		Welch's ANOVA	g Hedges	95% CI	p
Total skill importance score (n = 110)	UG‐P = 25.27	t Welch(79.95) = –3.31	–0.60	[–0.96, –0.023]	0.001
	GEP = 19.81
Total experience score (n = 110)	UG‐P = 52.97	t Welch(104.63) = –0.87	–0.16	[–0.54, 0.21]	0.388
	GEP = 51.79
Total skill difficulty score (n = 110)	UG‐P = 40.79	t Welch(98.16) = 0.93	0.18	[–0.20, 0.55]	0.352
	GEP = 41.81
Skill comfort score (n = 110)	UG‐P = 46.35	t Welch(100.65) = –0.25	–0.05	[–0.42, 0.33]	0.802
	GEP = 46.00
Model skill use (n = 110)	UG‐P = 24.32	t Welch(106.71) = –1.17	–0.22	[–0.59, 0.15]	0.246
	GEP = 22.60

Note: Welch's one‐way ANOVA model parameters provided.

Abbreviation: CI, confidence interval.

FIGURE 8

Impact of student programme (graduate entry programme [GEP] vs. undergraduate programme [UG‐P]) on summed skill importance, both summary statistics (boxplots) and distributions (violin plots) shown per programme. Programme means indicated in red. CI, confidence interval

EPA skills and COVID‐19 impact

Students who felt that COVID‐19 had no impact or a positive impact on their practical skill development were significantly more likely to have a lower rating of their total skill experience score (r = 0.29, 95% CI [0.10, 0.46], p = 0.002) and their total skill comfort score (r = 0.26, 95% CI [0.07, 0.43], p = 0.006). These correlations, while significant, do not represent strong relationships, and are presumed to be caused by the relatively low summed skill scores of the six students who felt COVID‐19 had positively impacted their skill development. Two of these students left free text comments, one of which did not elaborate on the COVID‐19 impact, but one which said that specific teaching sessions had ‘honed in’ on specific skills.

There was no evidence of a relationship between students’ ratings of COVID‐19 impact and the total skill importance score, their total skill difficulty score, or whether a model would support their skill learning (Table 3).

TABLE 3

The association between the impact of COVID‐19 on students’ perspectives on entrustable professional activity skill development (five‐point scale)

	Test parameters
Response variable	loge(S)	r	95% CI	p
Total skill importance score (n = 110)	12.38	–0.07	[–0.26, 0.12]	0.461
Total experience score (n = 110)	11.97	0.29	[0.10, 0.46]	0.002
Total skill difficulty score (n = 110)	12.36	–0.06	[–0.25, 0.14]	0.555
Skill comfort score (n = 110)	12.01	0.26	[0.07, 0.43]	0.006
Model skill use (n = 110)	12.24	0.06	[–0.13, 0.25]	0.509

Note: Spearman rank correlation test results provided.

Abbreviation: CI, confidence interval.

Free text comments

Of the free text comments (n = 32), 52% were left by GEP students and the remainder were left by UG‐P students. Free text comments were left by 6% by students who rated the impact of COVID‐19 on their skill development as positive. ‘EMS’, ‘practice’ and ‘COVID’ were common words within the comments, with students also discussing surgery types and specific skills such as suturing and spays.

The free text comments were broadly separated into students discussing their skill development during university‐based teaching and students discussing their skill development during EMS. While there was some overlap between these themes, they seemed to be considered two separate forms of teaching by the participants.

EMS was commonly considered to be negatively impacted by the COVID‐19 pandemic, with five of the 13 comments referencing EMS highlighting the impact of COVID‐19 specifically.

‘Covid‐19 caused many of my EMS placements to be cancelled or postponed, some of which I still have not been able to attend as the pandemic is only just beginning to be over (hopefully). I hope that by the time I graduate I am able to practice these skills, but as of today I have almost no clinical EMS experience and feel very unprepared’—Participant T

There were nine participants who felt that the academic curriculum was insufficient in preparing them for practicing veterinary surgical skills on EMS.

‘Our school teaches us/allows us to practice many different suture patterns but not intradermal sutures, which seems to be the most popular skin closure suture pattern for small animals. It was quite frustrating getting to an EMS clinic and telling them I've practiced sutures but had absolutely no clue how to perform an extremely important pattern. Because of this, my EMS vets did not let me close the skin on any patients (for good reason)’ —Participant Q

Nineteen of the 31 comments referenced additional surgical skill teaching, both positively and negatively. The proposed use of models was generally positively received and often linked to the impact of COVID‐19 on this cohort.

‘Getting more models to practice these skills would be very beneficial as most of us have not had the opportunity to practice these skills on EMS due to covid’ —Participant I

DISCUSSION

Favier et al. proposed 13 key surgical EPAs that a veterinary student must be able to perform with ‘indirect supervision—not present but quickly available’ at the time of graduation. Our aim was to explore the views of final‐year veterinary students on these 13 key surgical EPAs and to understand the impact of COVID‐19 on their development of surgical skills. This survey found that the majority of final‐year veterinary student respondents agreed that the 13 surgical EPAs were clinically relevant and important surgical skills for the newly graduated veterinarian. Over 50% of the cohort felt that they had no substantial experience with all of the listed key surgical skills. The more complex and procedural surgical skills (e.g., treating a traumatic wound, removing a simple mass in the skin and castration of a healthy dog) were deemed to be more difficult to master than the more granular skills (e.g., gloving, gowning, knot tying). Additionally, participants were generally not confident in their ability to perform any procedural surgical skills, in contrast to the granular skills where over 60% of participants felt comfortable performing them. Almost all students (95%) were concerned that the COVID‐19 pandemic and resulting lockdowns had a negative impact on their ability to gain clinical experience and their overall surgical skill development, which was reinforced with free text comments.

These results showed that students generally do not feel competent in surgical skills. There are several potential explanations for this finding. Firstly, this may be an inaccurate assessment on the part of the students, as they are self‐reporting their own perceptions. A student's self‐assessment of their own skills is an important aspect of the final‐year programme, but imposter syndrome and the resulting inability to accurately assess one's own abilities is a known phenomenon among medical students and throughout higher education. , , These students may be daunted by the scale of the tasks ahead of them and, knowingly or unknowingly, underreporting their abilities. It is possible that students perceive procedural skills to be more challenging to master due to the anxiety surrounding negative outcomes in ‘real patient’ scenarios compared to the granular skills that have no ‘life‐or‐death’ consequences. It is also likely that students feel more comfortable with granular skills due to their earlier introduction to these concepts in their training (e.g., patient preparation and aseptic techniques). This theory is supported by the significant correlation shown between the level of practical experience and student comfort levels performing the surgical EPAs. Alternatively, the students’ self‐assessment of their ability may be accurate, and these students are not competent in the more complex surgical skills. This may be influenced by the impact of the COVID‐19 restrictions in this cohort. COVID‐19 had an immeasurable impact on veterinary teaching in 2020 and 2021, with the RCVS temporarily amending its EMS guidance to accommodate COVID‐19 restrictions, as mentioned. This cohort is expected to complete 13 weeks of clinical EMS alongside online top‐up development as opposed to 26 weeks of placement learning. The impact of the COVID‐19 disruption on perceived competence was shown in our findings, although it should be noted that a few students felt that there had been no or limited impact.

Many students felt these key skills were important and valuable to being a veterinary practitioner. The only difference between the GEP and UG‐P cohorts of students was their overall skill importance score. GEP students typically complete a relevant science‐based degree and usually gain clinical experience within veterinary practice prior to progressing into a veterinary programme. This disparity in skill importance may reflect underlying differences in previous teaching methods or may be influenced by GEP students having more experience with observed clinical practice. Their view of surgical skill importance may be biased by what they have hitherto witnessed in everyday clinical practice. Interestingly, repositioning an ocular proptosis was perceived to be the least important skill for a newly graduated veterinarian by this cohort and was considered difficult to perform by 90% of students. It is reasonable to assume that not all skills reported are perceived as equally important, and the agreement within these results suggests that this skill is less essential on the expectation that key surgical skills should be straightforward for undergraduate students to master. A limitation is that the list of 13 surgical skills was devised based on a veterinary population in the Netherlands. The procedure list was directly comparable with the list published by Hill et al. using the American general practitioner veterinary population, but it is possible that individuals from different geographical locations may have differing views on key skill importance. Additionally, the scaling used within the questionnaire may have provided a potential source of error and less accurate responses, as the options used are likely to elicit a positive or negative response rather than true gradations when relating to opinions of skill importance and comfort. Further work exploring the value of surgical EPAs in practice would be useful.

It was suggested that the proposed surgical skill list could aid the development of a competency‐based assessment framework in veterinary education, enabling students to be deemed competent prior to graduation. Veterinary students within the UK believe these surgical EPAs to be necessary for practice, but respondents to the current survey do not feel they have sufficient experience with the skills to perform them. Currently, students do not have a formal clinical assessment of surgical skills in the context of a veterinary practice prior to graduation, and with this, we cannot assume surgical competence at the time of registration with the RCVS. As discussed, COVID‐19 restrictions have obstructed access to clinical experiences and the ability to develop these surgical skills. The impact on clinical EMS has been significant, and although additional practical teaching has been implemented to account for this, it is important to recognise the significant input that EMS has on the veterinary programme. The EMS system has been targeted for improvement, particularly regarding the diversity and availability of placement. The COVID‐19 crisis has highlighted another challenge with EMS, such as its inability to be delivered through programmes themselves and thus the impact on skill development. In the recovery from COVID‐19, access to clinical EMS placements and clinical rotations will continue to be affected to some degree, and consequently, the core EMS requirements for registration set out by the RCVS will continue to be impacted (minimum 13 weeks placement) for the 2022 and 2023 graduating cohorts. As such, alternative clinical and practical experiences for final‐year students should be explored. With this in mind, the authors consider it to be pertinent to the continued progression of the profession that surgical skill development is more fully explored, and as suggested by Favier et al., the EPAs could be used to build competence‐based assessment criteria to ensure and validate basic surgical competence prior to course completion. Not only will these criteria offer key quality assurance and skill enhancement within professional practice, but they may also support final‐year veterinary students in understanding their own competence, potentially reducing graduate stress and improving employability.

To amalgamate these skills into a curriculum and assess competence, students must have repeated opportunities to access, observe or partake in the completion of these skills. Bowlt et al. found that repeated exposure to surgical skills increases understanding, confidence and clinical ability in veterinary students. Most students in this study felt that surgical models were considered a potentially useful tool for mastering surgical skills, with the exception of granular surgical skills. Model‐based surgical skill training has already been shown to be successful in improving student confidence and ability at various stages in veterinary medical education. It has been shown that surgical courses significantly increase anxiety in veterinary students compared to nonsurgical courses, but interestingly, prior practice of surgical skill using low‐cost surgical training models reduced that anxiety. Smeak describes how the use of surgical models and simulators effectively guaranteed that basic surgical skills were mastered prior to exposure to cadavers. Au Yong et al. found a positive shift in students’ self‐perceived proficiency with gastrointestinal surgery following the use of a model. Similar findings were published by Badman et al. when looking at low‐fidelity ovariohysterectomy training models. Although high‐fidelity models are available, they are typically expensive and thus hard to provide to every student. A group in Copenhagen implemented the use of a low‐cost, build‐it‐yourself, ovariohysterectomy training model (the SimSpay) after it had been shown to improve student competence, confidence and anatomical knowledge. Limitations of model training include associated costs, staff demand and space availability.

In conclusion, the results of this study show that the 13 key EPAs proposed are considered important skills by the undergraduate cohort described and that experience levels when entering the final year are lacking, potentially due to reduced exposure to clinical cases influenced by the COVID‐19 pandemic. Participants felt that surgical training models may be beneficial in improving some of the proposed surgical skills. As such, creating EPA surgical models and allowing repeated exposure to those key surgical EPAs over a period of training should be considered, and the possibility of using the models to develop a competency‐based assessment framework should be explored. Model repeatability and uniformity may allow for quality assurance of basic surgical competence and safeguarding of patients, as well as having a positive effect on students’ performance ability, stress levels and emotions and overall learning. This study was primarily an exploratory analysis focused on the recruitment of undergraduate veterinary students from a single institute, which limits the scope for generalisation across other institutes. Additionally, this study relied on voluntary participation, which could introduce a degree of selection bias. A larger survey incorporating multiple veterinary schools in the UK would be useful in determining the perceived impact of COVID‐19 on surgical skill development among final‐year students across different programmes and would also aid in conclusion of the importance of the 13 key surgical EPAs in the wider population.

CONFLICTS OF INTEREST

The authors have no conflicts of interest to declare.

AUTHOR CONTRIBUTIONS

All authors contributed equally to this research paper.

ETHICS STATEMENT

This research received ethical approval from The Royal (Dick) School of Veterinary Studies Human Ethical Review Committee, reference HERC 705‐21.

FUNDING INFORMATION

The authors did not receive a specific grant for this research from any funding agency in the public, commercial or not‐for‐profit sectors.

Appendix. 1 – Survey used in the study

Click here for additional data file.