Impact of the first COVID-19 lockdown on study satisfaction and burnout in medical students in Split, Croatia: a cross-sectional presurvey and postsurvey.

OBJECTIVES: To evaluate the impact of the first COVID-19 lockdown in 2020 on the burnout and study satisfaction of medical students.
DESIGN: A cross-sectional study with a presurvey and postsurvey.
SETTING: University of Split School of Medicine (USSM), Split, Croatia. The lockdown in the COVID-19 pandemic lasted from late March to mid-May 2020. There was a full switch to e-learning at the USSM during this period, and all clinical teaching was stopped. PARTICIPANTS: Students enrolled in the 2019/2020 academic year. Data were collected before lockdown in December 2019 and January 2020 and again after the end of lockdown in June 2020. PRIMARY AND SECONDARY OUTCOME MEASURES: Study satisfaction was assessed using the study satisfaction survey. Burnout was assessed using two instruments: Oldenburg Burnout Inventory and Copenhagen Burnout Inventory. We used Bayesian statistics to compare before-and-after differences.
RESULTS: 437 independent responses (77.2% response rate) were collected before and 235 after lockdown (41.5% response rate). 160 participant responses were eligible for pairing. There was no significant difference for both paired and unpaired participants in study satisfaction before (3.38 on a 1-5 scale; 95% credible interval (95% CrI) 3.32 to 3.44) and after (3.49, 95% CrI 3.41 to 3.57) lockdown. We found no evidence (Bayes factor (BF10) >3.00 as a cut-off value) for an increase in the level of burnout before and after lockdown, both in independent and paired samples.
CONCLUSIONS: It seems that the first pandemic-related lockdown and a switch to e-learning did not affect burnout levels among medical students or their perception of their study programme. More insight is needed on the short-term and long-term effects of the COVID-19 pandemic on medical students and their education. Well-structured longitudinal studies on medical student burnout during the COVID-19 pandemic are needed. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Medical students were surveyed before and after the first COVID-19 lockdown, which allowed a clear comparison of all study satisfaction and burnout scores prelockdown and postlockdown.

Methodological strengths include a large sample size, pairing of students’ responses and two burnout inventories used.

Methodological limitations include a short follow-up time period and the loss of participants to follow-up.

Introduction

The 2020 COVID-19 pandemic has had far-reaching effects on global life and economy, with many countries enacting measures to prevent community infection through quarantine-based restrictions on social activities.1 A systematic review reported that the pandemic caused an increase in adverse psychiatric symptoms in general populations of high-income, middle-income and low-income countries.2 The question of mental health in specific populations has been discussed since very early in the outbreak,3 with calls for timely mental health interventions for healthcare professionals and providers. Immense psychological pressure has been placed on healthcare workers since early on in the pandemic,4 with resulting psychological distress encountered in hospital staff,5 along with numerous reports of burnout in healthcare providers.6–9 Burnout is defined as the state of mental and physical exhaustion related to improperly managed occupational or workplace stressors10 and may impact quality of care when present in healthcare providers.11 As a population being initiated into the healthcare profession, medical students are also at higher risk for depression, anxiety and psychological distress than their non-medical peers,12 as well as at higher risk of burnout, emotional exhaustion, and increased levels of fatigue.13 A 2019 prepandemic meta-analysis demonstrated a high prevalence of burnout among medical students.14 Even before the COVID-19 pandemic, the quality of the learning environment and any perceived poor organisation of clinical teaching was reported to contribute to burnout in medical students.15 The significance of burnout in medical students is concerning in light of its influence on unprofessional conduct, such as dishonest and cheating behaviour, as well as the likelihood of reporting less altruistic attitudes about physicians’ responsibility to society.16 It is also associated with an increased likelihood of suicidal ideation.17

Negative effects of the COVID-19 crisis on the mental health of non-medical university students have already been reported early on,18 19 with more students experiencing higher rates of stress and perceived social isolation,20 as well as a higher prevalence of anxiety and depression than in the pre-COVID-19 era.21 Medical students, along with experiencing increased deterioration of their mental health in the pandemic, also expressed concerns about the pandemic disrupting their studies and not allowing them to adequately prepare for clinical practice.22 It is without question that the COVID-19 pandemic affected both the preclinical and clinical aspects of medical education with very quick initial transitions into online and small group education formats.23 24 Although e-learning provides many opportunities, it may not be entirely suited to disciplines such as medicine, which require a hands-on approach.25 Sudden changes to the education format that require extensive screen use may negatively impact burnout, in addition to mental health,26 and it has been reported that medical students spend more time on online teaching platforms during the pandemic compared with prepandemic times.27

Rates of medical student burnout have not been widely reassessed in the context of the COVID-19 pandemic, and a small number of studies assessed burnout due to e-learning. One study found a negative impact of e-learning during lockdown on students’ mental health but no conclusive change in burnout.28 Another study found that e-learning improved medical students’ mental health and decreased their burnout levels.29 Findings on medical student satisfaction with e-learning programmes and online teaching are currently mixed. In some places, medical students readily accepted the changes in the education format,30 31 while others reported that students still preferred face-to-face teaching.27 Another study on health sciences students found higher satisfaction with e-learning in developed countries than in developing countries, as well as that e-learning was perceived as good for theoretical knowledge but lacking in terms of practical and clinical experience.32 The effectiveness of different e-learning modalities and differences between them is currently also still unexplored.33 Further elucidating the ways in which the COVID-19 pandemic has had an impact on medical education and students might help tailor appropriate interventions and solutions for remote learning and class organisation, especially in regard to clinical practice, where remote learning may be insufficient in the long term.23 34

Our study was initially planned in 2019 with the aim of comparing medical student study satisfaction and burnout between clinical and preclinical study years, as well as using a follow-up survey to assess possible changes as the academic year goes on. However, as our institution experienced a full transfer to e-learning on the institution of national lockdown in Croatia that began on 23 March 202035 and lasted until 11 May 2020,36 we were able to obtain a unique dataset right before and after the institution of the first ever pandemic-related lockdown. During this lockdown period, universities and schools were closed and classes were suspended throughout Croatia. Our institution transferred to a fully virtual environment, with no clinical hands-on teaching but with a full class-hour timetable.37 We decided to explore burnout in the context of the emerging situation and adapted our initial study plan in relation to it. Study satisfaction, which we also planned to investigate previously, now also provided a proxy measure of satisfaction with the change in teaching methods used during lockdown. Due to the lack of clinical teaching in the lockdown, we hypothesised that there would be an increase in burnout and that study satisfaction would decrease. Thus, the aim of our study was to evaluate the impact of a COVID-19 lockdown on burnout and study satisfaction of medical students.

Methods

Study design and setting

This was a cross-sectional study with a presurvey and postsurvey. The study was undertaken at the University of Split School of Medicine (USSM), a medical school in Split, Croatia. We assessed study satisfaction and burnout in students in December 2019 and January 2020, before COVID-19 lockdown, which was instituted by the Civil Protection Headquarters on 23 March.35 Full lockdown was in effect until 11 May.36 At the USSM, online classes and both written and oral exams were conducted via an e-learning platform (both synchronous and asynchronous forms of teaching, depending on the subject), while all forms of clinical teaching ceased in this period.37 A follow-up survey was performed in June 2020, after the end of national lockdown.

Participants

The participants were medical students at the USSM from all six academic years. The USSM medical programme in Croatian had 566 students in total enrolled into the 2019/2020 academic year. The school has multiple study programmes that, besides Croatian medical students, also include pharmacy students, dental medicine students and students attending medical studies in English. We included only medical students from the Croatian programme because of their increased vulnerability to burnout.12 Students from the medical studies in English were not included because not all of them were in Croatia at the time of lockdown.

Data collection

Data were collected via surveys given to students at two data collection points: (A) prior to the COVID-19 outbreak in Croatia and Europe (in December 2019 and January 2020); and (B) during the COVID-19 pandemic, after the period of national lockdown, while university courses were still held online (1–20 June 2020).

The first survey of the study was delivered as a paper-and-pen questionnaire during regular classes 2 months into the academic year for all six study years. Because of the lockdown, the second survey was conducted online via the SurveyMonkey web-based platform (SurveyMonkey, San Mateo, California, USA). To maximise the response rate, we disseminated the survey as an invitation link to students through social media groups related to the medical programme and groups for different study years (Facebook and WhatsApp), as well as through the USSM website. The students completed the surveys in Croatian (the official language at the university).

A student who had one or more items missing for a certain survey item was considered a non-responder for that survey. Students who submitted a blank questionnaire or an incomplete questionnaire with no full survey item available for analysis were also excluded from the analysis. Duplicate questionnaire submissions were resolved by excluding the second submission.

Participant pairing

We asked the students to generate a five-element unique identifier code to pair their responses at both data collection points, using methodology previously described in similar studies at the University of Split.38 39 The code consisted of the first letter of the participants’ name, first letter of their mothers’ name, first letter of their fathers’ name, last two digits of their birth year and first letter of the place they were born. Participants eligible for pairing responses had to have a minimum of one survey item available for analysis, that is, full completion of at least one of the following parts of the questionnaire: study satisfaction survey, disengagement or exhaustion subscales of the Oldenburg Burnout Inventory (OLBI), or personal burnout, studies-related burnout, professor-related burnout, student-related burnout and patient-related burnout subscales of the Copenhagen Burnout Inventory (CBI).

Patient and public involvement

Neither patients nor the public were involved in conceptualising or conducting this study. USSM students were involved in disseminating information about the online survey after lockdown on social media platforms. The first author (MFŽ) was a final-year medical student at the time of the lockdown and data collection phase and was thus involved in the conceptualisation and conduct of the study.

Ethical considerations

The study questionnaires were anonymous. The codes created by the respondents in order to pair presurvey and postsurvey data were not identifiable, and no attempt was made at identification of the respondents. Participants were informed at the beginning of the survey about the purpose of the study and that they can leave at any moment. They were also informed that the online survey was set not to collect IP addresses to protect personal data, according to the General Data Protection Regulation.40

This study was a part of the Professionalism in Health – Decision Making in Practice and Research project of the Croatian Science Foundation (grant No. IP-2019-04-4882).

The study design was based on a similar study conducted in 2013 by colleagues at the USSM. Their study, containing unpublished data, was not finalised and published. We contacted the authors and obtained their permission to pursue the topic of burnout at the USSM, and one of them (KJ) agreed to continue working with us on the present study, which was further adapted to the context of the COVID-19 pandemic.

Survey

The survey consisted of three questionnaires, one relating to study satisfaction and two relating to burnout. In order to confirm the validity of the translated survey, an independent language expert uninvolved in the study then back-translated the text the authors had translated into Croatian. The back-translated English version of the full survey questionnaire can be found in online supplemental appendix A.

The reliability of each instrument was assessed using Cronbach’s alpha coefficient. We also conducted an exploratory factor analysis to confirm the validity of instruments that were previously not validated in Croatian, the study satisfaction survey and CBI (online supplemental appendix B).

Demographic characteristics

At the beginning of the questionnaire, we collected basic demographic data: gender, age and the grade point average (GPA) on a 2.0–5.0 scale at the time of survey completion.

Instruments

Study satisfaction survey

Study satisfaction was measured by an eight-item questionnaire, previously developed as a modified version of the 2019 UK National Student Survey.41 The modification was performed by our colleagues in 2013, and we further adapted the questionnaire by shortening it to include only items strictly related to study satisfaction. Even though we initially adapted nine items, we piloted the questionnaire in this study, and a subsequent factor analysis showed that one item (question 4) had to be excluded (online supplemental appendix table B.1). The final version of the survey had eight items. The questions offered answers on a five-point Likert-type scale ranging from 1: never/almost never to 5: always/almost always (score range 8–40). The scale range was chosen because it matches scores in Croatian education system (including higher education), where 1: insufficient, 2: pass, 3: good, 4: very good and 5: outstanding. No items were coded in reverse. Cronbach’s alpha of the questionnaire was α=0.83 (95% CI 0.81 to 0.85). Higher scores indicated higher study satisfaction. Participants with scores >2.00 were considered adequately satisfied with their studies. This cut-off was chosen by the authors because 2.00 and above is the cut-off point for a passing grade in Croatian education system.

Oldenburg Burnout Inventory

Burnout was assessed by the 16-item OLBI questionnaire,42–44 which was previously validated in Croatian.45 OLBI measures two different aspects of burnout: disengagement and exhaustion dimensions. These two dimensions of burnout were treated as separate variables. Assessment for both dimensions was based on a four-point Likert-type scale ranging from 1: never/almost never to 4: often (score range 16–64). The items on the OLBI scale are usually scored on a 4-point Likert scale that ranges from 1: strongly agree to 4: strongly disagree. We, however, used a reverse four-point scale ranging from 1: never/almost never to 4: often. This was done to maintain consistency with the scale direction in the other two parts of the questionnaire and to avoid confusion during the survey. For the analysis, the points on the scale were reversed to match the original score, and the results were expressed as the mean score of all items for each dimension.

Disengagement was assessed by eight items (nos. 1, 3, 6, 7, 9, 11, 13 and 15); four of them reversely coded (nos. 3, 6, 9 and 11). Cronbach’s alpha was α=0.68 (95% CI 0.64 to 0.71).

Exhaustion was assessed by eight items (nos. 2, 4, 5, 8, 10, 12, 14 and 16); four of them reversely coded (nos. 2, 4, 8 and 12). Cronbach’s alpha was α=0.81 (95% CI 0.79 to 0.83).

Higher scores indicated more burnout for both dimensions. Those with exhaustion scores of ≥2.25 were considered to have high exhaustion, while disengagement scores ≥2.10 were considered to have high disengagement.46

Copenhagen Burnout Inventory

Burnout was also assessed by a modified 31-item CBI scale, which normally measures three dimensions of burnout: personal burnout, work-related burnout and client-related burnout.47 We modified the scale by expanding the client-related dimension into three dimensions relevant to our population: professor-related burnout, student-related burnout and patient-related burnout, similar to the modification of the CBI for medical students by Bolatov et al.29 Work-related burnout was renamed studies-related burnout. In total, we measured five dimensions of burnout: personal burnout, studies-related burnout, professor-related burnout, student-related burnout and patient-related burnout. The patient-related burnout dimension, although measured, was not calculated or included in any analyses for preclinical study years (years 1–3), as these years do not usually have extensive contact with patients.

Each dimension was treated as a separate variable. Assessment for all five dimensions was based on a five-point Likert-type scale ranging from 1: never/almost never to 5: always. Scoring of responses: always=100, often=75, sometimes=50, seldom=25 and never/almost never=0. Results were expressed as the mean score of all items for each dimension. Higher scores indicated higher burnout. Scores of 50–74 were considered moderate burnout, 75–99 were considered high burnout and a score of 100 was considered severe burnout.47 48

Personal burnout was assessed by six items (nos. 1–6, score range 0–600). No items were coded in reverse; Cronbach’s alpha was α=0.88 (95% CI 0.87 to 0.90).

Studies-related burnout was assessed by seven items (nos. 7–13, score range 0–700). One item was coded in reverse (no. 13); Cronbach’s alpha was α=0.87 (95% CI 0.85–0.88).

Professor-related burnout was assessed by six items (nos. 14–19, score range 0–600). No items were coded in reverse; Cronbach’s alpha was α=0.92 (95% CI 0.91 to 0.93).

Student-related burnout was assessed by six items (nos. 20–25, score range 0–600). No items were coded in reverse; Cronbach’s alpha was α=0.95 (95% CI 0.95 to 0.96).

Patient-related burnout was assessed by six items (nos. 26–31, score range 0–600). No items were coded in reverse; Cronbach’s alpha was α=0.94 (95% CI 0.93 to 0.95).

Sample size

We assumed that lockdown would have a mild negative affect on participants, so that in the second assessment, participants would have scores on burnout measures increased by at least 5%. With an alpha level of 5% and 80% power, we calculated that we would need a minimum of 65 paired participant responses to observe the desired difference. To account for any non-responder losses, we invited the whole student population of the medical studies in Croatian to complete the survey.

Data analysis

All statistical analyses were performed using JASP software V.0.13.1.0 (JASP Team, 2018, Amsterdam, Netherlands).

We expressed demographic characteristics of the sample as frequencies and percentages population. Study satisfaction was also expressed as median and IQR for before and after lockdown. The response rate was calculated for overall responses collected before and after lockdown and additionally for each study year for the responses that were paired, also expressed as N (%). For paired participants, we included their gender demographics and GPA separately for each study year. GPA was expressed as a mean and 95% CI on a 2.00–5.00 scale.

We then used the Bayesian independent samples t-test to compare all responses before and after lockdown. Next, we tested for differences between the paired participant responses before and after lockdown using the Bayesian paired samples t-test. We used the Bayesian independent samples t-test to compare study satisfaction and burnout in participants who provided responses before and after lockdown with those who did so only before or after, respectively (paired vs unpaired participant responses). The Bayesian independent samples t-test was also used to compare all variable scores by gender, separately for before and after lockdown, including all collected responses (paired and unpaired). A score difference for all paired participant responses was then calculated for each variable as the score before lockdown subtracted from the score after lockdown. Using this method, we obtained a ‘mean difference’ that we again compared by gender using the Bayesian independent samples t-test. This mean difference for each variable was also entered into a Bayesian correlation with the GPA that included all paired participant responses that provided GPA, and the results were expressed using Pearson’s r. Finally, we again used the Bayesian independent samples t-test to compare scores between preclinical (years 1–3) and clinical (years 4–6) study years, separately for before and after lockdown, including all collected responses.

For all Bayesian tests used in the analysis, the calculations were made assuming a default prior distribution, results were expressed using mean and 95% credible interval (95% CrI) and BF10 >3.00 was used as a cut-off value for strong evidence towards alternative hypothesis compared with null hypothesis.49

Results

Four hundred and thirty-seven responses were collected in the first data collection point, and 235 in the second, after the exclusion of 40 responses, according to predefined exclusion criteria (figure 1). In total, we analysed 160 paired responses.

Figure 1

Flow chart showing the number of participants at each stage in the study. USSM, University of Split School of Medicine.

The overall response rate before lockdown was 77.2%, and it fell to 41.5% in the postlockdown time point, when we used an online survey.

When looking at demographic characteristics of the paired student responses, there was a significant female gender majority (table 1). The response rate and GPA were calculated across six study years. One exception was that the GPA was not calculated for the first study year because the students have no determined GPA until the year is complete and they complete all courses.

Table 1

Demographic characteristics and response rate of all paired participant responses

	No. (%) of students in the school year
	First (n=75)	Second (n=12)	Third (n=21)	Fourth (n=15)	Fifth (n=27)	Sixth (n=10)
Response rate (%)	72.1	11.7	22.1	17.2	31.8	10.9
Gender (no. (%))*
Male	28 (37.3)	4 (33.3)	4 (19.0)	4 (26.7)	6 (22.2)	1 (10.0)
Female	44 (58.7)	8 (66.7)	17 (81.0)	11 (73.3)	21 (77.8)	9 (90.0)
GPA (M, 95%CI)	–	4.16 (95% CI 3.86 to 4.47)	4.11 (95% CI 3.91 to 4.31)	3.97 (95% CI 3.71 to 4.22)	4.23 (95% CI 4.05 to 4.40)	4.25 (95% CI 3.96 to 4.54)

*Three responses missing.

GPA range: 2.00–5.00.

GPA, grade point average; M, mean.

For all collected before and after responses, participants’ satisfaction with their studies was overall ambivalent (ie, neither very positive or negative) but adequate. Study satisfaction had a median score of 3.38 (IQR=3.00–3.75) before lockdown and 3.63 (IQR=3.13–3.88) after lockdown. The mean (M) study satisfaction score was not significantly affected by the lockdown (table 2).

Table 2

Comparison of all responses before and after lockdown in the total sample

Variable (theoretical range)	Mean (95% credible interval)		BF10*
	Before lockdown (n=437)	After lockdown (n=235)
Study satisfaction (1.00–5.00)	3.38 (3.32 to 3.44)	3.49 (3.41 to 3.57)	1.04
OLBI
Disengagement (1.00–4.00)	2.22 (2.17 to 2.26)	2.15 (2.09 to 2.21)	0.38
Exhaustion (1.00–4.00)	2.73 (2.68 to 2.78)	2.63 (2.56 to 2.70)	1.21
CBI
Personal burnout (0.00–100.00)	55.76 (53.95 to 57.58)	52.44 (49.79 to 55.08)	0.75
Studies-related burnout (0.00–100.00)	52.68 (50.76 to 54.59)	49.76 (47.08 to 52.44)	0.41
Professor-related burnout (0.00–100.00)	33.23 (31.16 to 35.29)	33.83 (30.80 to 36.86)	0.10
Student-related burnout (0.00–100.00)	29.39 (27.07 to 31.70)	30.24 (27.09 to 33.40)	0.10
Patient-related burnout (0.00–100.00)†	24.54 (21.71 to 27.37)	22.22 (18.29 to 26.16)	0.22

*Bayesian independent samples t-test, significance cut-off BF10 >3.00.

†Calculated only for clinical study years (years 4–6). Responses analysed before lockdown: n=172, missing: n=5, total: n=177. After lockdown, n=72 analysed, missing: n=4, total: n=76.

CBI, Copenhagen Burnout Inventory; OLBI, Oldenburg Burnout Inventory.

No significant differences were found in all collected participant responses before and after lockdown across measures assessing burnout (table 2). Participants had high exhaustion and disengagement scores on the OLBI both before and after lockdown, as well as moderate personal burnout on the CBI both before and after lockdown. Students had moderate studies-related burnout scores before lockdown according to predetermined cut-off values, but no statistically significant difference was found between prelockdown and postlockdown measures (BF10=0.41).

We also found no significant difference in mean scores for any of the questionnaire scales within our survey when comparing the data before and after lockdown for paired participant responses (table 3). Paired participants had high exhaustion and disengagement scores on the OLBI before and after lockdown, as well as moderate personal burnout on the CBI both before and after lockdown.

Table 3

Comparison of paired responses of participants that filled out the survey before and after lockdown

Variable (no. paired responses, theoretical range)	Mean (95% credible interval)		BF10*
	Before lockdown	After lockdown
Study satisfaction (n=143, 1.00–5.00)	3.50 (3.41 to 3.60)	3.53 (3.44 to 3.63)	0.10
OLBI
Disengagement (n=144, 1.00–4.00)	2.13 (2.05 to 2.21)	2.12 (2.05 to 2.19)	0.10
Exhaustion (n=151, 1.00–4.00)	2.68 (2.59 to 2.79)	2.60 (2.52 to 2.69)	1.03
CBI
Personal burnout (n=149, 0.00–100.00)	53.71 (50.59 to 56.83)	50.95 (47.79 to 54.11)	0.82
Studies-related burnout (n=146, 0.00–100.00)	49.41 (46.27 to 52.55)	48.04 (44.86 to 51.23)	0.16
Professor-related burnout (n=153, 0.00–100.00)	28.01 (24.51 to 31.51)	31.43 (27.71 to 35.15)	0.58
Student-related burnout (n=145, 0.00–100.00)	27.58 (23.82 to 31.33)	30.17 (26.25 to 34.09)	0.29
Patient-related burnout (n=49, 0.00–100.00)†	24.92 (19.54 to 30.29)	22.28 (17.62 to 26.94)	0.25

*Bayesian paired samples t-test, significance cut-off BF10 >3.00.

†Calculated only for clinical study years (years 4–6).

CBI, Copenhagen Burnout Inventory; OLBI, Oldenburg Burnout Inventory.

We also compared study satisfaction and burnout in paired versus unpaired participants for both before and after lockdown to address possible bias due to postlockdown attrition (online supplemental appendix C). Before lockdown, we compared unpaired participants (n=277) who only completed the survey at that time point with paired participants (n=160) (online supplemental appendix table C.1). Unpaired participants had significantly lower study satisfaction than paired participants (M=3.31 (95% CrI 3.24 to 3.38) vs M=3.50 (95% CrI 3.41 to 3.60), BF10=13.92). Unpaired participants also scored significantly higher on the disengagement dimension than paired participants (M=2.27 (95% CrI 2.22 to 2.33) vs M=2.13 (95% CrI 2.05 to 2.20), BF10=12.57), as well as on the professor-related burnout dimension of the CBI (M=36.31 (95% CrI 33.76 to 38.87) vs M=28.01 (95% CrI 24.61 to 31.41), BF10=144.77). The same comparison of study satisfaction and burnout scores for paired (n=160) and unpaired responses (n=75) after lockdown showed no significant differences (online supplemental appendix table C.2).

We also compared scores by gender, separately for before and after lockdown, including all collected responses (online supplemental appendix C). Before lockdown, female students experienced significantly more personal burnout (M=58.67 (95% CrI 56.48 to 60.85) vs M=49.61 (95% CrI 46.62 to 52.59), BF10=3156.43), and also scored higher on the exhaustion dimension than male students (M=2.79 (95% CrI 2.73 to 2.85) vs M=2.63 (95% CrI 2.54 to 2.71), BF10=8.75) (online supplemental appendix table C.3). These differences were no longer observable after lockdown, when no significant difference was found in study satisfaction and burnout scores between genders (online supplemental appendix table C.4). We compared the mean difference between prelockdown and postlockdown for all variable scores for paired participants by gender and found no significant difference between male and female students (online supplemental appendix table C.5).

Finally, we compared scores between preclinical (years 1–3) and clinical (years 4–6) study years, separately for before and after lockdown, for all collected responses (online supplemental appendix C). Before lockdown, students in clinical years had higher disengagement scores (M=2.34 (95% CrI 2.27 to 2.41) vs M=2.14 (95% CrI 2.08 to 2.19), BF10=1185.99), higher professor-related burnout scores (M=40.75 (95% CrI 37.64 to 43.85) vs M=27.98 (95% CrI 25.38 to 30.58), BF10=6.70×106) and higher student-related burnout scores (M=34.26 (95% CrI 30.60 to 37.93) vs M=25.63 (95% CrI 22.71 to 28.55), BF10=65.92) than students in pre clinical years (online supplemental appendix table C.6). After lockdown, students in clinical years again had higher professor-related burnout scores (M=39.17 (95% CrI 33.11 to 45.22) vs M=27.67 (95% CrI 23.10 to 32.24), BF10=8.79), higher student-related burnout scores (M=40.25 (95% CrI 34.19 to 46.31) vs M=24.42 (95% CrI 19.88 to 28.96), BF10=277.35), but no significant difference was found between pre clinical or clinical students in disengagement scores or any other variable (online supplemental appendix table C.7).

We also entered GPA scores in a Bayesian correlation with all mean differences in variables for paired participants, but no findings were statistically significant (online supplemental appendix table C.8).

Discussion

We found no significant difference in burnout among medical students at the USSM before and after COVID-19 lockdown, both for overall responses and for paired participant responses. Burnout levels were generally low for both time points, except for moderate levels of exhaustion, disengagement and personal burnout that also persisted after lockdown. Participants that were lost to follow-up had higher disengagement and professor-related burnout at baseline. There was no significant difference between the participants who completed both surveys and those completing it after only lockdown. Female students had higher exhaustion and personal burnout scores before lockdown than male students but not after lockdown. Using paired participant data, the mean difference in scores (prelockdown vs postlockdown) were compared for study satisfaction and burnout by gender, but no significant differences were found. Before lockdown, students in clinical study years had higher disengagement and experienced more professor-related and student-related burnout than those in preclinical study years. After lockdown, these differences persisted, except for disengagement scores, which were no longer significantly different for these two groups. Academic success, measured as GPA, before lockdown did not correlate with any prelockdown to postlockdown changes in scores for paired participants. Study satisfaction scores remained overall consistent before and after lockdown and were neither very positive or negative. No difference was observed in study satisfaction before and after lockdown. However, participants lost to follow-up had a lower study satisfaction than those that completed both surveys.

Our results show that, even after dramatic changes to the education format due to lockdown, study satisfaction and burnout levels did not change among medical students. It is likely that a sudden cessation of clinical teaching did not have a negative effect on the amount of burnout experienced by students. A Croatian study on undergraduate university health sciences students showed that the students were satisfied with the exclusive e-learning initiated during the same April/May 2020 lockdown period in Croatia, were able to adjust to the change and were satisfied with their institution’s rapid and adequate adaptation to e-learning.50 In our study, we assessed study satisfaction as a variable to see if any changes in burnout would co-occur with changes in study satisfaction and found no difference prelockdown and postlockdown. Since the USSM allowed students to continue taking exams and immediately switched to e-learning,37 it is possible that students were not burnt-out more than usual in this period because their advancement in the academic year may not have been significantly compromised in the observed time frame, regardless of the lack of clinical teaching. The time period covered in this study was short and limited to the beginning of the pandemic, so it is possible that the students did not have time to become very distressed due to the cessation of clinical teaching for a relatively brief time period, something that is expected to be an ongoing challenge in the pandemic.23 One multicentre survey of medical students showed that a majority of them thought that restricting clinical teaching was appropriate, but a large number of them also felt that their education was significantly disrupted.51 In this initial period, students in our study were generally not affected in terms of burnout, most likely because they understood that lockdown and e-learning were necessary to prevent the spread of COVID-19. At the time of the first lockdown described here, there was no clear information and knowledge on the long-term impacts and course of the pandemic. The question remains how long disruptions to medical education can be tolerated without negatively impacting the students’ perception of their studies, beyond an initial pandemic period. At the USSM, face-to-face teaching resumed during the next academic year, but there was a partial switch to e-learning in October 2020 and a recommendation to only shorten the duration of clinical practicals.52 During the third wave of the pandemic in March 2021, there was a complete switch to e-learning, but clinical teaching continued in full and face to face.53 During the pandemic, the USSM has generally tried to keep as many of the classes as possible going face to face, with significant increases in COVID-19 transmission mandating switches to e-learning for shorter periods of time. Students were also offered SARS-CoV-2 vaccines,54 and face-to-face clinical practical teaching was never fully discontinued after the initial lockdown described in this study.

The higher scores in exhaustion and personal burnout found in female students before lockdown were no longer observable after lockdown. This difference between before and after lockdown could not be additionally confirmed, because we did not find any difference for paired participants. We presently cannot make any conclusions on possible gender differences, as the loss of participants to follow-up limits our interpretation. However, women seem to face specific challenges in the COVID-19 pandemic, and fear of infection significantly contributes to their stress and burnout.55 The cessation of clinical teaching, a measure to protect students and patients from infection with COVID-19, may have had a positive effect on female students’ burnout levels. Similarly, higher disengagement, professor-related and student-related burnout scores found in students in clinical study years remained unchanged, except for student-related burnout, which was comparable between clinical versus preclinical study years after lockdown. The characteristics of the clinical study year sample could suggest that more students in clinical years were lost to follow-up, as higher disengagement and professor-related burnout scores were also observed in participants lost to follow-up. Clinical study years also had smaller sample sizes than preclinical years before and after lockdown. Students lost to follow-up also had a lower study satisfaction, and thus it is possible that a proportion of students dissatisfied with the lack of clinical teaching did not fill out the survey after lockdown and that there are some differences that we have not been able to observe. Students’ prepandemic academic success did not seem to be related to how they perceived the lockdown changes, as it did not correlate with any score changes from before to after lockdown. We can conclude that it is more likely that individual and personal factors (such as gender, demographic information and living conditions) contribute more to the perception of lockdown than the level of academic success.

Our results and comparisons of burnout levels are consistent with the findings of Zis et al28 that showed no changes in burnout levels when comparing prepandemic and postpandemic data. However, our study had a larger sample, pairing of prequestionnaire and postquestionnaire results, and used two burnout instruments. The study by Bolatov et al found positive effects of e-learning on burnout but did not clearly explain how clinical teaching was or was not conducted in their observed setting.29 However, their study does indicate a potential positive effect of e-learning on burnout, and the existence of this effect could possibly account for some of our female participants experiencing a decrease in burnout, although these findings were inconclusive. Baseline, prepandemic burnout levels reported in our study were not as high as the prepandemic findings in 2019 meta-analysis,14 but this could be due to the previously observed existence of regional variations in burnout levels.56 Postlockdown burnout in our study was also lower than prepandemic findings,14 but these burnout levels have to be interpreted in the context of our lower prelockdown burnout levels and the fact that our institution did not stop teaching and made clear that the e-learning format was temporary. Unchanged study satisfaction scores before and after lockdown that we observed are in accordance with other observations of successful implementations of e-learning.30 31 50

Since the COVID-19 pandemic is expected to continue, well-structured longitudinal studies could provide better insight on rates of burnout in medical students over a longer period as the pandemic fluctuates in intensity. They could also answer the question what quantity of disruptions to teaching can be tolerated by medical students over time and how they perceive them over a longer time period, especially changes to clinical teaching. For example, Croatia has already experienced three waves of the pandemic. Using longitudinal study designs is generally encouraged to measure burnout in medical students, especially to assess any cause–effect relationships that may be present.15 Since it is possible that medical studies are inherently more stressful due to their structure, workload and the specific nature of clinical teaching, other study programmes can be used as a control when comparing burnout rates. To be able to more closely explore the cause of any pandemic-related burnout changes in medical students, future studies should specify in detail the type and amount of changes to the curriculum, for example, the impact on clinical teaching and whether e-learning, if present, was performed in an asynchronous or synchronous format. Studies that will assess how satisfied students are with e-learning would also be relevant, since the effectiveness of different e-learning modalities is still unexplored,33 even though it is one of the major changes the COVID-19 pandemic has brought to medical education.24 Future studies on burnout could also measure if students experience and anxiety and fear specifically related to COVID-19,57 especially if they attend clinical classes and have contact with patients, as this could impact their burnout levels. Focus should also be placed on comparing burnout effects between students that have different levels of contact with clinical teaching, as we have assessed in our study. Although a pre-COVID-19 meta-analysis showed no gender differences in medical student burnout,14 our findings open the possibility that differences could possibly exist in context of the pandemic and should also be explored. Future research on burnout in medical students could also explore how the COVID-19 pandemic affected female students, as there is a knowledge gap on women in healthcare during COVID-19.55 Any pandemic-related changes in burnout are probably highly dependent on the context and the study setting, especially since regional differences in burnout already existed before the pandemic.56

The main limitation with our study was that we experienced issues with the response rate of participants in the second data collection point. Since the second data collection was performed online, we had significantly fewer participants completing the survey. This significantly decreased the number of responses eligible for pairing, affecting the response rates and data analysis and limiting interpretation. We slightly increased the response rate by sending multiple reminders to students to fill out the survey. The first study year had the largest response rate (72.1%), most likely because reminders were also sent out to students as a part of their online curriculum in anatomy. We performed an additional comparison between paired and unpaired participants (who only completed the survey either before or after) to address the bias related to postlockdown attrition. The fact that there was no overall change in burnout before and after lockdown should be interpreted with caution, as it is also possible that we lost some burnt-out students during follow-up. Our analysis of participants lost to follow-up showed that the participants who only completed the survey before lockdown were significantly more burnt-out than the students who completed it before and after lockdown, with higher scores in disengagement, exhaustion and professor-related burnout. Our overall online response rate (41.5% for the second data collection point) is not unusual, as it is similar to that of other studies that performed online surveys of burnout in medical students and have encountered similar methodological issues.58 Another limitation is that our study covered a limited time period, which did not allow for long-term effects to be assessed. The strength of our study is its unique timing and design, as its first dataset was obtained prior to the COVID-19 pandemic and the second one after an almost 2 month-long full lockdown in the whole country. Thus, it allowed us to see in isolation the effects of an unprecedented lockdown measure on medical students.

Conclusions

Our study found no changes in overall burnout levels experienced by medical students before and after a full national lockdown due to the COVID-19 pandemic that caused a switch to complete e-learning. However, it is possible that female students’ burnout may have decreased due to the lockdown, although the findings were inconclusive. We found no difference in study satisfaction experienced by students in the prelockdown and the postlockdown periods, despite an abrupt change in teaching delivery and the cessation of clinical teaching. This study, although small and focused on the initial pandemic period and the first-ever lockdown measure, gives preliminary results and considerations for future research. More insight is needed on the short-term and long-term effects of the COVID-19 pandemic on medical students and their education. Further studies are needed to assess long-term and region-specific rates of burnout in medical students in the rapidly evolving context of the COVID-19 pandemic, as well as how disruptions to medical teaching will be tolerated long term.