Impact of COVID-19 Pandemic on Undergraduate Psychiatry Teaching, Educational Environment, and Learning Processes.

PURPOSE: The COVID-19 pandemic has brought about significant changes in the way undergraduate medical education is conducted including psychiatry teaching. In view of sparse data on how the perception of the undergraduate educational environment (EE) is affected by the pandemic, we sought to compare the ratings of EE and learning processes (motivation, engagement, equipping, appreciation of psychiatry) between the 2020 (during pandemic) and 2019 (before pandemic) cohorts of students, and examined how the perception of the EE would influence overall experience within the psychiatry rotation. PATIENTS AND METHODS: The DREEM (Dundee Ready Education Environment Measure) was administered to fourth-year medical undergraduate students undergoing a psychiatry rotation in 2020 during the pandemic and these ratings were compared with those of the preceding cohort in 2019. Students also completed five additional items evaluating various learning processes and overall rating of the posting. Relationships between the DREEM scores, learning processes, and overall effectiveness of rotation were assessed using correlation and mediational analyses.
RESULTS: Altogether, 84 (response rate 93.3%) and 269 (response rate 89.7%) medical undergraduates participated in the study from 2020 and 2019 cohorts, respectively. The 2020 cohort had higher scores on the total DREEM (p = 0.032), academic self-perception DREEM subscale (p = 0.002), felt more engaged (p = 0.043) and better equipped (p = 0.003) compared with the 2019 cohort. Overall, DREEM and subdomain scores correlated significantly with specific learning processes. The direct effect of total DREEM and overall rating of psychiatry posting was significant in mediational analyses.
CONCLUSION: Our results highlighted that students' perception of the EE remained positive during the pandemic and impacts overall experience of the psychiatry posting. Undergraduate psychiatry training should continually seek to enhance the EE so as to optimize learning through better engagement and equipping of the learners even during the pandemic.

Introduction

The coronavirus disease 2019 (COVID-19) brought about an unprecedented disruption to various spheres of life, including that of undergraduate medical education and psychiatry teaching. The restriction on large group gatherings, strain on healthcare resources, as well as the increased risk that came with exposure to clinical settings1 meant that the traditional mode of medical education required rapid reshaping. These changes could have affected the perception of the education environment (EE) by undergraduate medical residents in both positive and negative ways. The perception of the EE by learners reflects their learning experience,2 and a negative perception could adversely affect cognitive and affective outcomes, academic achievement, and attitudes.3–6 This is a pertinent issue given that the infection control measures as a result of COVID-19 are likely to remain in place for a considerable length of time. There is a need to understand medical students’ perception of their EE in order to better evaluate the impact of changes due to COVID-19, and specific areas that can be enhanced so as to optimize the EE in terms of teaching, role autonomy, and support.7–9 There are few longitudinal studies currently available that can shed light on these changes, prompting the need for our study. One tool that can be used to assess learners’ perception of EE is the Dundee Ready Education Environment Measure (DREEM), which has been used extensively within undergraduate medical education.2,10

Changes to Undergraduate Psychiatry Teaching During COVID-19

The undergraduate medical students we surveyed for this study were undergoing their psychiatry posting at the Institute of Mental Health (IMH), the only local tertiary psychiatric hospital in Singapore. They experienced the following changes to their posting as a result of the pandemic, which largely mirrored the situation for all undergraduate medical students undergoing postings in the various healthcare institutions locally. A smaller cohort was assigned for posting to IMH, and the posting lasted for a longer duration than before. Strict infection control measures were also in place throughout the posting, providing assurance of their safety and allowing them to concentrate on learning. This was particularly important as they were assigned to a dedicated inpatient team for a full 3 weeks (compared with 1.5 weeks previously at our hospital before rotation to another site) which allowed them to clerk and interact with patients face to face. There were also aspects of the posting that the faculty managed to retain despite the pandemic. The faculty were still able to cover diverse topics through an online platform, with tutorials conducted daily for students in the afternoon. This allowed continuity of learning and dedicated time for students to consolidate their knowledge, including areas pertinent to the cases seen earlier in the day. Students were also able to undergo the posting in groups of three, ensuring that there was still a sense of community despite the social distancing measures in place.

Theories of Learning

Situated cognition and situated learning theories propose that thinking and learning, respectively, are situated in experience.11 Learning will result from the student’s interaction with their environment,11 and a learner’s identity and being is transformed through the process of learning.12 The pandemic has resulted in changes for both the EE as well as the learner, bringing about a state of disjuncture that could affect reflective learning in some learners.12 The learning process comprises several essential components, namely motivation,14 engagement,15,16 appreciation,17,18 and equipping.17,18 Motivation explains why students learn,13 and intrinsic motivation has been thought to be the desired type of motivation.14 The disjuncture brought about by the pandemic means that methods of learning and experiencing rotations in the past (such as physical teaching in larger groups) may no longer be feasible at least for the near future. This provokes students to bridge the gap by finding new methods of learning and adapting to the situation.12 Engagement represents how psychologically and cognitively invested students are in purposeful learning.15,16 Engaged students take keen interest in their work, persist despite challenges, and take pride in accomplishment.17 Motivation and engagement can be interrelated15 with greater motivation giving rise to better engagement. During the learning process, students construct meaning through learning experiences that could prompt intentional reflection, leading to better equipping and appreciation of the subject matter.18

Aims of Study

In this cross-sectional study, we had three aims in light of changes to the psychiatry posting and the need to evaluate the impact on learning of our undergraduate students. First, we sought to evaluate the perception of the EE of undergraduate medical students following completion of their psychiatry rotations during the ongoing COVID-19 pandemic at the IMH, the only tertiary psychiatric hospital in Singapore.

Second, we wanted to determine if perception of EE relates to the different learning processes, namely 1) motivation to learn about psychiatric disorders, 2) engagement with the discipline of psychiatry, 3) appreciation of relevant topics in psychiatry, and 4) how equipped students feel regarding their understanding of psychiatric conditions. We were also interested to understand if there were domains of learning that were more greatly affected by the pandemic than others.

Third, we compared the perception of EE as well as different learning processes between the cohort who had undergone psychiatry rotations during the COVID-19 (2020 cohort) with the previous cohort in 2019 before the COVID-19 pandemic.

Based on extant data, we hypothesized that a positive perception of the EE would be positively associated with various learning processes in our cohort of medical undergraduates undergoing their psychiatry rotations, and would influence overall experience within the psychiatry posting.

Materials and Methods

Participants

The study was conducted both in the 2020 cohort during the COVID-19 pandemic as well as 2019 cohort before the pandemic. Batches of fourth-year undergraduate medical students from the Yong Loo Lin School of Medicine at National University of Singapore who completed a Psychiatry rotation at IMH filled in the DREEM questionnaire, which explored their perception of the overall EE as well as its specific subdomains in relation to this psychiatry rotation. This was compared with the ratings by the previous cohort of undergraduate medical students who completed their psychiatry postings in 2019 before the pandemic. Fewer students completed their Psychiatry rotations at our hospital in 2020 as the whole cohort was divided and assigned to more training sites to adhere with infection control measures in the context of the current pandemic. Hence, the sample size for the 2020 cohort was smaller than the 2019 cohort. The study was approved by the Institutional Review Board of the National Healthcare Group, Singapore (NHG DSRB Ref: 2014/00422). No identifiers were collected as part of this study. Hence, it was exempted from the need for informed consent.

Measures

The DREEM questionnaire consists of 50 items rated on a five-point Likert scale ranging from 0 (strongly disagree) to 4 (strongly agree). Reverse coding was carried out for negatively worded items. Adding up all items would yield a total DREEM score upon a maximum possible score of 200. The DREEM can also be further broken down into five domains: 1) Student Perception of Learning (SPL), 2) Student Perception of Teachers (SPT), 3) Student Perception of the Atmosphere of the Learning Environment (SPA), 4) Student Academic Self-Perceptions (SASP), and 5) Student Social Self-Perceptions (SSSP).19,20 Higher scores suggest that learners had a more positive perception of the EE.21

In addition to the 50 standard DREEM items, we included five items with the aim of measuring various learning processes. These five items were related to 1) learner’s motivation to learn about psychiatric disorders, 2) their extent of engagement with the topic of psychiatry, 3) learner’s appreciation of psychiatry, 4) how equipped learners felt about managing psychiatric conditions, and 5) learner’s overall rating of the psychiatry rotation. Learners rated these items on a five-point Likert scale ranging from Strongly Disagree to Strongly Agree, while the item on Overall Rating of Session consisted of options ranging from Poor, Below Average, Average, Above average, to Excellent. A collection box was placed at the front of the classroom where participants could drop their completed anonymous forms into before leaving. This was done so that course instructors would not have any involvement in the collection of forms, reducing possible risk of coercion.

Data Analysis

All analyses were conducted using IBM SPSS 23 (IBM Corp, Armonk, NY). We first examined DREEM overall and subdomain scores. We then proceeded to look at differences between genders and cohorts on DREEM score and ratings on learning processes using independent t-tests. We also performed Pearson correlations of the DREEM subdomains with the four specific learning processes and learners’ overall rating of posting. Finally, we tested a mediation model using the PROCESS macro22 on relationship between DREEM scores, learning processes and overall rating of psychiatry posting.

Results

Descriptive Statistics

We obtained data from 84 students who participated in the study (out of 90 students), with a response rate of 93.3% for the cohort during the COVID-19 pandemic. This was compared with scores from 269 students out of 300 students (response rate 89.7%) before the COVID-19 pandemic in 2019. Overall DREEM score was found to be in the highly positive range19 (Table 1). All domain scores were in the top quartile, suggesting that students thought positively of the teaching, of their teachers, as well as the atmosphere of the EE.19 They were confident of their academic abilities and felt positive socially.19 The mean scores of items about learning and the overall rating of the posting are displayed in Table 2.

Table 1

Scores for Dundee Ready Education Environment Measure (DREEM) Domains

DREEM Domain	Mean (SD)/Maximum Score (2020 Cohort)	Percentage Score	Mean (SD)/Maximum Score (2019 Cohort)	Percentage Score
SPL	39.09 (5.31)/48	81.4	38.05 (4.68)/48	79.3
SPT	38.36 (4.37)/44	87.2	37.59 (4.25)/44	85.4
SASP**	25.11 (3.38)/32	78.5	23.84 (3.28)/32	74.5
SPA	39.69 (4.78)/48	82.7	38.51 (4.88)/48	80.2
SSSP	22.25 (3.18)/28	79.5	21.59 (2.95)/28	77.1
Total*	164.48 (18.76)/200	82.2	159.65 (17.41)/200	79.8

Notes: *p<0.05, **p<0.005 indicate significant differences in DREEM scores between the two cohorts.

Abbreviations: SPL, Student Perception of Learning; SPT, Student Perception of Teachers; SASP, Student Academic Self-Perception; SPA, Student Perception of the Atmosphere of the Learning Environment; SSSP, Student Social Self-Perception.

Table 2

Scores for Learning Domains and Overall Rating of Posting

		2020 Cohort Mean (SD)	2019 Cohort Mean (SD)
Items about learning	I am more motivated to learn psychiatric disorders	3.41 (0.61)	3.26 (0.65)
	I am more engaged in psychiatry*	3.46 (0.57)	3.29 (0.66)
	I better appreciate relevant psychiatric topics	3.52 (0.69)	3.45 (0.55)
	I am better equipped to manage psychiatric conditions**	3.48 (0.53)	3.27 (0.59)
Overall rating	Overall rating of the posting	3.52 (0.59)	3.46 (0.58)

Notes: *p<0.05, **p<0.005 indicate significant differences in ratings of learning process between the two cohorts.

Our results also showed that the total DREEM score and all DREEM domains were significantly correlated with items measuring learning processes and overall rating of the posting for the 2020 cohort (see Table 3).

Table 3

Pearson Correlations Among Dundee Ready Education Environment Measure (DREEM) Scores, Items on Learning Processes, and Overall Rating of Posting (2020 Cohort)

	SPL	SPT	SASP	SPA	SSSP	DREEM
More motivated to learn psychiatric disorders	0.349**	0.312**	0.347**	0.365**	0.376**	0.394**
More engaged with psychiatry	0.402**	0.281*	0.385**	0.385**	0.400**	0.418**
Better appreciation of relevant psychiatric topics	0.329**	0.278*	0.353**	0.325**	0.319**	0.362**
Better equipped at management of psychiatric conditions	0.316**	0.348**	0.395**	0.439**	0.429**	0.431**
Overall rating of posting	0.515**	0.507**	0.429**	0.597**	0.507**	0.583**

Notes: *Correlation is significant at the 0.05 level (2-tailed). **Correlation is significant at the 0.01 level (2-tailed).

Abbreviations: SPL, Student Perception of Learning; SPT, Student Perception of Teachers; SASP, Student Academic Self-Perception; SPA, Student Perception of the Atmosphere of the Learning Environment; SSSP, Student Social Self-Perception.

Comparison with 2019 Cohort Before Pandemic

When compared to the 2019 cohort, we found that the 2020 cohort of students had higher scores on total DREEM (t(344) = 2.153, p = 0.032) and the academic self-perception subscale of the DREEM (t(348) = 3.058, p = 0.002). In addition, the 2020 cohort of students felt more engaged (t(350) = 2.035, p = 0.043) and better equipped (t(350) = 2.971, p = 0.003) compared with the 2019 cohort of students before the pandemic (see Tables 1 and 2).

Mediation Analyses

We standardized the DREEM scores, scores on learning processes, and overall rating of the posting before entering it into a mediation model in order to obtain standardized coefficients. The learning process variable “Better equipped at management of psychiatric conditions” had to be left out of the model due to ceiling effects, leaving “Motivation”, “Engagement”, and “Appreciation” as mediators in the model. There was a direct effect of total DREEM scores on overall rating of the posting which were found in both cohorts of students. The results for the 2020 cohort are presented in Figure 1.

Figure 1

Effect of total DREEM score on overall rating of posting with learning outcomes as parallel mediators for 2020 cohort. Direct effects: Total DREEM score → Overall rating of posting (direct): β=0.491, p < 0.001 (significant). Indirect effects: Total DREEM score → Overall rating of posting (indirect through all mediators): β=0.092, CI=−0.011 to 0.244 (not significant). Total DREEM score → Overall rating of posting (indirect through Motivated): β=0.013, CI=−0.126 to 0.184 (not significant). Total DREEM score → Overall rating of posting (indirect through Engaged): β=0.149, CI=−0.019 to 0.311 (not significant). Total DREEM score → Overall rating of posting (indirect through appreciation): β=−0.070, CI=−0.159 to 0.101 (not significant).

Discussion

There were two main findings from this study. First, overall DREEM and subscale scores remained highly positive despite the posting changes brought about by the pandemic. The cohort during the COVID-19 pandemic scored significantly higher in academic self-perception subscale and total scores on DREEM scale, and felt more engaged and equipped compared with the 2019 cohort. Second, there were statistically significant correlations of the DREEM overall and subscale scores with items related to learning processes and overall rating of the posting. Also, the direct effect of total DREEM and overall rating of posting was significant in the mediational analyses, thus affirming our hypothesis.

The overall and academic self-perception DREEM scores in the 2020 cohort were higher compared with those obtained from the 2019 cohort that was not affected by the pandemic. Although this may be surprising given the extensive and tumultuous changes brought about by the pandemic, the new structure of the psychiatry rotation may have helped learners both academically and socially during these trying times. A previous review of relevant studies has shown that supportive peers and colleagues serve to reduce learner stress.23 The smaller group in the ward and longer duration of posting enabled students to build closer bonds with the inpatients and clinical team. Constructive and individualized feedback could be provided, which increases motivation24 and a sense of belonging for learners. This in turn has been found to increase student confidence.25 When combined with the daily tutorials and supervision format including online sessions provided by the faculty, this could have allowed them to develop greater competency and confidence in the topics being taught. In addition, the opportunity to clerk and interact with patients in-person for a longer period of posting promoted greater engagement and application of knowledge. This was also reflected in significantly higher scores reported for the learning process variables “More engaged with psychiatry” and “Better equipped to manage psychiatric conditions” in the 2020 cohort. Students also underwent the posting in groups of three which allowed them to maintain a sense of autonomy and yet social connectedness and relatedness. This was in contrast to most of the local population in other sectors of work that had been assigned to work from home and were isolated from their colleagues. Given that the elements of competence, autonomy and relatedness have been found to contribute to intrinsic motivation to learn, life satisfaction, and well-being within the Self Determination Theory,26,27 this could serve to explain the positive perceptions of the EE and overall rating of posting that we observed despite the pandemic situation. The significant improvement in perceptions of the EE in the context of a pandemic was also reported by a recent study involving German medical students.28

We found that DREEM scores were significantly correlated to learning processes. This suggests the presence of inter-related features between these variables. In addition, we sought to explore if the three components of learning (motivation, engagement, appreciation) were mediators of the relationship between the EE and students’ overall rating of the posting. We found that only the direct relationship between overall perception of EE and overall rating of posting was significant. We posit that the rating of the posting during the pandemic was more affected by the overall personal experience of the EE including perception of teaching, autonomy, level of social support and relatedness during the posting. The relational aspect of learning may have been especially salient during the pandemic, with earlier studies reporting that students who had a sense of belonging and mattering in their EEs also had higher scores on positive indicators (eg positive experiences), and lower scores on negative indicators (eg negative experiences).29,30

What are some practical implications from the findings of this study, which are applicable to other training programs? First, it highlights the importance of knowing and continually improving the overall EE while taking into account the challenges brought about by the pandemic. Emphasis on the relatedness aspect of the EE can help students to feel a greater sense of belonging in their posting, which could translate to more motivation, engagement, depth of learning and equipment, and emotional well-being.30 Second, the overall DREEM and specific subscale scores can be used as a periodic assessment tool to gather feedback on areas of EE that warrant further attention or continual efforts to improve the overall EE. Constant communication and discussion between faculty and students have been highlighted as crucial elements for adaptation to curricular changes during the pandemic.31 Third, the consistent high scores across all DREEM domains in spite of the pandemic suggest that disruptive changes to learning during the past year can also give rise to positive adaptations and enhancements in EE and pedagogy. To maximize existing clinical teaching and learning experiences, the faculty are encouraged to make their teaching in the new virtual platform as or even more engaging so as to allow learners to acquire the necessary competencies.32 Fourth, when entry to certain wards were not possible due to infection control measures, technology can be harnessed to allow students to join the virtual inter-professional ward rounds so that there is no disruption to their participation in clinical discussions, which can improve learner’s knowledge, skills and attitudes.33,34

There are several limitations. First, the items were self-reported and may be affected by other factors apart from the EE such as coping methods during times of change. Future studies may consider including additional measures such as coping strategies of the students to evaluate its influence on the perception of the EE. Second, this study is cross-sectional in nature and is unable to support a definite model of causality between the factors examined. Future studies may want to capture qualitative comments in addition to quantitative data to allow educators to get a better picture of how each domain of the EE has been influenced by the pandemic. Longitudinal follow-up will also be helpful in determining the impact of the pandemic on the EE and learning processes over time, as well as the interrelationships between these variables.

Conclusion

We found that the perception of the EE in a recent cohort of medical undergraduates who underwent psychiatry rotations during COVID-19 pandemic remained highly positive. This had a significant impact on rating of the overall posting, and the learners felt better engaged and equipped. In the context of relevant learning theories (such as situated learning, self-determination theory), undergraduate training programs in psychiatry ca n continue to adapt and enhance different aspects of the EE to optimize learning amongst our undergraduate learners even in the midst of the evolving pandemic. Recommendations include the harnessing of technology to facilitate more constant discussion and communication between faculty and students, minimize disruptions to inter-professional rounds even in the midst of infection control measures by using virtual platforms, enhancing interactivity during the use of the virtual platform in teaching, and periodically using a specific tool such as DREEM to ascertain the perception of learning environment amongst the learners.