Tutors' Perceptions of the Transition to Video and Simulated Patients in Pre-clinical Psychiatry Training.

OBJECTIVE: The use of virtual learning in psychiatric education has been required to address COVID-19-related challenges. Research regarding the implementation of virtual teaching environments and standardized patients for simulation remains limited. Here, educators' outcomes were evaluated following a transition from in-person teaching with "real" patients, to a standardized patient-based simulation in pre-clerkship psychiatric clinical skills teaching for medical students.
METHODS: The Integrated Clinical Experiences course at the University of Toronto is a pre-clerkship clinical skills curriculum for second-year medical students. Four psychiatric clinical skills sessions were transitioned from in-person teaching to virtual teaching environments with standardized patient-based simulation. Educators (tutors) were assigned to teach groups of four to seven medical students, with a total of 45 groups. Tutors were then asked to complete an online questionnaire, and data was analyzed by quantitative and qualitative means.
RESULTS: Of 30 tutors, 21 (75.0%) had previously taught the course for an average of 6.52 ± 6.85 years. Twenty-four of 30 (80%) tutors described their ease of virtual teaching as "extremely easy" or "moderately easy". Twenty-three of 30 (76.6%) were "extremely satisfied" or "moderately satisfied" with standardized patient-based simulation. Various advantages and disadvantages of the virtual teaching environment with standardized patient-based simulation were identified.
CONCLUSIONS: The transition to a virtual teaching environment utilizing standardized patients in a pre-clerkship simulation-based curriculum did not result in significant challenges that would limit educators' use of these teaching tools. Implementation of virtual teaching environments with standardized patients may thus serve to address challenges related to COVID-19 and resource limitations.

Human simulation is a powerful pedagogical tool for simultaneously teaching knowledge, skills, and attitudes in medicine [1]. Simulation in psychiatric education involves human actors or role players who interact with learners in a wide range of contexts and experiential learning environments [2]. Simulation offers the opportunity to improve exposure to complex mental illness presentations, particularly given the compounding social, cultural, and economic factors that may limit patient recruitment for teaching purposes [3].

Various challenges have been described in relation to the use of simulation in psychiatric education, including the ability of standardized patients (SPs) to fully portray the complexity of psychiatric illness, such as affective, behavioral, and cognitive elements [3]. Furthermore, questions regarding the relational authenticity established with the use of SPs have also been raised as a concern [4]. Moreover, a recent systematic review of psychiatry simulation-based training in undergraduate medical education (UME) identified a necessary role for virtual delivery of simulation as a promising area for future development [5].

Conversely, concerns have also been described in the literature in relation to the use of actual or “real” patients (RPs). RPs may be unable to engage in rehearsal training and may be prone to symptom misrepresentation [1, 6]. In addition, RPs may also experience negative consequences through interactions with inexperienced interviewers, such as re-traumatization [3]. The availability and ability to schedule RPs for training events are an additional challenge [6].

The coronavirus disease 2019 (COVID-19) global pandemic has resulted in a need to transition traditional learning settings and RP encounters to virtual teaching environments (VTE). Medical education has faced formidable challenges to introduce a rapid change in teaching models, with a focus upon virtual education to ensure limited social contacts and safety for learners and teachers alike [7-9]. The use of SPs in psychiatric medical education has only recently been investigated as an area of intense educational interest, with a particular focus on the impact that virtual learning has upon learners [8, 10]. There remains a significant lack of research related to the impacts of these changes upon educators in psychiatry. Furthermore, the COVID-19 pandemic has resulted in significant stressors upon medical educators, who face high levels of burnout due to COVID-19 challenges [11]. This underscores the importance of ensuring transitions to VTE are minimally stressful on educators [12, 13].

Here, we explore (1) the implications of transitioning to a VTE from traditional in-person teaching, combined with; (2) the transition from using RPs to SPs in a psychiatric interview skills simulation for pre-clinical medical students; and (3) the challenges and benefits experienced by educators.

Methods

The Integrated Clinical Experiences course (ICE2) is a longitudinal curriculum in pre-clerkship UME at the University of Toronto that teaches various clinical skills to second year medical students. For many students, ICE2 is the first exposure to clinical settings and completion of this curriculum is a mandatory requirement for attaining the Medical Doctorate degree. Prior to COVID-19, educators (herein referred to as tutors) were required to identify and recruit their own participants, or real patients (RPs) engage in interviews with the students. These interviews were conducted in-person and there were no elements of virtual learning or SPs in these sessions. The RPs consented to the process of being interviewed for educational purposes and were provided with financial reimbursement for their time and contributions to medical education.

As of March 2020, the UME curriculum at the University of Toronto transitioned to online virtual learning in response to the COVID-19 pandemic. Actors were subsequently hired to play the role of individuals with mental illness. A subcommittee was formed in the UME Psychiatry Committee to create scripts and background information for the actors to utilize in establishing their characters’ SPs. Eight thematic areas were identified: major depression/grief, anxiety, early-onset psychosis/schizophrenia, substance abuse, self-harm/borderline personality, childhood externalizing disorders (attention deficit disorder, oppositional defiance disorder, conduct disorder), and geriatric dementia/depression. Case synopses were developed by five volunteer staff physicians and six volunteer medical students. Feedback was solicited from the University of Toronto Standardized Patient Program (SPP), which hired actors for medical educational purposes. Training for the actors was conducted over September and October 2020 by the SPP.

All tutors for the psychiatry component of ICE2 were invited to participate in an online Continuing Professional Development session held in October 2020. Training was provided regarding the online teaching platform, Zoom™, and changes to the curriculum. Tutors were made aware of the themes for each SP role. An online training manual was also published and provided to all tutors.

The ICE2 sessions were held annually in the fall semester. Psychiatric interview techniques and mental status examinations were taught over the course of four half-days. Tutorial groups consisted of four to seven students and were led by a single tutor, who was a staff psychiatrist (psychiatrist tutor, PT) or psychiatry resident (resident tutor, RT). The first interview was completed by the tutor, who demonstrated interview and mental status examination skills. This was followed by each student completing their interview with the SP over the course of up to 50 min with additional time for feedback. Experiential learning theory informed this process [14].

Ethics approval was obtained from the University of Toronto Research Ethics Board (REB). Inclusion criteria consisted of active enrollment as a tutor for the psychiatry component of ICE2 (see Table 1). All 45 tutors, comprised of 40 PTs and 5 RTs, were invited to complete an online questionnaire where informed consent to participate was obtained. The questionnaire asked participants to rate their level of satisfaction and the level of engagement of learners using a 5-point scale (1 = terrible, 2 = poor, 3 = average, 4 = good, 5 = excellent). For questions pertaining to ease of use and familiarity or comfort with elements of virtual learning and SPs, a 7-point Likert scale was used to increase the fidelity of responses (1= extremely difficult, 2 = moderately difficult; 3 = slightly difficult, 4 = neither easy nor difficult; 5 = slightly easy; 6 = moderately easy; 7 = extremely easy). The questionnaire included open-ended questions to list the advantages and disadvantages of utilizing SPs versus RPs and of teaching students using a VTE. The investigators remained blinded to the identity of respondents to the questionnaire.

Table 1

Tutors’ ranking of ease and satisfaction in transitioning from in-person teaching to a virtual learning environment (N = 45). *1 = extremely difficult, 2 = moderately difficult; 3=slightly difficult, 4 = neither easy nor difficult; 5 = slightly easy; 6 = moderately easy; 7 = extremely easy. Response rate was 69%

Questionnaire item	Mean*	SD
Ease of joining sessions	6.40	0.86
Ease of teaching topics in psychiatry in VTE	6.06	1.14
Satisfaction with diversity of themes in cases	6.24	1.18
Comfort with themes in cases	5.90	1.08
Ability to solicit feedback from students	5.80	1.42
Ability to engage with students	6.03	1.18

Questionnaire items using 5-point Likert scales were scored 1 to 5, and for 7-point Likert scales, 1 to 7. Group means are reported in the form of mean ± standard deviation. Open-ended questionnaire responses were analyzed using thematic analysis [15]. Two authors (JH, CKM) independently reviewed the responses before coming to a consensus regarding the final themes generated from the data. The responses were again reviewed and sorted into themes relevant to the focus of the study and any association between themes was identified.

Results

Thirty-one responses were received (69% response rate) of which 30 were included in the analysis; one response only completed 22% and was excluded from the analysis. Of the 28 PTs that completed the survey, the mean duration of practice was 11.96 ± 8.83 years and 21 (75.0%) had previously taught the course for an average of 6.52 ± 6.85 years. Of the 2 RTs that completed the survey, the average year of training was 2.50 ± 0.71 and none had taught the course previously. The mean number of students assigned to each tutor’s group was 5.73 ± 0.74.

Tutors’ familiarity and comfort using ZoomTM in a VTE were 3.60 ± 1.10 prior to ICE2 sessions and 4.27 ± 0.64 after teaching ICE2. They ranked their ease and satisfaction in transitioning to a virtual learning environment (VTE) with a 7-point Likert scale (Table 1). Twenty-eight of 30 (93.3%) tutors found it was “extremely easy” or “moderately easy” for participants to join sessions on the virtual platform. When asked to rank their ease in teaching in a VTE, 24/30 (80.0%) tutors indicated that it was “extremely easy” or “moderately easy.” The response distribution was more varied regarding the levels of perceived student interest and engagement with the virtual sessions. Twenty-four of 30 (80.0%) tutors indicated that it was “extremely easy” or “moderately easy” to engage with students through a virtual platform. Overall, majority of tutors were satisfied with the diversity of themes in the cases, with 26/29 (89.7%) reporting that they were “extremely satisfied” or “moderately satisfied.”

Tutors’ familiarity and comfort in working with SPs were 4.20 ± 0.85 prior to the ICE2 VTE and 4.47 ± 0.57 after teaching in an ICE2 VTE. When asked to rank their satisfaction with SPs, 23/30 (76.6%) tutors were “extremely satisfied” or “moderately satisfied.” Additionally, 9/30 (30.0%) tutors perceived the SP’s genuineness was “excellent,” 14/30 (46.6%) rated it as “good,” 5/30 (16.6%) found it “average,” and 2/30 (6.6%) indicated it was “poor.” Finally, 15/30 (50.0%) tutors reported their comfort level was “excellent” when directly interviewing the SPs, while 15/30 (50.0%) reported a “good” comfort level.

Table 2 describes the commonly identified advantages and disadvantages of the use of VTE and SP-based simulation. Practice convenience, functions of Zoom™ that enriched student engagement, learning, and student comfort were highlighted as advantages of the VTE over traditional in-person learning, while technological challenges, reduced student engagement, and screen fatigue were listed as disadvantages. Regarding the use of SP-based simulation, advantages included a greater diversity of clinical scenarios, lower complexity of cases, helpful feedback, consistent experience for students, and student comfort. Disadvantages that emerged in the responses included over-simplification of cases, lack of diversity in mental status delivery, inability to replicate a genuine patient experience, and variable quality of acting by the SPs.

Table 2

Advantages and disadvantages of using virtual teaching environments and standardized patient-based simulation in clinical skills training

Key themes	Representative quotations
Advantages of using a virtual teaching environment
Convenience	“Everyone could attend easily.” “Convenience - no travel involved!”
Features of Zoom™ to liven up interactions	“Breakout rooms were a very easy and powerful function to facilitate interaction and genuine feedback in a safer small group or 1:1 space.”
Student comfort	“Students seemed less anxious when interviewing patients as compared to an in-person class.”
Easy to access and share learning materials	“Ability to pull up (electronic) resources and share them in the moment to support learning, rather than flag, search and share after sessions.”
Disadvantages of using a virtual teaching environment
Technical difficulties	“The only challenges we encountered were related to technology (e.g. speakers/earbuds failing, needing to reboot a computer).”
Trouble with engaging and interacting with students	“Harder to liven up the interactions with a virtual platform.”
Screen fatigue	“Students were tired and felt the effects of screen fatigue.”
Advantages of using standardized patient-based simulation
Diversity of clinical scenarios	“Diverse and clearly delineated clinical presentations.”
Lower complexity of cases	“Simplified case presentations were more appropriate for the students’ level of training.”
Feedback	“Learners received constructive and valuable feedback about their interview style.”
Consistent learning experience	“A more consistent experience for each learner/standardization of experience.”
Student comfort	“Students felt more comfortable and in taking risks throughout the interview.”
Disadvantages of using standardized patient-based simulation
Over-simplification of cases	“Some scenarios had a linear, unrealistic narrative that lacked details in the script and history.”
Lack of diversity in mental status	“Mental status findings were anemic and repetitive.”
Inability to replicate genuine patient experience	“Students miss out on seeing the ward milieu and connecting with someone with lived experience of mental illness.” “Hard to demonstrate psychosis and thought disorganizations. Feels forced.”
Variable quality of acting and providing feedback	“Variable quality of acting and ability to provide educational feedback.”

Discussion

Our results demonstrated that tutors were able to successfully transition from in-person teaching to VTE with SP-based simulation to teach clinical skills related to psychiatry. Most tutors described relative ease in joining the virtual sessions, adapting their teaching, and ability to interact with students. Furthermore, tutors reported overall satisfaction with the genuineness and diversity of cases presented by the SPs. Only a minority of tutors cited issues relating to technology, student engagement, variability in case complexity, and quality of SP acting.

There are many logistical advantages in using SPs in medical education that have been well described in the literature [2, 4]. In our study, tutors provided positive feedback regarding practical convenience of engaging with SPs in VTE. Tutors also utilized a range of ZoomTM features to support their teaching and facilitate interactions with students. Considering public health and safety concerns surrounding the COVID-19 pandemic, the virtual format of ICE2 sessions provided a reliable option for physician tutors and medical students to engage in this training program.

Using SP-based simulation, students were provided with a consistent and standardized learning experience while avoiding some challenges typically associated with RPs, such as variability in presentation and inconsistent disclosure regarding symptom description [16]. Students were also perceived to be more comfortable in conducting a psychiatric interview in the VTE compared to traditional in-person sessions. The complexity of standardized case presentations may have played a role in this, as these were carefully tailored to the students’ level of training. Furthermore, it was noted that students appeared to be more comfortable in taking risks during their interviews. Medical students often worry about saying the “wrong thing” when speaking with psychiatric patients and these simulated encounters removed the possibility of harming or upsetting a RP [4]. Challenges associated with the use of SPs were consistent with limitations previously reported in the literature, such as reduced realism and lived experiences of the individuals being interviewed [2, 4].

Due to COVID-19, a digital platform to teach clinical skills was required to be introduced in a relatively short period of time. Rapid design prototyping heavily informed our process and we successfully sustained education for undergraduate medical learners while innovating changes to the curriculum that allowed students to be exposed to a wider range of psychiatric clinical presentations [9, 17]. Within a VTE with SP-based simulation, junior medical students were able to safely take risks when conducting patient interviews. This format may be most suitable to providing pre-clerkship medical students with an introduction to clinical interview skills; it may also be possible to consider enhancing clinical training with this format. Further skills development with RPs in authentic clinical settings may be reserved for senior medical students, thereby allowing educators to focus upon this area given limited resources. Our findings and success with a transition from in-person teaching with RPs to VTE with SP-based simulation may be adaptable to other areas of medical education, particularly where patient interviews are a focus of teaching and evaluation. Furthermore, our project has demonstrated the possibility of implementing changes to a curriculum utilizing rapid design prototyping [10].

There were several limitations to our study. The response rate of our online questionnaire may be indicative of biased reporting, such that the experiences of tutors who had more positive experiences might be more represented in the results. Our relatively small sample size also limited our ability to conduct inferential statistics and generalize our findings to other settings and teaching environments. Going forward, future iterations of the ICE2 sessions will include revisions of case presentations to ensure consistent complexity and further training for SPs to exhibit a diverse range of mental statuses or conditions. Further evaluations regarding the concomitant transition from in-person teaching with RPs to VTE with SP-based simulation are required to better understand what impact this form of education may have upon learners and medical education.