Same Material, Different Formats: Comparing In-Person and Distance Learning in Undergraduate Medical Education.

In the winter 2019, the authors launched a new course for senior medical students entitled “Frontiers in Medicine: Behavioral Health and Neuromedicine.” The Frontiers in Medicine course was launched to provide post-clerkship medical students an opportunity to return to basic and clinical research topics after completing their clinical clerkships with a goal of emphasizing the importance of science to medical practice [1, 2]. This course aims to explore “frontier” areas and emphasize the value in integrating clinical experience with the study of basic and translational science [3, 4]. The Frontiers in Medicine course was created as part of the restructuring of the medical school curriculum that decreased pre-clerkship course work and added several elements to the fourth year post-clerkship period, consistent with national trends to modernize medical education [2]. The institution created seven sections of the Frontiers in Medicine course focused on different specialty topics and required that all senior medical students complete one section of the course prior to graduation.

The Behavioral Health and Neuromedicine section of the Frontiers in Medicine course integrates basic and clinical neuroscience and the neuroscience-related medical fields, focused on neurology and psychiatry and including faculty from neurosurgery, neuropsychology, and neuroradiology for students in the post-clerkship period. The authors structured the course by focusing on three major topics using a paradigm of molecules, networks, and structure: (1) the NMDA receptor in health and disease (molecule); (2) circuits, seizures, and neurostimulation (network); and (3) trauma and recovery (structure). The authors chose these topics due to the ability to cover innovative areas in both psychiatry and neurology (e.g., autoimmune encephalitis in the NMDA receptor section), areas of emerging research and clinical practice development (“frontiers”; [1]), and highlight diseases and treatment not emphasized elsewhere in the medical school curriculum. Each topic also allows for explicit connection between either scientific research (basic or clinical) and clinical practice [5]. A portion of the course is devoted to strengthening skills in critical review of the literature and evidence-based practice, including research methods and design, consistent with the skills necessary for neuroscience literacy [1], with the goal of fostering life-long learning [2].

In February of 2019 and 2020, the authors ran this course as originally designed, with all learning sessions held in person. In April 2020, the medical school leadership approached the course directors to repeat the course in a remote format so that medical students who were unable to complete clerkship rotations due to restrictions associated with the COVID-19 pandemic [6] could instead complete this graduation requirement. Thus, instead of taking the course in the second semester of their fourth year, this group of students completed the course as third year students, while clerkships were on hold.

In this report, the authors describe the differential experiences with the course with two different groups of students in two different formats: (a) in-person teaching with fourth-year medical students and (b) synchronous distance learning [7] with third-year medical students. The frame of the course was the same in both formats (course sessions held for approximately 4 h per day, 4 days per week), and the faculty and topics were unchanged, except in cases in which a particular member of the faculty was not able to teach due to other institutional responsibilities. In both versions of the course, class sessions were a mixture of traditional didactic lecture and active learning sessions utilizing a flipped classroom model to facilitate learning [4, 8, 9]. The course was graded as pass/fail as per institutional policy, with class attendance and completion of group and individual presentations necessary to pass the course. The data on student evaluation of the course presented here were obtained from the aggregated, de-identified post-course evaluations completed in the MedHub online evaluation repository. The institutional Office of Medical Education coordinates survey distribution and completion for the purpose of program evaluation. Following IRB analyst review, this project has been determined to be exempt from review by the institutional IRB as it is consistent with program evaluation and not human subjects research.

Experiences with In-Person Course

There were 38 fourth-year students enrolled in the in-person course in February 2020. Overall, the in-person course was well received by students, with 24 of 29 (83%) respondents to the post-course evaluation indicating that they would choose this course again or recommend it to another student (Table 1). Students had mixed reactions to the course material: some praised the topics covered, and others expressed concerns that the material covered in the class had been previously covered in the pre-clerkship and clerkship curriculum. For example, students noted that the lectures on concussion and neuropsychiatric evaluations were among their favorite course activities, while other students specifically noted these were among their least favorite activities. Interactive sessions were consistently well received, with several students naming an activity in which they built models of the NMDA receptor using Play-Doh as a favorite course activity. Students consistently praised the small group discussion sections. Several students specifically noted that course logistical and infrastructure issues (e.g., lack of student parking close to the classroom) were their least favorite aspects of the course.

Table 1

End of course ratings

Evaluation Question	4th year studentsIn person courseAverage (SD)	3rd year studentsRemote course Average (SD)
Number of respondents/Number of students enrolled in course	29/38 (76%)	15/20 (75%)
I feel confident that I can formulate clinical questions relevant to a patient scenario under most circumstances.	4.21 (0.5)	4.47 (0.5)
I feel confident that I can access literature relevant to answering clinical questions under most circumstances.	4.24 (0.8)	4.47 (0.5)
I feel confident that I can critically evaluate literature relevant to answering clinical questions.	4.10 (0.5)	4.33 (0.6)
I have the tools I need to stay abreast of latest developments in my field of interest and to integrate new knowledge into clinical practice	4.21 (0.7)	4.33 (0.6)
This course is relevant to my development as a physician.	4.00 (1.0)	4.47 (0.5)
I have a deeper understanding of the foundational science relevant to the clinical care of patients with brain-based disorders.	4.17 (0.8)	4.67 (0.5)
After taking this class, I am better able to educate patients and their families.	3.93 (1.1)	4.13 (0.5)
The faculty were knowledgeable in the subject material they taught.	4.55 (0.5)	4.67 (0.5)
The course covered an appropriate range of topics at an appropriate depth.	3.83 (1.2)	4.53 (0.5)
Reason that you would or would not recommend the course (selected)	• “I would recommend if they were into neurology, but barely has any behavioral health aspects in the course” • “Good exposure to critical thinking, research, clinical applicability” • “Practicing research review was valuable, neuro and psych diseases are high yield to almost all fields.” • “I think [course] did a relatively good job of making the lectures relevant to our clinical interests and helped re-enforce some important skills. Can’t help but feel that this course would be much more relevant and helpful significantly earlier in our education.”	• “Inspiring and learned a lot! If interested in neuroscience or psych, great course to hear from experts and see the overlap in the two fields” • “I learned a lot of useful skills in the class such as how to assess a paper while reading it” • “Great for looking at basic research and tying it to clinical practice. There is psych in every specialty, and it is useful for us all to have experience with this.” • “I found it both interesting and useful, even for someone not going into psychiatry or neurology”

Response to statements assessed on a 5 point Likert scale, from 1 = strongly disagree to 5 = strongly agree

Development of Distance Learning

The course directors faced several challenges in rapid creation of a distance learning course. The first challenge was the rapid time frame: the course directors were first contacted by the medical school leadership on March 21, 2020 to enquire about the feasibility of organizing a course to start on April 6, 2020. The course directors immediately reached out to the faculty who had taught in the course in February 2020, and almost all were willing and able to teach via distance learning. Two faculty members were unable to teach, one due to health reasons and one due to clinical responsibilities caring for patients with COVID-19. Other faculty—both internal and external to the institution—were able to take the place of the unavailable faculty. The inclusion of faculty external to the institution was made possible by the distance learning format, which meant that outside faculty could teach a class session without traveling to the institution.

The next challenge was that the course directors and faculty had very limited experience with distance learning. While delivery of a didactic lecture via a videoconference platform was straightforward, converting the small group activities to remote teaching posed a greater challenge. The lead course director (AMD) worked closely with the institutional Office of Medical Education to identify alterations to course activities. In particular, the course directors utilized the breakout room function of Zoom and the discussion boards in the learning management system to facilitate small group discussion. The course directors had not used either of these tools prior to leading the course.

An additional challenge was inherent to the course design: the medical school curriculum specifically designated that all Frontiers in Medicine courses were to take place during the second semester of the fourth year of medical school, after completion of all clerkships and acting internship rotations. The distance learning course, in contrast, was to be offered to third year medical students who were unable to complete clerkships due to institutional and regulatory limitations on medical student involvement in patient care in the spring of 2020 [6]. The course directors faced the challenge of deciding which third year students were appropriate for the course. Given that the course material is designed to build on the material taught in the psychiatry and neurology clerkships, the course directors limited enrollment to students who had completed the neurology and psychiatry clerkships earlier in the academic year, prior to the start of the pandemic. To facilitate classroom management and small group participation, particularly in the unfamiliar terrain of distance learning, the course directors limited enrollment to 20 students.

Experiences with the Remote Course

Fifteen of the twenty third-year students enrolled in the distance learning course completed the post-course evaluation; as in the in-person course, completion of the post-course evaluation was less than 100% as the institution does not require students to complete this evaluation. All respondents to the post-course evaluation indicated that they would choose the course again or recommend it to another student (Table 1). While some students noted difficulty in maintaining attention during lectures given remotely, other students reported that they felt the course was well balanced between lectures and interactive sessions carried out utilizing the breakout room function of Zoom. Of note, the session that required students to build a model of the NMDA receptor with Play-Doh was not as well received in the remote format, with one student expressing frustration about the need to purchase Play-Doh for this activity. Students praised the topics covered in the course as “very interesting” and “relevant to multiple fields,” with many students reporting that they “learned a lot” from the course. As with the in-person course, some individual lectures were noted to be the most favorite part of the course by some students and the least favorite by others. Logistical issues remained a significant student concern, with several students reporting that the course website was difficult to navigate and in need of re-organization.

Several faculty provided informal feedback on teaching in the remote format. Faculty consistently commented on the lack of real-time feedback during a teaching session, as most students chose to have their video feeds turned off, consistent with guidelines provided by the medical school for behavior during “large group” remote sessions. Thus, faculty were not able to see facial expressions and make adjustments in their teaching in real time. The lack of real-time feedback was also notable in small group sessions. In traditional, in-person teaching, it is straightforward for faculty facilitators to walk between small groups and listen to ongoing student discussions to identify likely topics for larger group discussion. This is possible, though much more difficult, in the virtual teaching format utilizing Zoom breakout rooms, as it requires a course director or other meeting host to move the faculty in and out of the virtual breakout rooms where students are engaged in discussion. Additionally, in the virtual format, facilitators are not able to use the ambient noise level or observation of student behavior as cues for ending small group discussion time.

Synthesis

An encouraging take-away from the course directors’ experiences is that a course with active and interactive learning sessions can be successfully adapted for distance learning without harm to the learning experience [7, 10], as has been previously demonstrated for different medical education settings, including didactic lectures among urology residents [11] and facilitation of family medicine clerkship problem-based learning small group sessions [12]. One previous report has suggested that certain students may learn more in the distance learning environment [13]. The mechanism for enhanced learning in the distance environment is not well understood, though it may be related to the comfort and convenience of learning at home. For example, the student concerns regarding the difficulty of finding parking close to the classroom for the in-person course was moot for the remote course. The authors also speculate that the remote course was more convenient for students with significant family care-giving responsibilities and potentially less anxiety-inducing for all students. The students who took the distance learning course gave the course similar, though numerically higher, ratings overall compared with the students who took the in-person course, both in terms of overall course acceptability (measured by percent of students who would recommend the course) and at the end of course rating of confidence in skills and knowledge consistent with the course objectives (Table 1). These ratings are consistent with the written comments provided by the students, in which the third year students more consistently described the class as beneficial to their education. These data were not statistically compared and must be interpreted cautiously due to the context in which the distance learning course occurred—third year students had been removed from clerkships, and many were anxious to complete any course available in order to progress in their medical education. Thus, any comparison of the course is not truly a comparison of in-person versus distance learning; instead, these metrics demonstrate the feasibility and acceptability of virtual learning in the context of a systemic upheaval due to a viral pandemic. The authors’ experience with both course formats demonstrates the feasibility of integrating cross-disciplinary neuroscience education in the latter half of medical school education, in keeping with previously reported incorporation of neuroscience education in the psychiatry clerkship [14] and consistent calls for enhanced integration of neuroscience into clinical disciplines [1, 4, 5].

Several factors likely contributed to successful implementation of distance learning. At the outset of the viral pandemic and transition to remote teaching, the authors’ institution created and made available to all faculty a series of webinars and guides on remote teaching, similar to initiatives described at other institutions [15]. These sessions served as high-level introductions to the principles of and tools for remote teaching available through the institution, including the breakout room and polling features of Zoom that the course directors utilized throughout the course. The course directors also utilized individual guidance provided by faculty in the Office of Medical Education on which of the available tools were best suited to adapt each aspect of the course. The discussion boards on the course website, which had not been utilized during the in-person course, were essential to facilitating discussion in the remote course. Additionally, use of the Zoom platform, which allows for audience polls and break out rooms for small group discussion, was essential for creating remote active learning sessions. The course director’s open attitude and willingness to try multiple new techniques to determine which were best suited for course activity was also important for course success.

The question of whether students should be encouraged (or required) to maintain their video feeds on during remote learning sessions remains highly discussed among the faculty. Guidance for remote learning provided by the medical school to all students stated “you are expected to be on camera for the entire time of the session” and noted that “it is acceptable to turn video off when not speaking” in large group sessions. The latter statement was in bold font, and no definition of “large group session” was provided, which may explain the authors’ observation that students typically had video feeds off. Given the course directors’ lack of experience with remote teaching, the course directors did not state an expectation regarding video feeds at the course outset. Based on the course directors’ experience, students typically had cameras turned off during didactic lectures, which was acceptable to some but not all faculty. The ability to have the video feed turned off also made it possible for students to sign in for sessions but then not remain present for session; this is a larger concern for lectures than for small group discussion sessions. In the future, the course directors will set clear expectations that students will be expected to maintain video feeds on during any live course activity, including didactic lectures. However, the course directors remain aware of growing concerns that attempting to require video feeds to be on does not respect medical student equity (e.g., financial expense of high bandwidth needed to maintain video feed may not be possible for some students) or privacy and will continue to work with the students and faculty to balance these needs.

Future Directions

The course directors are currently working to determine the format of the course for the next time it will be taught, February 2021. The course directors recognize the possibility that the COVID-19 pandemic will again require the course to be taught exclusively in a distance learning format. The course directors are hopeful, however, that in-person teaching and learning will be possible, which will allow for a blended course utilizing techniques of both in person and remote learning to enhance our goal of sharing excitement and engagement with the clinical neurosciences [16].