From Trial to Implementation, Bringing Team-Based Learning Online-Duke-NUS Medical School's Response to the COVID-19 Pandemic.

In response to the increasing community spread of COVID-19 in the first quarter of 2020, the Singapore healthcare system moved to prioritize pandemic management over other non-essential services. The delivery of medical school curriculum and learning processes, which heavily involves clinical faculty from various healthcare institutions, had to be reconfigured and reengineered as healthcare staff reassignment and infection containment practices are escalated.

A few years earlier, in 2015, Duke-NUS had implemented an eLearning week, where face-to-face activities were conducted online in order to prepare for unforeseen disruptions to the curriculum. During the COVID-19 pandemic, we first put these practices to use when enhanced social distancing restrictions were mandated across the country in February 2020, and university guidelines required that all classes with 50 or more students be moved online [1]. As the pandemic escalated, Singapore imposed increased movement restrictions in April 2020 and all educational institutions were instructed to move to full home-based learning [2].

The transition to online learning may be smoother for a traditional lecture-based course, as educators can share pre-recorded lectures, or even deliver live lectures through video-conferencing platforms. Learning resources can be posted on their institution’s learning management system for their students to access remotely. However, a growing trend in higher education is towards the use of “flipped classrooms”, where the acquisition of information is done by the student prior to class, and the faculty use class time to reinforce the knowledge that students have acquired by providing opportunities for students to apply the knowledge through application exercises [3].

A disruption in face-to-face teaching is challenging for Duke-NUS, for two reasons. First, the foundational science concepts covered in the first academic year are revisited and built upon later during the clinical years. Disrupting the delivery of the foundational science content would have profound effects on subsequent parts of the curriculum. Second, Duke-NUS employs a complex learning pedagogy which emphasizes peer-to-peer learning through faculty-facilitated, student-led discussions [4]. Known as TeamLEAD (Learn, Engage, Apply and Develop), the primary mode of teaching for the first-year medical students at Duke-NUS is an adaptation of Team-Based Learning (TBL). The processes of TeamLEAD and TBL have been described elsewhere [5-7]. In brief, the three defining features of TeamLEAD and TBL are pre-class preparation, the readiness assurance tests (RATs), and the team application exercises [8]. In the TeamLEAD and TBL class formats, students first take the readiness assurance tests individually (individual readiness assurance test (IRAT)), then as a team (group readiness assurance test (GRAT)). It is after completion of the GRAT that TeamLEAD diverges from TBL. Known as the modified TeamLEAD Readiness Assurance Process (mTRAP), teams will now submit any unresolved queries to the whole class. The facilitator then assigns other teams to work on these queries, after which these queries are addressed in a faculty-facilitated, class-wide discussion. Thus, at Duke-NUS, TBL represents a flipped classroom process that requires face-to-face interaction and is heavily dependent on student attendance and participation.

Here, we detail the experiences learnt from the eLearning week in 2015 and the challenges faced in moving an entire module to online learning during the COVID-19 outbreak in 2020.

eLearning Week of 2015

In 2015, a week of the Body and Disease module was delivered in an online format to the class of 2018 during eLearning week. Body and Disease is the final module of the first-year medical curriculum, where students learn to integrate 5 different knowledge tracks: microbiology, immunology, pharmacology, pathology and clinical investigations [9]. The educational leadership designed an online TBL format de novo, adhering to the core principles of TBL [5]. During the eLearning week, students received 8 hours’ worth of video-recorded lectures as preassigned material. The online class session started in a similar manner to a face-to-face session, with students attempting the IRAT by logging into a test-taking platform, but from their homes. For the GRAT, the students logged onto a virtual chat room (chatzy.com) to carry out their discussions and keyed their answers into an in-house assessment tool that provided immediate feedback. The mTRAP process was carried out using a shared Google Document (Google LLC, Menlo Park, CA, USA). Students would post their questions on the shared document and the assigned team would submit their responses in the same shared document. A faculty member, a content expert on the topic, served as a facilitator. Members of the education administration team provided logistical and technical support for the class. Students did not use any video-conferencing platform for their discussions in 2015. Here, we present the impact of eLearning on students’ perception of TBL by analysing students’ feedback from the eLearning week, which occurred during Academic Week 7, and for the preceding week, Academic Week 6, which was carried out face-to-face. We analysed the students’ ratings on whether eLearning achieved the same desired TBL outcomes of developing verbal, written, collaboration skills as a face-to-face session. As a control, we analysed the standard evaluation feedback, which was administered after every session, from both academic weeks. Lastly, we compared the standard evaluation feedback from the eLearning week of AY2014/15 with a corresponding academic week from AY2013/14 within the Body and Disease module.

Lessons Learnt During eLearning Week in 2015

The students from the AY2014/15 cohort felt that eLearning mode of TBL was less effective in fostering their verbal, collaboration and leadership skills, compared with the face-to-face mode in Academic Week 6 (Table 1). A comparison of the feedback from those two academic weeks showed no difference in the perception on how eLearning versus face-to-face class sessions would affect their written skills (Table 1). This is perhaps unsurprising, as only a chat room utility and a shared online document were used to conduct intra- and inter-team communication during the eLearning week. We then compared the students’ standard evaluation feedback for the eLearning week and Academic Week 6 within AY2014/15 cohort. We found that moving the class sessions online did not significantly affect the students’ perceptions of their learning experience (Supplementary Table 1). The different study material assigned across the eLearning week and Academic Week 6, however, may play a role in how the students perceived the class session. As such, we compared the standard evaluation feedback from the AY2014/15 cohort’s eLearning week with the AY2013/14 cohort’s corresponding academic week (Supplementary Table 2), where students received a similar set of study material. No significant differences were found between each cohort’s perceptions of their learning experience for this particular academic week. To ensure that the lack of difference in students’ perception was not due to a cohort effect, we looked at the overall perceptions that students had for the face-to-face class sessions in the Body and Disease module. No significant differences were found between the cohorts of AY2014/15 and AY2013/14 either (Supplementary Table 2). Taken together, these findings suggested that the study material assigned during this academic week and cohort effect did not have an impact on the students’ perceptions of TBL outcomes.

Table 1

Comparison of student perceptions of desired TBL outcomes from the eLearning week versus the preceding face-to-face week within the same cohort in AY2014/15

TBL reaction outcomes*	AY2014/15
Overall, the instructional strategies allowed me the opportunity to develop my:	eLearning (Week 7)	Face-to-face (Week 6)	p value
Verbal communication skills	2.56 (0.99)	3.88 (0.73)	< 0.001
Written communication skills	3.34 (0.96)	3.17 (0.92)	0.307
Collaboration skills	3.36 (0.90)	4.17 (0.68)	< 0.001
Leadership skills	3.00 (0.88)	3.63 (0.79)	< 0.001

The TBL reaction outcomes for the eLearning week, which occurred during Academic Week 7 of the Body and Disease module, and for the preceding week that had face-to-face classes, during Academic Week 6 were compared. Students from the AY2014/15 cohort felt that the eLearning mode of instruction did not facilitate their development in the following TBL outcomes: verbal communication, collaboration and leadership skills

n = 50 for the eLearning week, and n = 52 for the face-to-face Week 6 in the cohort of AY2014/15

Data is presented as mean (standard deviation). All comparisons were analysed using Wilcoxon rank sums tests

*Evaluation items are scored on the following Likert scale: 1 = strongly disagree; 2 = disagree; 3 = neutral; 4 = agree; 5 = strongly agree

Given how moving the class online did not affect students’ perceptions of their learning experience, this suggests that the eLearning mode reached most of the same objectives as the face-to-face mode. Students were overall receptive of this innovation. However, students felt that the online instructional mode adopted during the eLearning week did not provide the opportunities to develop verbal communication skills, which they would have had in a face-to-face TeamLEAD session. This perception was likely due to the communication during the class being limited to text-based exchanges, rather than other factors. These findings indicated to the educational leadership that a sustainable online version of this module would need to provide avenues to develop verbal communication skills.

Transitioning TeamLEAD from Face-to-Face to Online in 2020

When the National University of Singapore mandated that all classes be moved online in February 2020, the educational leadership deployed the infrastructure for online learning that was in-place after the 2015 eLearning experience. While the 2015 eLearning experience was a week-long, in 2020 all classes were moved online indefinitely. To ensure successful implementation and continuity of TBL online, constant feedback was sought from students, faculty and the administrators through student feedback surveys and regular faculty debriefs.

Keeping in mind that students preferred to have verbal communication during online classes, in 2020, students were instructed to log on to a video-conferencing software for the entirety of the TeamLEAD session (Zoom.us, San Jose, CA, USA). This proved to be a useful medium to dispense instructions to the class and for facilitated discussions during mTRAP. Students chose their preferred platform for communicating with their teammates during GRAT. This may be an alternate video-conferencing program or instant messaging platforms on their mobile phone. Although the breakout room function in Zoom has proven useful for online TBL [10], we found manually assigning 82 students into 13 teams to be cumbersome. Additionally, using a separate communication platform allowed teams to be connected even during class-wide discussions. This helped us to recreate the “side-discussions” amongst teammates that were frequently seen in class. Similar to the eLearning experience in 2015, students used a university-sanctioned, password-protected, shared online document to submit their queries during the mTRAP. Teams assigned to answer those questions responded on the same document. To enhance communication, the assigned teams are instructed to present their answers through the video-conferencing tool. After which, the facilitators gave the class time for an open discussion. Students could take this time to interact with the faculty through video. This provision was crucial as, according to the social cognitive theory, individual consciousness can only be formed through communicative interactions [11].

In online TBL, the role of the facilitator took on a different form from face-to-face TBL [12-14]. In brief, the facilitator ensured that all the participants were muted throughout the session and that only one person speaks at any one time. Students were allowed to key question into the text-chat function of the video-conferencing platform. The facilitator had to be cognizant of these different prompts and channel them accordingly. The facilitator’s ability to manage the participants and all the key processes in a time-sensitive manner was key in creating a conducive online learning environment.

The Education Administrative Team Played a Crucial Role in Implementing Online TBL in 2020

The backbone of the online classes is the administrative team from the Office of Education at Duke-NUS that helms the logistical demands of TBL. This team of 7 administrators was pivotal in the transition and ensuring the continuity of the curriculum during the pandemic. Prior to the classes, the administrative team was responsible for maintaining the online learning resources and granting access for all participants. They were also responsible for conducting training sessions for students and faculty on how to use the various online platforms and helping users to resolve technological problems in real time [14]. For example, they worked with hospital-based faculty to overcome the poor Wi-Fi access in their institutions. Three to 5 administrators would attend each class session to provide technical support. This included monitoring the attendance of all participants, cueing students on when to begin each phase of the assessment and communicating the scores and item analysis of the RATs to the faculty. In addition, the administrative team was also responsible for troubleshooting any technological issues. They did so by remaining contactable via multiple communication modes during the class, through email, instant messaging or phone calls. Examples of contingencies that they had in place include sending soft copies of the resources through email and giving instructions or acting as a conduit for student-faculty discussions through the phone.

SWOT Analysis of the Online Learning Experience of 2020

Strengths

The greatest strength of this online TBL is the ability to provide an effective alternative TBL format to ensure continuity of learning amidst a global pandemic. Students were still able to engage in an interactive intellectual discourse with peers and faculty [15].

Additionally, in our online iteration of the mTRAP session, we used a combination of text and video. The real-time shared online document created a dynamic mTRAP session, due to the ability to edit and comment on responses during the discussion. This encouraged students to remain engaged throughout the discussion. Having the mTRAP hosted on a shared online document is an improvement from face-to-face sessions, as the information is now stored electronically for the entirety of the module, instead of being written on a whiteboard and erased at the end of the session. Having an electronic record available for an extended period also allows faculty to insert their comments retrospectively. This feature was important as clinical faculty, faced with increasing clinical demand during the pandemic, were not always available during the virtual class time.

The combined use of a shared online document and a video-conferencing platform for the facilitated class discussion during the mTRAP is a unique feature of our iteration of online TBL. Based on the cognitive theory of multimedia learning, the simultaneous use of text and video enhances the learning experience. This theory posits that the brain interprets data from multimedia sources in an organized and dynamic manner, resulting in the production of logical mental constructs [16]. With this, we provided students with a dynamic and interactive learning environment [17].

Weaknesses

Difficulty in getting access to a reliable internet network or internet-enabled devices posed a major obstacle for all participants. Perhaps to ensure that students can carry out online learning, the school should consider supporting their access to devices that would allow for the various processes of online learning. For the clinical faculty, they are in locations where internet separation was practised. Due to the Singapore Health Services being breached by a series of cyber-attacks in 2018 [18], all workstations in hospitals were disconnected from the internet. To overcome this obstacle, faculty participating from the hospitals did so from their personal devices.

Technological literacy ability of participants was another challenge that had to be overcome. Some participants were unfamiliar with the video-conferencing platform or unsure about how to access the online documents. We overcame this through the efforts of the administrative team who provided pre- and in-class technological support.

The strong dependence on the administrative team also presents as a weakness with our iteration of online TBL. Without dedicated staff to manage the learning process, the faculty would face an uphill task in teaching and running the logistics of the course. Institutions interested in implementing online TBL should train an administrative team to ensure the smooth running of the course or ensure that there is a co-host available to help with the administrative demands of the class.

This format of online learning may suit the teaching of theoretical knowledge well but may not be ideal for the teaching of practical skills, which is an important component of medical education. Faculty have also expressed concern about the inability to assess professionalism through online learning. Avenues to teach practical skills and assess professionalism online need to be developed to overcome these obstacles.

Opportunity

The potential for online TBL is far-reaching. While initially deployed to facilitate remote learning in a pandemic, this mode of learning can connect students with faculty who are not in the same geographical locality. This will be highly useful in teaching niche subjects where few experts are available.

There is also an opportunity to develop a single online platform that can house all the different features of online TBL. Currently, students switch between multiple platforms for each phase of the lesson. An all-encompassing video-conferencing software which contains features that allow the administration of the IRAT/GRAT, mTRAP and discussion processes will be ideal. Such features should include the ability to administer tests securely and store shared files.

Being able to carry out online TBL will give the institution the capability to conduct remote learning TBL courses. These can be credit-bearing courses which are open for any student regardless of location. Most major universities offer online courses, but these courses are predominantly lecture-based modules with limited collaborative learning capacity. Having this option will increase the repertoire of courses that reaches the unique learning objectives reached by TBL and appeal to the segment of learners who prefer TBL to traditional learning.

Threats

While preliminary observations (Table 1 and Supplementary Tables 1 and 2) suggest that learning can occur through this modified online TBL format, it is unclear whether this mode of learning is sustainable and/or achieves the same the educational objectives as face-to-face modules. Taking the tests online and unsupervised presents a tangible threat to the learning process. Students with integrity issues might not adhere to the university’s Honor Code. Testing higher-order thinking skills and developing secure exam-taking software may mitigate this shortcoming.

A worrying trend observed with the increased use of video-conferencing is hacking. Incidences of “Zoom Bombing” have been reported by multiple institutions globally, where hackers gained access and interrupted ongoing online classes [19]. If this trend escalates, confidential data might be compromised. In response to these intrusions, the university had suggested several guidelines to increase the security during video-conferencing sessions [20]. Video-conferencing software providers must maintain vigilance and continuously work to enhance security in order to prevent unauthorized interruptions.

Moving Forward

The data collected from the 2015 eLearning experience was instrumental in our response to the COVID-19 pandemic in 2020. The concerns raised by the students in 2015 guided the leadership in modifying the online format for the 2020 roll-out. In 2020, we found that we were able to reproduce the face-to-face TBL conditions online by utilizing the video-conferencing tool judiciously. Furthermore, students in 2020 generally appreciated the online TBL format of learning and found it easy to communicate with their peers. Given our observations, further studies comparing the impact of online and face-to-face TBL classes on student’s academic performance will need to be carried out. These studies would also need to ascertain if taking an online course for a prolonged period has an effect on student’s well-being and mental health. Overall, the preparedness of the educational leadership and the dynamic work ethic of the administrative team allowed for the successful deployment of online TBL.

(DOCX 26 kb)

(DOCX 22 kb)