Perceptions, satisfactions, and performance of undergraduate students during Covid-19 emergency remote teaching.

Due to the Covid-19 pandemic, the education system worldwide faced sudden and unforeseen challenges. Many academic institutions closed their doors, forcing both educators and students to transition to Emergency Remote Teaching (ERT) for the remainder of the semester. This transition eliminated hands-on experiences, increased workload, and altered curricula. However, these aspects, as well as students' perceptions, study habits, and performance in response to ERT remain poorly documented. This contribution describes changes in the curriculum of an undergraduate cadaver-based laboratory, and explores students' performance, self-perceived learning, and overall satisfaction during this educational crisis. Online content delivery for this course included both asynchronous instruction and synchronous discussion sessions. While formative assessments remained the same, online spotter examinations included short answer, multiple choice, multiple answer, ordering, and true and false questions. Despite examination grades improving 20% during ERT, students reported lower levels of learning, confidence, and engagement with the course materials when compared to the face-to-face portion of the class. The most prevalent challenges identified by students were those related to the loss of access to cadaver-based learning, including difficulty identifying and visualizing structures in three dimensions, and the loss of context and sensorial cues. Flexibility in taking examinations and learning the material at their own pace were recognized as positive outcomes of the ERT transition. While the resulting student perceptions and performances are unsurprising, they offer insight into the challenges of fostering a productive learning environment in a future threatened by epidemic outbreak and economic uncertainty.

INTRODUCTION

In March 2020, educators around the world faced one of the most abrupt and unforeseen challenges to education in modern history. The novel Coronavirus (Covid‐19), first reported by the World Health Organization (WHO, 2020) in January 2020, shuttered colleges and universities worldwide, sometimes with just hours of notice, forcing students and instructors alike to transition to an “Emergency Remote Teaching” (ERT; Hodges et al., 2020) strategy for the remainder of the semester (UNESCO, 2020). This strategy differs materially and conceptually from standard remote learning (Simonson et al., 2019), not only in the time invested in its development, but also in the learning experience and environment offered to students. A typical online course is planned and organized over a period of six to nine months, with layers of online infrastructure and additional resources built around it (Hodges et al., 2020). In contrast, instructors during the Covid‐19 pandemic were forced to adjust their courses to an online format on a significantly shorter timeline, and with limited instructional resources. While some institutions cancelled or postponed classes, others continued delivering course content either via synchronous or asynchronous lessons, or a combination of both. For example, in Australia and New Zealand, some universities offered their faculty a brief period (4–10 days) to make this transition, but others did not; and while some relied on pre‐recorded materials, others used available videos from YouTube (YouTube LLC., San Bruno, CA) and other websites (Pather et al., 2020).

Understandably, educators have been assessing both the immediate, short‐term impacts of Covid‐19 on education, as well as the long‐term ones that may persist well‐beyond the pandemic. Professional schools such as medical and dental schools, for instance, were forced to cancel many rotations, elective courses, and academic conferences, which can be both a present and future detriment to students (Ahmed et al., 2020; Ferrel & Ryan, 2020; Iyer et al., 2020). Rose (2020) pointed out that such changes may have differing effects on different cohorts of medical students, with those in their clerkship years potentially being impacted more severely. To increase student involvement and to mitigate the effects of the academic disruption, Harvard Medical School formed a “medical student response team”, which allowed students to assist in the Covid‐19 relief effort both clinically and educationally (Soled et al., 2020). In addition, the academic disruption may lead to increased inequality among students of different economic backgrounds (Burgess & Sievertsen, 2020). For example, socioeconomically disadvantaged physical education students in Brazil were unable to access online classes due to the lack of computer or Internet, and many were only able to attend classes using their cell phone (Pacheco et al., 2020). Finally, students graduating during the time of Covid‐19 will likely enter the job market during a global recession, experiencing job uncertainty and insecurity plagued by the economic crisis.

Anatomy students and instructors have faced unique challenges in response to the ERT transition (Evans et al., 2020; Franchi, 2020; Pather et al., 2020; Ravi, 2020). The online transition has reduced or eliminated hands‐on experiences, increased workload, and altered curricula. Harmon et al. (2021) quantified changes to anatomy courses at medical schools throughout the US and abroad, finding that in‐person lectures and the use of cadavers decreased dramatically after the outbreak of Covid‐19. Numerous authors have warned of the lasting effects of Covid‐19 on cadaver donation programs, as there is potential for a shortage of cadavers for future educational use (Bond & Franchi, 2020; Ooi & Ooi, 2020; Singal et al., 2020; Cheng et al., 2021; Lemos et al., 2021).

One of the most concerning consequences of Covid‐19 and the changes it has invoked to education has been its negative effects on mental health. Since March 2020, a great deal of attention has been afforded to understanding and documenting the psychological effects of the shift to ERT on undergraduate and graduate students (Li et al., 2020; Zhai & Du, 2020a). For example, the origin of Covid‐19 at Wuhan city of China has led to theories of its conception (Mohan & Nambiar, 2020) with collateral social stigmas and prejudice toward Asian peoples (Roberto et al., 2020). Nearly one‐in‐four Chinese undergraduate students experienced some form of anxiety regarding Covid‐19 (Cao et al., 2020). These fears were amplified among Chinese students living abroad, who faced additional concerns about discrimination, hate crimes, and being stranded from home (Zhai & Du, 2020b). In other cases, levels of reported anxiety were even higher. Balkhi et al. (2020) found that as much as 62.5% of residents of Karachi, Pakistan, experienced anxiety daily because of Covid‐19, with an even higher percentage among respondents 35 years of age or older. These findings were similar among Pakistani medical students, where 69% believed that Covid‐19 affected their social, mental, and psychological well‐being (Noreen et al., 2020). While these consequences are beyond the scope of this contribution, they highlight an important consideration for educators operating during these moments of crises.

This study sought to document the curricular changes adopted by an undergraduate human anatomy laboratory course during the Covid‐19 disruption, as well as to explore students' perceptions and performance during the change from face‐to‐face (F2F) instruction to ERT. As in many other schools, students at the University of Kansas (Lawrence, Kansas, USA; hereafter KU) attended F2F classes for approximately half of the first academic semester of 2020, and ERT for the remainder of the semester. To assess their perceptions of this experience, an optional Likert‐style survey was distributed once classes concluded. Many studies in anatomy education have used Likert‐style surveys to measure student and/or instructor preferences regarding online instruction (Petersson et al., 2009; Ogard, 2014; Mathiowetz et al., 2016; Srinivasan, 2020), and there are abundant resources discussing the best ways to make this transition (Trelease, 2002; Rizzolo et al., 2010; Sugand et al., 2010; Trelease, 2016; Evans et al., 2020; Gewin, 2020). Student feedback during such transitions is critical to understanding the challenges faced by students, as well as understanding effective study materials and course satisfaction (Gewin, 2020). Much less is known about abrupt shifts to online education imposed by crises, and thus the transition period is a blind spot in understanding student response to dramatic course and curriculum re‐structure. As the situation around Covid‐19 and its slew of variants (He, 2021) continues to evolve globally, and as many universities worldwide consider re‐opening their doors fully in the near future, understanding the effects of such changes is particularly pertinent.

The KU Anatomy laboratory course is atypical among undergraduate‐level classes because it utilizes prosected human cadavers as the main teaching material, which are regarded as premier resources for anatomy education (Ellis, 2001; Patel & Moxham, 2006; Estai & Bunt, 2016). The use of cadavers in anatomy education not only improves students' spatial reasoning and touch‐mediated perceptions, but also fosters attitudes of ethics, empathy, teamwork, and professionalism (Rizzolo & Stewart, 2006). Considering the myriad benefits of cadaver‐based instruction, it is likely that students more highly value the knowledge and experience gained during the F2F phase than during the online‐based ERT phase. Although computer‐based instruction is frequently lauded as a supplemental resource (Durosaro et al., 2008; McNulty et al., 2009; Tam et al., 2010; Jaffar, 2012; Green & Hughes, 2013; Green et al., 2014; Topping, 2014; Srinivasan, 2020), students often prefer cadaver‐based learning over computer‐based alternatives (Azer & Eizenberg, 2007; Petersson et al., 2009; Kerby et al., 2011; Davis et al., 2014; Ogard, 2014; Attardi et al., 2016). In addition, numerous studies have shown computer‐based learning to be less effective than traditional methods when used as a substitute in anatomy (Khot et al., 2013; Preece et al., 2013; Saltarelli et al., 2014; Mathiowetz et al., 2016), further suggesting that student performance and satisfaction may depreciate during the ERT phase. Other studies have reported that students experienced psychological distress after the onset of the Covid‐19 pandemic (Cao et al., 2020; Li et al., 2020; Zhai & Du, 2020a), affecting students' learning and performance (Vogel & Schwabe, 2016).

To this end, the present study sought to address two central research questions: (1) how did students perceive this new teaching modality compared to the norm? and (2) how did the transition to ERT affect students' grades and study habits? The authors hypothesized (1) lower levels of satisfaction among students during the ERT phase when compared with the F2F phase and (2) lower examination grades during the ERT phase due to an increase of stress related to the transition to an online environment and isolation due to Covid‐19 restrictions. Both of these negative expectations seem especially likely considering that the online learning experiences and environment during the ERT phase were undoubtedly different from those of a fully‐formed, well‐planned online anatomy course (Wright, 2012).

MATERIAL AND METHODS

Synopsis of examined course

The examined undergraduate course (Human anatomy observation laboratory) is a 200‐level, two‐credit‐hour laboratory course that is separate from the lecture. It utilizes seven prosected human cadavers and follows a regional approach. The course is divided into five content units: (1) introduction, back, and central nervous system; (2) upper limb and pectoral girdle; (3) lower limb and pelvic girdle; (4) thorax and abdomen; and (5) head and neck. In addition to cadavers, which are dissected by junior and senior students as part of a 400‐level biology course, the laboratory incorporates several ex situ prosections (i.e., prosected body parts removed from the cadavers) in all content units (Mattingly et al., 2021). Learning is complemented with the use of plastic models and multimedia sources (for example, 75 labeled pictures and 43 video walkthroughs of content) presented through Blackboard Learn learning management system (Blackboard® Inc., Washington, DC).

To avoid confusion with the terms spring, summer, and fall between the Northern and Southern Hemispheres, and to facilitate comparisons with other institutions, the academic semesters in the spring (January–May), summer (June–July), and fall (August–December) are hereafter referred as to Semester I, II, and III, respectively. There is no semester IV (winter) offered for this course. Nine laboratory sections are offered during Semesters I and III, each consisting of a maximum of 25 students and meeting for two two‐hour sessions per week. A single laboratory section is offered in Semester II. A graduate teaching assistant (GTA–either a master's or Ph.D. student) and three undergraduate teaching assistants (UTA) instruct each laboratory section. At the beginning of each class, the GTA provides brief instructions, and then students rotate among four or five teaching stations, which are each facilitated by one instructor (UTA or GTA). About four students work with each cadaver at a given time, and they spend approximately 20 minutes per teaching station. Throughout the semester (Semesters I or III), 23 regular laboratory sessions are scheduled, with five of them devoted to reviewing material prior to an examination using a mock test. In addition, 17 weekend review sessions are available to students, each two hours long, led by one GTA and three or four UTAs. Thus, students have access to the laboratory about 5.3 hours per week.

Student learning is assessed during each content unit through practical summative assessments, two quizzes and one examination, all in short‐answer format and in the form of spotters (Inuwa et al., 2011). Quizzes consist of 10 questions each and are worth 15% of the course grade, while examinations consist of 60 questions, are timed (approximately 54 seconds per question), and are worth 75% of the final grade. Quizzes and examinations include questions on both in situ and ex situ prosections, which students are allowed to handle, as well as on plastic models. In addition, students must complete 19 formative assessments (three or four per content unit) delivered through Blackboard. These assessments are due on the day of each unit examination and together are worth 5% of the course grade. Each formative assessment consists of 10 multiple‐choice questions, which students can take multiple times, with only the highest grade used in the final grade calculation. Participation in the laboratory accounts for the remaining 5% of the final grade. Students use a laboratory manual developed by the director of the course (VHG). Other information regarding this course is available in Sparacino et al. (2018).

Administration of post‐course survey

Student perceptions and satisfactions on learning outcomes during the online transition were assessed by administering a post‐course survey. The survey intended to document students' perceived changes in knowledge, confidence, satisfaction, and preparedness for their future career because of ERT. This survey was designed by the authors, created using Qualtrics© (Qualtrics, Provo, UT), pretested by two UTAs, and distributed via email to all students enrolled in the course. The survey was available from April 30 to May 7, 2020, and students were offered extra credit for its completion. It consisted of 26 questions that covered demographic aspects, student study practices during both F2F and ERT phases, and students' attitudes about the course change (Appendix 1). Using multiple choice, multiple answer, and short answers, questions 1–8 gathered demographic data and students' experience taking the course as well as online courses. Questions 9–15 assessed students' study practices on both F2F and ERT phases through multiple choice and multiple answer questions. Using a seven‐point Likert scale (1 = strongly disagree, 2 = disagree, 3 = somewhat disagree, 4 = neither agree nor disagree, 5 = somewhat agree, 6 = agree, and 7 = strongly agree), questions 16–24 assessed students' overall levels of confidence in their ability to learn anatomy, the effectiveness of the learning materials, and their satisfaction with the course before and after the online transition. The last two questions, 25 and 26, were open‐ended and sought to understand students' challenges and benefits of the ERT. For the sake of clarity, the survey questions referred to the F2F and ERT phases as “pre‐” and “post‐Covid‐19” or as “in‐person” and “online”.

The survey received 219 responses, 14 of which were incomplete and discarded. An additional 22 (10%) duplicate responses were received, wherein the same student completed the survey more than once. This was due to a software‐user error, in which the survey was emailed to the students on three separate occasions, inadvertently allowing them to complete the survey three times. However, these duplicates were identified and removed from the total responses. Hence, 183 out of 197 students enrolled in the course completed the survey.

Student performance

To explore changes in students' performance due to the ERT transition, the grades from the summative examinations of each content unit were compared during the F2F phase (examinations 1 and 2) and ERT phase (examinations 3–5). In addition, the grades from these examinations were compared with those of Semesters I–III of 2018 and 2019, as well as all semesters after the Covid‐19 disruption (Semesters II and III of 2020, and Semester I of 2021). Other metrics for student performance, namely the ten in‐class quizzes, online homework, and participation were excluded because they were either unsuitable factors for analysis or remained stagnant throughout the transition of the course (e.g., online homework material).

Statistical analyses

Unless indicated otherwise, graphical and statistical analyses were performed using R computing software, version 1.3.1093 (R Foundation for Statistical Computing, Vienna, Austria). Categorical variables are expressed as counts and percentages while continuous variables as means and standard deviations. To assess the validity and reliability of the survey items using Likert scales, Kendall's Tau b coefficient and Cronbach's alpha were calculated for questions related to the students' experiences during the ERT (Q16, 17, and 20) and F2F phases (Q18, 19, and 21–24).

To identify the potential main challenges and benefits of the ERT experience, as well as their relative importance among students, responses from open‐ended Questions 25 and 26 were analyzed using qualitative content analysis. Qualitative data, i.e., student written responses, were first classified based on characteristic themes (Kuckartz, 2019). Then, the responses from each student were reanalyzed, coded, and categorized in accordance with these themes and listed in the same order as they were mentioned. This process was done manually given the small sample size of this study. The dataset was then analyzed using Anthropac software, version 4.97, (Analytic Technologies, Lexington, KY) to calculate a Smith's Salience Index (S), which ranks responses based on the frequency and order of mentioned (Borgatti, 1996). Resulting S values are bounded (between 0 and 1), with higher values indicating higher consensus and importance among participant's responses. To visualize these responses, a word cloud was created using the R Word Cloud statistical package, version 2.5, (R Foundation for Statistical Computing, Vienna, Austria).

To assess for differences among examination grades during Semester I of 2020, a Generalized Linear Mixed Effect Model with normal distribution was used. Examination served as a fixed factor and laboratory section and student as random factors. A Generalized Linear Model with normal distribution was used to assess the effect of content unit examination, semester, and year on the grades. In this model, examination, semester, and year were used as factors. These models were implemented using the lm4 and glmmTMB packages in R (Bates et al., 2015; Brooks et al., 2017) and assessed the significance of fixed effects using a Type II Wald χ 2 test with the car package (Fox & Weisberg, 2019). The lsmeans package (Lenht, 2016) was used to conduct multiple pairwise comparisons with Bonferroni adjustment. Finally, a goodness‐of‐fit test was used to compare the weekly number of hours studying between the F2F and ERT phases and the proportion of students within each final course letter grade between the spring 2020 semester and the previous six semesters combined. Average values are given with standard error and sample size.

RESULTS

Course demographics and online experience

Nearly half of the 197 students enrolled in the course were sophomores in the pre‐nursing program. Exercise science, applied behavioral science, and biology were the next most‐prevalent majors. The vast majority of students were females (83% female; 16% male; 1% non‐binary) between 18 and 26 years of age. Most (90%) students self‐identified as white, non‐Hispanic or Latino. Similarly, a greater number of students (91%) were enrolled in the course for the first time, and many (74%) had various degrees of experience with online courses (Table 1).

TABLE 1

Characteristics of students enrolled in the undergraduate human anatomy laboratory course at The University of Kansas

Characteristics	Frequency a n (%)
Total number of students	183 (100.0)
Gender
Female	149 (81.4)
Male	32 (17.5)
Non‐binary	2 (1.1)
College level
First‐year (freshman)	60 (32.8)
Second‐year (sophomore)	87 (47.5)
Third‐year (junior)	21 (11.5)
Fourth‐year (senior)	15 (8.2)
Major of study
Pre‐nursing	91 (50.0)
Community health	2 (1.0)
Applied behavioral science	5 (2.7)
Exercise science	42 (23.0)
Biology	19 (10.4)
Others	24 (13.1)
Ethnicity
Hispanic or Latino	16 (8.75)
Not Hispanic or Latino	164 (89.6)
Prefer not to respond	3 (1.65)
Race
Native Hawaiian or Pacific Islander	1 (0.05)
American Indian/Alaska Native	2 (1.1)
Asian	15 (8.2)
Black or African American	10 (5.5)
Hispanic or Latino	7 (3.8)
White	136 (74.3)
Multiracial	12 (6.6)
Number of online classes taken
None	46 (25.1)
One	49 (26.8)
Two	39 (21.3)
Three	24 (13.1)
More than three	25 (13.7)

Data were pooled across nine laboratory sections during the spring semester of 2020.

Curricular modifications

Beginning the week of March 23 (week 10 of Semester I of 2020), following a one‐week suspension due to Covid‐19, all in‐person courses at the University of Kansas transitioned to an ERT format for social‐distancing purposes. This entailed shifting all current instructional materials to Blackboard. Fortunately, some resources, such as images, videos, Microsoft PowerPoint® presentations, version 2105 (Microsoft Corp., WA), and online practice quizzes, were already available on Blackboard prior to the start of the Covid‐19 pandemic, thereby reducing the difficulty of the transition. To maintain the “near peer” teaching model (Evans & Cuffe, 2009), the UTA's created an additional 45 instructional videos (each ranging from 1:43–13:26 minutes long) pertaining to Units 3, 4, and 5 that were either tested only (Unit 3) or entirely covered (Units 4 and 5) during the ERT phase. These videos were created with the Kaltura capture application (Kaltura Inc., New York, USA) and were subsequently posted to Blackboard. Videos included previously developed photographs and illustrations, as well as images from commercial software used in other courses (Anatomy and Physiology Revealed 3.0; Schneider et al., 2011), or freely available on the Internet (BioDigital®, New York, NY). These videos were delivered to students asynchronously. Synchronous online discussion and office hour sessions were offered to students during normal laboratory times. These sessions were optional and were led by either a GTA or UTA via Zoom video communication platforms (Zoom Video Communications, Inc., San Jose, CA). Supplemental instruction was consistent throughout the semester and was also offered synchronously online.

Assessments for Units 3–5 were delivered throughout Blackboard. Due to the timing of the shift to ERT, the content delivery and assessment for Unit 3 was a hybrid; most of the material for this unit was taught F2F (utilizing cadavers), but the summative assessments were conducted online. Overall, the number and type of summative assessments remained the same during the ERT phase (two quizzes and one examination per unit). However, the questions therein differed in both type (identification or conceptual) and format (short answer, multiple choice, etc.) from those contained in the F2F assessments. This primarily involved the addition of multiple‐choice questions during online assessments (Table 2). Because the students learned the Unit 3 content mostly during the F2F phase, the online examination for this unit consisted only of short answer questions, thereby resembling the traditional practical spot test of the course. In contrast, examinations for Units 4 and 5 contained a mixture of multiple choice, multiple answer, ordering, and true or false questions. The unit quizzes were merged into one larger quiz, which also served as a mock examination. Each mock examination consisted of 40 questions and students were allowed multiple attempts, except for that of Unit 3 in which students were allowed only two attempts. In all mock examinations, students had a time limit of two hours (3 minutes per question). The formative assessments, which were already online through Blackboard, remained the same. Each examination was available to the students for two to three days and they had three hours to complete it (3 minutes per question). Two attempts were permitted, with only the highest grade recorded. To discourage academic dishonesty, questions were taken randomly from question pools, and these presented to students one at a time and in different order. The extended time per question in comparison to the F2F examination was chosen to minimize internet issues, as some students reported low internet connections that prevented them to see images.

TABLE 2

Type of questions and format used during summative examinations

Descriptor	Examinations a
	Unit 1 n (%)	Unit 2 n (%)	Unit 3 n (%)	Unit 4 n (%)	Unit 5 n (%)
Total number of questions	60	60	79 b	121 b	115 b
Type of question
Identification c	46 (76.7)	44 (73.3)	69 (87.3)	86 (71.1)	78 (67.8)
Conceptual d	14 (23.3)	16 (26.7)	10 (12.7)	35 (28.9)	37 (32.2)
Format
Short answer	60 (100.0)	60 (100.0)	79 (100.0)	0 (0.0)	0 (0.0)
Multiple choice	–	–	–	112 (92.6)	112 (97.4)
Multiple answers	–	–	–	4 (3.3)	2 (1.7)
Ordering	–	–	–	1 (0.8)	0 (0.0)
True/False	–	–	–	4 (3.3)	1 (0.9)
Other
Illustrations	–	–	78 (98.7)	97 (80.2)	98 (85.2)

Summative examinations for Units 1 and 2 delivered during the face‐to‐face phase while remaining units during the emergency remote teaching phase.

Total number of questions created in the question pools.

Questions related to structures that students are required to identify.

Questions that students are required to provide an explanation for, and not necessarily to visually identify. For example, functions, origins and insertions of muscles, and knowledge on blood flow through the heart.

The course continued fully online during the remaining semesters of 2020 and Semester I of 2021, maintaining the same structure as indicated above, except for Semester II of 2020 in which eight summative examinations were delivered (one per week) instead of five. In addition, class meetings were synchronous via Microsoft Teams (Microsoft Corp., Redmond, WA), students were required to submit a short video per each content unit using VoiceThread® software, version 9 (VoiceThread LLC., Durham, NC), and both formative and summative examinations were delivered throughout McGraw‐Hill Connect® using Anatomy and Physiology Revealed 3.0. Finally, Proctorio (Proctorio Inc., Scottsdale, AZ), a software to monitor student activity during an online examination, was implemented for all summative examinations in Semester I of 2021 when it became available throughout McGraw‐Hill Connect®.

Validity and reliability of survey items using Likert scales

All survey items using Likert scales were significantly correlated (P < 0.01) to one another according to Kendall's Tau b coefficients. Such a relationship was positive between pairs of questions related to either the ERT phase (Q16, 17, and 20) or F2F phase (Q18, 19, and 21–24), and negative between any pair of items within these two sets of questions (Appendix 2). Cronbach's alpha was α = 0.801 for questions related to the ERT phase and α = 0.864 for those related the F2F phase. Thus, these results indicate acceptable validity and reliability of the survey items.

Study habits and perceived engagement of learning materials

The transition to the online format significantly shifted the time invested by the students learning the course material (Chi‐square test, χ 2 = 1187.3, df = 4, P < 0.001). Most students spent more time studying for the summative examinations during the F2F phase than during the ERT phase. For example, many students reported that during the F2F phase they invested more than six hours each week to prepare for the class, whereas during the ERT phase most students spent fewer than six hours (Figure 1).

FIGURE 1

Self‐reported time invested by undergraduate anatomy students learning the course material during the face‐to‐face (F2F) and emergency remote teaching (ERT) phases. Data pooled across nine laboratory sections (n = 183)

The students' reported reliance on class materials and other resources was largely similar between the F2F and ERT phases (Table 3), though some resources were not available to students during both phases. For example, online discussion sessions were only offered to students during the ERT phase, and bone models were only available to students during the F2F phase. The only additional study material that either did not transition over or was not offered during the F2F was Zoom office hours, though these were rarely used by students (most students (87.4%) never attended these optional, synchronous discussion sessions). Some study habits, such as supplemental instruction sessions and study groups, did not transition well, with high attendance during the F2F phase, but far less use during ERT phase (Table 3).

TABLE 3

Reported usage of class materials and learning strategies

Resources	Teaching phase a
	F2F n (%)	ERT n (%)
Laboratory manual	177 (96.7)	183 (100.0)
Blackboard online laboratory resources (e.g., PowerPoints, videos, podcasts)	155 (84.7)	170 (92.9)
Open laboratory sessions	103 (56.3)	Not offered
Other online resources (other than Blackboard online lab resources)	102 (55.7)	104 (56.8)
Quizlets	104 (56.8)	92 (50.3)
Zoom discussion/office hours	Not offered	3 (1.6)
Email messaging	4 (2.2)	6 (3.3)
In‐class notes	137 (74.9)	Not offered
Study group	70 (38.3)	27 (14.8)
Blackboard messaging/discussion board	2 (1.1)	5 (2.7)
Library bone boxes	88 (48.0)	Not offered
Supplemental instruction sessions	25 (13.7)	1 (0.05)
Other	25 (13.7)	22 (12.0)

Face‐to‐face (F2F) and emergency remote teaching (ERT) (n = 183 students).

Most students agreed that their level of engagement (Q16) and the teaching effectiveness (Q17) of the materials used in the course were greater during the F2F phase than in the ERT phase. Similarly, almost every student believed that their connections with classmates and instructors during the F2F format helped them to learn better anatomy (Q18), while very few disagreed or strongly disagreed (Figure 2; Appendix 3).

FIGURE 2

Results of Likert scale questions #16–24. ERT, emergency remote teaching; F2F, face‐to‐face

When asked directly which tools students used to compensate for the lack of cadavers during the ERT phase (Q15), a large number of students (n = 93), determined that no tools could adequately compensate for no longer having access to the cadavers. Students equally responded that YouTube (n = 79) and pre‐existing Blackboard materials (n = 102) compensated for this. The remaining category ‘other’ yielded 25 responses, and included resources such as: Biodigital®, online resources (e.g., pictures of cadavers), other anatomy textbooks, Quizlet (Quizlet.com), anatomy simulation phone applications, and Anatomy and Physiology Revealed. Some students included written responses: ‘I feel like I am not learning even close to as much’, ‘… Nothing compared to the learning that occurred in lab’, and ‘there are not many tools that adequately compensate …’

Self‐perceived learning and satisfaction

Most students perceived a greater learning confidence (Q19, 20), better knowledge gained (Q21) and felt better prepared for their future career (Q23) with the knowledge gained during the F2F phase than during the ERT. Similarly, students were more satisfied with learning anatomy during the F2F than ERT (Q22). By contrast, responses were mixed regarding whether student grades obtained during the F2F portion better reflected their true anatomical knowledge (Q24) (Figure 2; Appendix 3).

Self‐perceived challenges and benefits

When asked which skillsets students had learned that they otherwise would not have because of ERT, most students (n = 152) reported Zoom as the most prominent learned skillset followed by other online resources (Figure 3). Students also listed better time management, self‐motivation, and scheduling self ‘class’ time as valuable skills learned due to ERT.

FIGURE 3

Self‐reported skillsets developed by undergraduate anatomy students because of the emergency remote teaching. OR, ability to find online resources; Soc Med, social media. Data pooled across nine laboratory sections (n = 183)

The thematic characterization of students' responses from the open‐ended questions regarding the challenges and benefits of ERT, resulted in 14 themes for challenges and 11 for benefits (Table 4, Figure 4). According to the Salience index, the most important challenges were those related to the loss of access to cadaver‐based learning. Namely, the difficulty to mentally translate and visualize structures from images to cadavers, difficulties in self‐learning and understanding materials online, and problems with the identification of anatomical structures, such as distinguishing between arteries and veins. Regarding these challenges, students commented:

TABLE 4

Examples and salience index values of thematic characterization of students' self‐perceived challenges and benefits

Theme	Description	Example	Salience index a
Challenges
Visualizing cadavers	Problems translating 2D images into cadavers	“It was difficult transitioning to being able to see and touch things on the cadaver to only being able to see things 2D on a screen”	0.251
Self‐learning	Difficulties with studying, learning, and understanding material	“I really did not understand a lot of what we learned during online learning”	0.246
Identification	Problems identifying and distinguishing anatomical structures	“It was very hard to distinguish between an artery, vein, or nerve for the exams”	0.131
Technology	Internet/computer problems	“My internet connection is unstable at times, making it hard to load the tests”	0.113
Time management	Having and sticking to a schedule	“It was hard to manage schedule”	0.090
Isolation	Feeling of isolation	“I never could attend the zoom meetings so I felt kind of isolated in my learning”	0.077
Motivation	Lack of motivation to study or complete homework	“This made it [online format] difficult to stay motivated for the class”	0.077
Illustrations used in exams	Quality and orientation of photographs used in the examinations	“Some pictures on the exams were not so clear compared to viewing them on a cadaver. I felt as if I was more confused learning these topics online”	0.069
Limited access to teaching assistants	Learning difficulties related to access to teaching assistants	“Without UTAs identifying things on picture based questions were hard when they were referencing cadaver pictures”	0.060
Limited online resources on cadavers	Not finding enough online resources to study	“Mostly just figuring out what to do and find resources to replace the cadavers”	0.034
Home issues	Problems studying from home	“It was hard to find a quiet place to study”	0.037
No challenges	Easy transition	“Overall I did not face many challenges with actual course material”	0.033
Concerns with future	Concerns about being well‐prepared for medical school or applying knowledge to other courses	“I feel like I am not prepared to apply to graduate school …”	0.014
Stress	Reported stress	“Stress and problems studying”	0.009
Benefits
Flexibility in taking exams	Flexibility in completing course assignments and exams	“Being able to take exams multiple times, at different times”	0.413
Learning time management	Learning to create their own schedule	“Being able to set my own schedule”	0.170
Learning at own pace	Studying at their own schedule	“Not being at KU and having class has made me focus on learning the material more on my own …”	0.156
Improving grades	Increase in course grades	“I got much better grades because of remote learning …”	0.101
No benefits	No benefits	“Im sorry, there were no benefits to switching online …”	0.093
More study time	Increase in allotted time to study	“There was a lot more time to study …”	0.078
Less stress	Online transition was less stressful than in person class	“I was not being as stressed about the class. I definitely still spend lots of time for this class and studying, but it's not nearly as stressful”	0.070
Learning to find online resources	Access to online resources	“more resources online to help understand where certain things were …”	0.053
Learning new software	Opportunity to learn new software	“Learning new software”	0.044
Faster communication	Easier access to instructors	“Better email communication with professor/teaching assistants …”	0.011
Help from family	Opportunities to obtain help from family members	“I was able to have my mom help me study since she is a doctor”	0.005

Analysis is based on 183 responses from open‐ended questions 25 (challenges) and 26 (benefits).

Higher salience index values indicate greater consensus and importance among participant's responses.

FIGURE 4

Word cloud displaying (A) self‐perceived challenges; and (B) benefits of the emergency remote teaching by undergraduate anatomy students

It was difficult transitioning to being able to see and touch things on the cadaver to only being able to see things 2D on a screen,

I wasn't able to fully comprehend the content and don't think I could apply it as well in the future. It was very hard to distinguish between an artery, vein, or nerve for the exams.

Among the most important perceived benefits, students recognized the flexibility in taking examinations, the ability to learn how to manage their own time, and the ability to develop their own learning strategies (Table 4, Figure 4). For example, one student commented: “Learning at my own pace, having better time management, learning at different times, being able to take a break in between some of the content and come back to it later when it gets too overwhelming”.

Students' performance

During Semester I of 2020, grades from summative examinations differed before and after the disruption (χ 2 = 111,721, df = 4, P < 0.001). Pairwise comparisons with Bonferroni adjustment indicated that grades from Units 3–5 (ERT phase) were similar to each other (df = 966, P > 0.1) and each greater than those of Units 1 and 2 (F2F phase). The grades from these first two units were also different from each other (df = 966, P < 0.001) (Figure 5). Comparing the grades of the Semester I of 2020 with those from the previous two years (six semesters including Semester II) revealed significant differences among examinations, semesters, years, and the interactions among these three factors (Appendix 4). The grades from Semester I of 2020 remained extremes in the analyses, even after comparing them with grades pooled across the prior six semesters (Appendices 5–7).

FIGURE 5

Academic performance of students enrolled in an undergraduate human anatomy course at The University of Kansas during face‐to‐face (Units 1 and 2) and emergency remote teaching phases (remaining Units) in the Semester I of 2020

The distribution of the final course grade varied among the semesters analyzed, from a rather normal distribution in Semesters I and III to a right‐skewed distribution in the Semester II (Appendices 8 and 9). Although the final course grade distribution during the Semester I of 2020 falls within this variation, it was different from that expected when grades are pooled across semesters and years (χ 2 = 21.2, df = 4, P < 0.001, n = 195).

The grades of the summative examinations from Semester III of 2020 and Semester I of 2021, after the Covid‐19 disruption and with more online teaching‐type style, were also different from that of Semester I of 2020. The average examination grade was atypically high for Semester III of 2020 (≥88% for examinations of all content units) whereas in Semester I of 2021, they were comparable to the average examination grades during 2018 and 2019 (Appendices 6 and 7). The distribution of the final course grades was also different in those semesters with entirely online instruction (Semester III of 2020: χ 2 = 288.2, df = 4, n = 250; Semester I of 2021: χ 2 = 76.0, df = 4, n = 261, P < 0.001 in both cases). The highest percentage of a final grade of A was observed in Semester III of 2020 (65.2% vs. ≤40% from previous years) while the lowest percentage of a final grade of C was observed in Semester I of 2021 (12.3% vs. ≥21.5 from previous semesters excluding Semester II) (Appendices 8 and 9). The percentage of students who withdrew from the examined course each semester also decreased after the interruption experienced by the Covid‐19 pandemic, from an average of 13% per semester (±5.25, n = 5 semesters) during 2018 and 2019 to 4.5% (±2.54, n = 4 semesters) in 2020 and 2021 (Appendix 8).

DISCUSSION

Curricular changes

Few studies have yet reported the specific curricular changes that occurred in anatomy courses during the Covid‐19 disruption, but the few examples indicate that a wide range of approaches were adopted. In the undergraduate course examined, content delivery during ERT included both asynchronous pre‐recorded concept videos, discussion boards, and synchronous video discussion sessions. Summative assessments consisted of online spotter examinations with a mixture of short answers, multiple choice, ordering, and true and false questions. Thus, this course falls within the wide range of mitigation strategies adopted by anatomy instructors and institutions after the onset of the pandemic, which ranged from entirely synchronous to asynchronous sessions, and from multiple choice assessments to open‐book examinations with pass‐fail grading (Longhurst et al., 2020; Pather et al., 2020). In the course examined, several pre‐existing videos available to students during the F2F course also facilitated ERT. Given the large size of the course (197 students distributed among 9 laboratory sections), these pre‐existing videos, as well as other newly created videos, were the basis for the asynchronous delivery of content. Synchronous discussion sessions were optional and poorly attended.

Student assessment has been recognized as one of the most challenging aspects to address during the ERT shift and multiple formats were incorporated during this transition (Evans et al., 2020). The variation in assessment format in the examined course was the result of differences in its content delivery. For example, the Unit 3 examination, despite being administered online, attempted to replicate the spotter test (short answer questions), because students learned the Unit 3 material during the F2F portion of the course. However, the examinations for the remaining two units, which students learned entirely during the ERT phase, consisted mostly of multiple‐choice questions, with a small number of other question types included for variation (Table 2). In all cases, questions were taken from a pool and students were allowed to practice them in a “mock examination”, as is customary in the F2F course. Although multiple choice questions may not be the most appropriate way to assess students' anatomical learning and understanding (Evans et al., 2020), this temporary solution was adopted by many institutions during the ERT (Longhurst et al., 2020; Pather et al., 2020), thus highlighting the utility of this format during contingency measurements in moments of crises.

Students' perceptions of ERT

Results from the survey indicate that students were generally dissatisfied with the quality of instruction during the ERT phase of the course, as more than 80% of respondents in some form of agreement indicated that they were more satisfied with the F2F phase than the ERT phase (Figure 2; average Likert scale 5.95–6.27; see Appendix 3). Thus, these results support the hypothesis that students would experience lower levels of satisfaction during the ERT when compared to the F2F phase. While this pattern is consistent with previous studies that have indicated student preferences of cadaver‐based learning (or other physical, F2F methods) in anatomy over computer‐based methods (Azer & Eizenberg, 2007; Petersson et al., 2009; Kerby et al., 2011; Davis et al., 2014; Ogard, 2014; Attardi et al., 2016; Cuschieri & Calleja Agius, 2020; Banovac et al., 2021), the near unanimity of its consensus is somewhat unexpected. This could be due to the students' self‐perceived learning, the results of which nearly parallel those of overall satisfaction. Despite increases in student grades, roughly three out of every four students reported less confidence in their ability to learn, less knowledge gained, and less preparedness for the future during the ERT duration (Figure 2). These results agree with previous studies demonstrating that computer‐based learning is not as efficient as traditional methods to learn anatomy (Khot et al., 2013; Preece et al., 2013; Saltarelli et al., 2014; Mathiowetz et al., 2016). They also reinforce the notion that cadaver laboratories offer a “hidden curriculum”, wherein students learn aspects of professionalism and ethics (i.e., “preparedness for the future”) through their interactions with cadavers (Bamber et al., 2014; Sándor et al., 2015; Karunakaran et al., 2017; Evans et al., 2018; Kumar Gosh & Kumar, 2019). These principles are likely not as easily transmitted through computer‐based learning, which the students of this course may have inadvertently alluded to in their responses.

Even so, there were some perceived benefits of the shift to ERT among students. Most notably, they responded that increased flexibility in studying and taking examinations was the greatest advantage of ERT. A reduction in time constraints in computer‐based learning has been reported previously regarding online anatomy education (Attardi et al., 2016; Mathiowetz et al., 2016), though some instructors faced the opposite scenario of increased time concerns due to Covid‐19 (Longhurst et al., 2020). Students in this course reported other advantages of ERT like time management skills, learning at their own pace, and more study time, all of which further emphasize the perceived benefits of reducing time constraints on students. Interestingly, one of the most common themes to emerge from studies detailing anatomy students' perceptions of ERT is that a plurality of students worldwide almost invariably prefer the use of pre‐recorded lecture videos as an instructional resource above any other method (Cuschieri & Calleja Agius, 2020; Gupta & Pandey, 2020; Kim et al., 2020; Banovac et al., 2021; Yoo et al., 2021). This seems to apply mainly to didactic components of anatomy courses, however, and does not necessarily extend to cadaver‐based dissection (Banovac et al., 2021).

The most frequent challenge cited by students during ERT was difficulty with mentally visualizing the cadavers. Previous studies have reported similar complaints among students (Attardi et al., 2016; Srinivasan, 2020), and a key focus of online anatomy education is the 3D visualization of anatomical structures (Estai & Bunt, 2016). Regrettably, students in this course did not have access to advanced 3D visualization software (other than those which are freely available online) due to the abruptness of the online transition, and thus may have struggled more with cadaver visualization as a result. Another intriguing challenge cited by students during ERT was a feeling of isolation, and lack of access to an instructor/TA, the latter of which is commonly reported (Attardi et al., 2016; Longhurst et al., 2020). These responses were given even though instructors were available through synchronous discussion sessions, as well as by email and through specially requested appointments. Such a consensus suggests that optional Zoom appointments and email/Blackboard communications may not be enough during remote instruction to foster feelings of connectedness and adequate mentorship among students.

Based on the authors' experience with ERT and the Covid‐19 interruption, additional methods for building a connected online learning environment and mitigating feelings of isolation are recommended. For example, implementing mandatory synchronous video learning sessions and/or office hours, rather than optional ones; implementing platforms to encourage peer‐to‐peer feedback, such as the use of VoiceThread® to share and comment on fellow students' work; and the use of a communication platform such as Slack® (Slack Technologies, Vancouver, BC, Canada) or Microsoft Teams. The latter platform seems to increase communication among students, as well as between students and instructors, as it provides a less formal and more convenient mode of communication than standard email. Further, dividing students into smaller chatrooms (“channels”) within these media, each with an assigned instructor/TA, can also promote feelings of connectedness and engagement. Some of the suggestions listed above were implemented in the examined course during the remaining semesters of 2020 and Semester I of 2021. Although the impact of these changes to the course on the students' engagement were not measured, some experiences highlight the benefits of using such tools in online education (Mejia, 2020; Poston et al., 2020).

Examination grades improved during ERT, resulting in high final course grades relative to previous semesters (Figure 5; Appendices 6–9). Thus, the hypothesis that lower examination grades were expected during the ERT phase was not supported. This increase is likely due to at least four reasons: first, the grades from the examinations of Units 1 and 2 were lower than in previous semesters, which might have resulted from a new grading practice. Historically, GTA's graded examinations manually, but beginning Semester I of 2020, the first two examinations were graded using Gradescope® (Turnitin LLC., Berkeley, CA), a Web App software that uses artificial intelligence and allows for more consistent, faster, and less biased grading (Singh et al., 2017). Unlike manual grading, Gradescope grades one question for all students across sections at once and thus it reduces grading bias among laboratory sections, in addition to grading fatigue (Singh et al., 2017). Second, students were able to take the examinations twice and had three minutes per question during the ERT phase. In contrast, students had only one opportunity and less than a minute per question during F2F. Third, examinations during the ERT phase included primarily multiple‐choice questions, except for that of Unit 3 that consisted of short answers only. These results agree with some studies demonstrating that students obtain higher grades in multiple choice tests than in short answer tests (Mujeeb et al., 2010; Pepple et al., 2010), although this might depend on the students' college level (Adamu et al., 2018). Fourth, because of the sudden change to an online format, examinations could not be proctored using a remote proctoring software, as would have been the case for a well‐planned online course. Although examinations were timed and questions were taken at random from question pools each time to reduce cheating during ERT, these measures may not have been effective in completely removing student use of online resources or the course laboratory manual during examinations. It is interesting to note that when the Proctorio software was implemented in Semester I of 2021, the examination grades were significantly lower than those observed during the ERT phase and Semester III of 2020, and thus more like grades before the Covid‐19 disruption. Although the impact of implementing Proctorio was not addressed in the course, there is evidence indicating that this software deters students from cheating and leads to grades from online examinations like those taken in person (Lewis, 2020).

Although students' examination performance appears to be similar between an anatomy course delivered online and F2F (Inuwa et al., 2012), available information indicates that course grades significantly increased during the ERT in a wide range of classes, at least in some Spanish academic institutions (Gonzalez et al., 2020; Iglesias‐Pradas et al., 2021). However, it is unclear whether this is a result from improvements in students' learning strategies due to confinement, structural changes made to examination type, or possible misconduct. Interestingly, the students themselves were unsure of the influence of F2F compared with ERT on the changes witnessed in their examination grades. The responses to the survey question 24 (“my exam grades better reflected my true knowledge of anatomy in the in‐person, cadaver‐based portion of the class …”; Figure 2) were mixed and the average response score for that question was 4.8 ± 1.77 (Neither agree nor disagree to somewhat agree), thus indicating students' uncertainty regarding the accuracy of their grades. Undoubtedly, further studies are needed to determine if grade improvement was a general pattern during the disruption, and whether this pattern correlates to students' self‐perceived learning.

Limitations of the study

This study documents the curricular modifications of a single undergraduate anatomy course during the Covid‐19 pandemic, assesses student perceptions and performance during the transitional F2F‐to‐ERT period, and further documents performance over sustained course‐history. Undergraduate anatomy courses vary significantly in their approach, teaching materials and personnel, resources, and use of cadavers (Sparacino et al., 2018), therefore it is likely that students' perceptions and performances may also vary. The variability in teaching practices of anatomy and other types of education is best given due consideration when making comparisons with the students' performance and perceptions in this undergraduate course.

It should also be noted that student attitudes and perceptions were asked during a time of great uncertainty, when Covid‐19 was unresolved and remained a constant threat to personal and professional lives. There may be large differences in what people reported during this time, and what they perceive months or years later, after receiving their final grade, moving forward in their career path, and persisting through the continuation of the Covid‐19 pandemic. Thus, the current views of these participants should not be the sole factor of long‐term or extrapolated decision making. These results should instead be taken as a contribution to the evidence upon which decisions regarding education and best learning practices are made.

Finally, the order and wording of some questions using Likert scales might skew participant responses (McFarland, 1981; Malhotra, 2008; Garbarski et al., 2015). However, the survey had two questions (19 and 20) with reverse wording (Appendix 1) to reduce the effects of response styles or tendency of respondents to answer items regardless of their content. In addition, Kendall's Tau b correlation coefficients were significant (P < 0.01) among survey items using Likert scales and thus indicating acceptable internal reliability among the survey items (Appendix 2).

CONCLUSION

This study revealed that, despite examination grades improving during ERT, students reported lower levels of learning, confidence, and engagement with the course materials. Grades remained abnormally high during the subsequent online‐only semesters (Semesters II and III of 2020), but reverted toward the norm following the implementation of Proctorio during Semester I of 2021. Like other instructors and institutions challenged to promptly change their in‐person classrooms to online, the teaching and examination methods of this undergraduate anatomy course fell within the range of approaches adopted by other instructors and institutions worldwide. Despite earning higher grades, students highlighted several learning challenges during ERT resulting from the loss of cadaver‐based anatomy education. These included the loss of hands‐on, tactile, and sensorial training, in addition to the social and teamwork interactions students develop through peer‐to‐peer and instructor‐to‐peer mentorship in an interactive laboratory setting. To circumvent challenges faced in future iterations of ERT, and in online anatomy courses more generally, the authors suggest mandatory synchronous video sessions, peer‐to‐peer interaction and feedback, and the use of accessible, informal means of communication, such as Slack or Microsoft Teams. These findings provide context for the expectations of students and instructors experiencing an academic learning environment challenged with the effects of a pandemic, economic uncertainty, and future uncharted disruptions. This study helps to prepare students and instructors to develop resilient teaching and learning practices and create effective classrooms whether they be in‐person, online, or both.

CONFLICT OF INTEREST

The authors declare no conflict of interest.

ETHICS STATEMENT

The University of Kansas' Human Research Protection Program and Internal Review Board (IRB) reviewed and approved this study on April 27, 2020 (IRB #00145687). All study participants were asked for consent prior to collection of data and were informed that the survey was anonymous and voluntary. Respondents were informed that the survey results and data were to be used for research purposes. To ensure privacy, no IP addresses were linked with completed surveys, thus the results were analyzed anonymously.

Supplementary Material

Click here for additional data file.