How student perceptions about online learning difficulty influenced their satisfaction during Canada's Covid-19 response.

The COVID-19 pandemic has posed a significant challenge to higher education and forced academic institutions across the globe to abruptly shift to remote teaching. Because of the emergent transition, higher education institutions continuously face difficulties in creating satisfactory online learning experiences that adhere to the new norms. This study investigates the transition to online learning during Covid-19 to identify factors that influenced students' satisfaction with the online learning environment. Adopting a mixed-method design, we find that students' experience with online learning can be negatively affected by information overload, and perceived technical skill requirements, and describe qualitative evidence that suggest a lack of social interactions, class format, and ambiguous communication also affected perceived learning. This study suggests that to digitalize higher education successfully, institutions need to redesign students' learning experience systematically and re-evaluate traditional pedagogical approaches in the online context.

INTRODUCTION

The Covid‐19 pandemic and subsequent public health responses had an unprecedented impact on higher education. In March 2020 Canadian universities, which had then largely delivered in‐person classes, announced that they would abruptly transition their classrooms to an entirely online learning environment (Houlden & Valetsianos, 2020). In a matter of a weeks, many offline courses transitioned to an online offering, a trend which continued throughout subsequent terms in 2020 and 2021 (Myrick et al., 2020). Though many university programs transitioned to a formal online offering over the summer of 2020, this transition was undertaken by faculty and students who often had no online learning experience (VanLeeuwen et al., 2021). This transition has reportedly contributed to pervasive negative reactions among students (Besser et al., 2020) and has even taken a toll on many students' mental health (Copeland et al., 2021).

Why did it happen this way? Since the widespread adoption of the internet in the 1990s, numerous papers have been reinforcing the need for organizations to introduce online learning and have identified a variety of manageable factors can influence online learning outcomes (Boling et al., 2012; Sloman & Reynolds, 2003). For example, Sun et al. (2008) identified that computer anxiety negatively influences reported online learning satisfaction, while perceived course quality or ease of online learning technology use positively shape it. Similarly, cognitive overload is known to influence online learning outcomes (Mayer, 2009; Mayer & Moreno, 2003), and steps can be taken to distinguish and limit information or communication overload. Research concerning online learning during the Covid‐19 pandemic has identified how factors such as attitudes towards technology (Aguilera‐Hermida, 2020), degree of learner attentiveness (Conrad & Newman, 2021) and effective course design (Orlov et al., 2021) can positively influence online learning outcomes.

As the pandemic progressed, it became clear that there were additional considerations that affected online teaching success. Online courses still require faculty to provide learning material and communicate directly with students (Swan, 2019), and the change in communication format initially became a source of anxiety for many (Unger & Meiran, 2020; VanLeeuwen et al., 2021). Though many instructors were unprepared for the new teaching medium, many instructors and students alike later benefitted from the flexibility of material and classroom “space” that could be accessed anytime and from any place. Some researchers believe that this level of flexibility makes online learning interesting to students (Korhonen et al., 2019; Mitchell et al., 2015), and there is evidence that this influenced perceived usefulness of online learning technology, and subsequently attitudes towards the technology generally (Oliveira et al., 2021; Vladova et al., 2021). Faculty and students' abilities to use technology as well as their motivation in online learning can affect learning success (Ellis & Bliuc, 2016; Oliveira et al., 2021; Wei & Chou, 2020). Students similarly need to change how they perceived learning through the use of multiple online tools and strategies.

By studying the transition to online learning during Covid‐19, we can identify methods for improving outcomes from future global disruptions. As a team of educators in a management faculty who had primarily contributed to an in‐person learning environment before the pandemic response, we were particularly interested in factors that affected students at our university who were similarly not accustomed to online learning. Following an abrupt transition to emergency online learning in the Spring of 2020, instructors at our university prepared optional online learning training, and were encouraged to create pre‐recorded lectures, hands‐on activities with weekly deliverables, and asynchronous class discussion forms to help accommodate exam difficulty and support students in different time zones. We were quickly attuned to reports by our students about perceived difficulty with e‐learning technology and being overwhelmed by information and communications as our university. We were thus motivated to investigate the way these factors influenced perceived difficulty and satisfaction with our university's online learning environment.

In this paper, we describe the results of a survey that we administered in October and November 2020 at a single university undergoing a transition from in‐person to entirely online learning, in which instructors were asked to emphasize asynchronous teaching content, such as pre‐recorded videos, lecture slides and the provision of other learning materials that could be used by students at any time, rather than a synchronous lecture or tutorial. The survey sought to identify the sources of satisfaction and dissatisfaction with our online environment (Sun et al., 2008) and whether cognitive overload generated by the quantity of information and communications influenced satisfaction (Lee et al., 2016). The quantitative results of the survey were interpreted in light of qualitative responses also provided by our students concerning the environment and their difficulties. The paper is structured as follows: we begin by reviewing some related work on online learning satisfaction and cognitive load in light of the forced transition to online learning during the Covid‐19 pandemic. We then identify testable hypotheses and describe our qualitative and quantitative approaches to the research. We conclude with a discussion of theoretical contributions and practical recommendations for educators for adapting to future crises.

THEORETICAL BACKGROUND

Cognitive load and information technology satisfaction

Online learning, conceptualized as the delivery of learning experiences using some sort of internet technology, is by no means a new idea (Conrad, 2002; Moore et al., 2011). Having its origins in the early 2000s, researchers have spent nearly two decades investigating factors that influence online learning success, and have notably identified design factors, but also learner characteristics that can predict outcomes (Kauffman, 2015). During the transition to online learning during Covid‐19, universities that had not yet adopted online learning technologies were forced to do so quickly, with faculty or staff persons who had been minimally trained in online delivery, and the success of these efforts continues to be a topic of interest (Alqahtani & Rajkhan, 2020).

With respect to the success of a specific implementation of an online learning technological artifact, there are many ways to measure success, and among the most prominent is that of learner satisfaction (Sun et al., 2008). While prominent e‐learning models have investigated factors such as cognitive, social and teaching presence on student satisfaction (Garrison et al., 1999), these approaches do not account for the specific role of cognitive load in influencing perceived difficulty and satisfaction with the e‐learning technology as a sociotechnical artifact. By contrast, information systems researchers have investigated the role that satisfaction plays in information technology use broadly, and it has even been designated as a theoretical category within the information systems field (Vaezi et al., 2016). With its origins in marketing and evaluative cognitive processes, technology satisfaction theories often emphasize the perception and evaluations that lead to satisfaction formation. For example, Sun et al. (2012) describe satisfaction as “the level of emotional response to needs fulfilment through IT services,” in the context of IT service delivery (Sun et al., 2012, p. 1198; Vaezi et al., 2016). In the case of the transition to online learning during Covid‐19, we can thus similarly understand satisfaction to reflect an emotional response to needs fulfilment of online learning delivery, conceptualized as a sociotechnical artifact. Factors observed influencing online learning satisfaction can be similarly understood to influence students' perceptions of online learning success.

The degree of cognitive overload experienced during an online learning experience has been a well‐studied predictor of online learning success, and of satisfaction with information systems broadly (Roetzel, 2019). For example, the Cognitive Theory of Multimedia Learning describes how multiple learning media offered in conjunction (eg, words and pictures) translate to better learning outcomes than either medium alone (Mayer, 2009). Rooted in Sweller's theory of cognitive load (1994), the Cognitive Theory of Multimedia Learning is built on an assumption about humans' limited capacity to process sensory information, which are in turn moderated by the limited capacity of the cognitive mechanisms associated with our sensory organs. A balanced level of sensory information generated by diverse sources of online learning multimedia would help limit extraneous cognitive load, leaving greater cognitive capacity for learning (Sweller, 1994). It follows, then, that too much sensory information (eg, from long lectures, from overwhelming emails or instant messages) would result in greater course difficulty.

We are thus motivated to create a research model for investigating the ways that sources of cognitive load played in the transition to online learning during the Covid‐19 pandemic. By studying the specific sources of overload, such as information quantity, communication quantity, or poor online learning skills, we might develop factors that increased learning difficulty for students during such rapid transitions to online learning. By further observing the relationship between learning difficulty and online learning satisfaction (Sun et al., 2008), we can develop more comprehensive insight into factors that led to quality online learning experience.

Research model and hypotheses

We can develop our hypotheses regarding the effects of information overload, communication overload, and perceived technical skill requirement on the perceived difficulty of online learning, and in turn, on online learning satisfaction. Figure 1 shows the conceptual framework that guides our hypotheses development.

FIGURE 1

Online learning satisfaction: a conceptual framework

Information overload

In contrast to general cognitive load, information overload refers to the specific state where “an individual's information processing capabilities are exceeded by the information processing requirements” (Karr‐Wisniewski & Lu, 2010, p. 1062). It often occurs when an individual receives more information than what the individual's cognitive ability can process at a specific time (Eppler & Mengis, 2004) and concerns the quantity of information provided to users, as opposed to qualitative or extraneous factors (eg, difficulty to comprehend, challenges related to accents) that are often associated with learning multimedia (Mayer, 2009). In an e‐learning context, the multiplication of communication tools used by instructors (online discussion systems, emails, platform announcements, social media, active online discussions, etc.) can contribute to increase the perceived information overload. In a traditional classroom setting, instructors can more easily adapt their communication tools to the needs of their students, which is more difficult to do in an online teaching context.

While previous research in various disciplines has studied information overload from different perspectives, a consistent finding emerged from these studies is that information overload, in general, has a negative effect on individuals (Eppler & Mengis, 2004). For instance, previous research found that information overload can lead to a strenuous personal situation where individuals will experience many dysfunctional consequences, such as demotivation (Baldacchino et al., 2002), stress, confusion, and cognitive strain (Jones, 1997; Malhotra, 1982; Schick et al., 1990), lack of learning (Sparrow, 1999), lack of perspective (Schick et al., 1990), and sense of loss of control (Schneider, 1987). These consequences can further lead to arbitrary information analysis and organization, such as ignoring information (Bawden, 2001; Sparrow, 1999), requiring higher time for information handling (Hiltz & Turoff, 1985; Jacoby, 1984), and misinterpreting information (Sparrow, 1999). In the context of online learning, information overload has been identified as a significant barrier to effective learning (Chen et al., 2011). For example, research has suggested that, in online learning, students with information overload tend to have difficulty participating in online discussions (Chen et al., 2012). Therefore, we argue that students experiencing information overload in online learning are likely to experience the dysfunctional consequences and, as a result, will perceive online learning to be more difficult. As such, we predict:

Information overload will positively influence perceived online learning difficulty.

Communication overload

Communication overload, according to Lee et al. (2016, p. 53), refers to a “state when communication demands from ICT channels such as SNS (eg, emails, instant messaging, and news feeds) exceed users' communication capacities.” It mainly focuses on the receiving of messages; thus, it is often initiated by a third party (Karr‐Wisniewski & Lu, 2010; Lee et al., 2016). Prior research has found that communication overload can cause frequent interruptions that make it hard for knowledge workers to stay concentrated and continue their current work (Karr‐Wisniewski & Lu, 2010; Lee et al., 2016; Udemy, 2018). As a result, communication overload has been found to negatively influence work efficiency, accuracy, and performance (Karr‐Wisniewski & Lu, 2010; Mansi & Levy, 2013; McFarlane & Latorella, 2002). Moreover, later studies found that communication overload can make individuals feel overwhelmed, fatigued, and stressed (Lee et al., 2016). If the overload situation persists, it can even lead to severe mental or physical diseases (Klapp, 1986).

In online learning, where computer‐mediated communication is heavily relied, interruptions caused by communication overload are more likely to happen and can occur in various forms, such as emails or messages sent by instructors, updated announcements in the online learning system, or even the reminders of the assignment deadline. Every time an interruption occurs, the primary task is suspended, and students need to reprocess some of the primary task's information to retrieve them (Federman, 2019). This process increases cognitive load and can be effortful. It has been reported that every time an interruption occurs, it takes an average of 25 minutes for individuals to regain the focus (Mark et al., 2005). Even a brief interruption as short as 3 seconds can double the error rate in the following tasks (Trafton et al., 2003). As such, interruption has been found to be a barrier to online learning (Becker et al., 2013), which might be compounded by the general perception of communication overload during the pandemic (Ahmed, 2020). Therefore, it is likely that students experiencing communication overload will have difficulties with online learning. Thus, we predict:

Communication overload will positively influence perceived online learning difficulty.

Perceived tech skill requirement

Online learning is heavily dependent on information and communication technologies. Depending on the design of course delivery format (eg, synchronous, asynchronous, hybrid) and course content, online learning could involve adoption and use of a variety of technologies, such as online learning management systems (eg, Brightspace, Blackboard), video conferencing tools (eg, Zoom, Microsoft Teams), messaging software (eg, email, WhatsApp), and sometimes virtual private networks (VPNs). As such, students' attitudes toward technologies may affect their online learning perceptions (Becker et al., 2013; Lee et al., 2016).

Research in Management Information Systems has widely investigated the effect of technology characteristics on individuals' attitudes toward the technology. This research found that technology complexity (ie, the extent to which use of technology requires effort) can create technostress because users have to take a lot of efforts to understand the technology and learn how to use it (Ayyagari et al., 2011; Tarafdar et al., 2007). Furthermore, highly complex technology could lead to the feeling of system feature overload, which has been found to increase technology fatigue (Karr‐Wisniewski & Lu, 2010; Lee et al., 2016). In online learning, students' attitude towards technology is an important factor that affects online learning effectiveness (Šumak et al., 2011). When students have a lower level of technology anxiety, they tend to have a more positive attitude toward online learning (Piccoli et al., 2001; Sun et al., 2008). Therefore, we argue that when students believe learning online requires a high level of tech skills, they are likely to perceive online learning to be more difficult. As such, we predict:

Perceived tech skill requirement will positively influence perceived online learning difficulty.

Online learning difficulty and satisfaction

In the context of online learning, previous research found a positive relationship between perceived ease of learning online and students' satisfaction towards online learning (Islam et al., 2018; Joo et al., 2011; Šumak et al., 2011). Students' perceived ease of using online learning technologies were found to be positively related to their satisfaction towards online learning and intention to adopt and continuously use online learning technologies (Joo et al., 2011; Lee, 2010; Sun et al., 2008). Despite the focus on online learning technologies, this research constantly suggests that the easier students perceive online learning, the more likely they are satisfied with online learning. Moreover, when students perceive online learning to be easy to conduct, they will spend more time learning, which leads to a higher level of satisfaction (Lee, 2010; Teo & Wong, 2013). As such, from a flip aspect, we predict:

Perceived online learning difficulty will negatively influence online learning satisfaction.

METHODS

Research design

The present study adopts a mixed method design to understand factors that affect students' attitude towards online learning during the pandemic. Building on the work of Sun et al. (2008), we sought to use questionnaires to investigate how our hypothesized factors could influence online learning satisfaction. Recognizing the reported duress of our target population, we sought to create a simple questionnaire that was fast to complete and adapted quantitative measures to identify a causal model of the specific cognitive load‐related factors on difficulty and satisfaction. To address these, we adapted measures described by Sun et al. (2008) and Lee et al. (2016). In addition, we asked participants to respond to a single open‐ended question about their experience to corroborate our findings, but to also identify other possible factors that might have influenced our findings or limited our survey approach. Given that we were motivated to investigate student struggles due to being overwhelmed, our priority was to place as little burden on the participants as possible, by asking specific questions that they could answer in seven minutes or less.

Questionnaire instrument

To test our hypotheses, we developed a measurement instrument based on extant scales in previous literature. Since some of these scales were developed in the context of technology adoption and social media usage, we rephrased the questions to make the instrument better suit the online learning context. The measurements and their sources were presented in Table 1. All items were evaluated on a 7‐point Likert scale (1 = “Strongly disagree”, 7 = “Strongly agree”).

TABLE 1

Measurement of variables

Variables	Notation	Items	Sources
Information overload	IFO1	I am often distracted by the excessive amount of information related to my online courses	Lee et al. (2016); Karr‐Wisniewski and Lu (2010)
	IFO2	I find that I am overwhelmed by the amount of information related to my online courses
	IFO3	I feel like I have too much information to synthesize, rather than not enough information
Communication overload	COM1	I receive too many messages from instructors (or peers) related to online courses	Lee et al. (2016); Cho et al. (2011)
	COM2	I feel that I generally get too many notifications of new postings, push messages, news feeds, etc. related to my online courses while performing other tasks
	COM3	I receive more communication messages related to online courses than I can process
Perceived tech skill requirement	PTR1	I believe that learning online requires technical ability	Gattiker and Hlavka (1992)
	PTR2	I believe that learning online can only be done if one has advanced technical skills
Perceived difficulty	DIF1	I believe that learning online is very difficult	Sun et al. (2008)
	DIF2	I believe that learning online is very complicated
	DIF3	I believe that learning online makes me feel psychological stress very greatly
Satisfaction	SAT1	I am very satisfied with my online courses	Sun et al. (2008)
	SAT2	I feel that my courses serve my needs well
	SAT3	I am disappointed with the way my courses worked out

Qualitative analysis

To analyse qualitative data, a manual coding method was used. The coding was done with the help of the QDA Miner Software that allowed classifying data under relevant categories. We utilized tags and labels to assign codes to the students' answers to open‐ended questions. When the data were classified, we were able to identify the key words, patterns, and common themes in the textual data.

Data collection

To collect data for this study, we conducted an online survey in October–November 2020 at a major university in Canada. At the time of data collection, the university has transitioned to online teaching for approximately seven months because of the COVID‐19 pandemic and was delivering the courses using synchronous, asynchronous, and hybrid formats; though it is worth noting that many instructors did not run courses over the summer months. This data collection timing provides an interesting context for the present study as it helps us capture students' perceptions of online learning during the pandemic, a time when online learning was not adopted voluntarily. The first round of survey invitations was sent out to students via email in October 2020, followed by the second round of invitations sent in November 2020.

The survey had 32 questions, on a 7‐point Likert scale with 1 being totally disagree, 4 being neither agree or disagree, and 7 being totally agree, and five demographics questions. Many of the questions asked in the survey concerned demographic and academic factors which are not reported in these results, and subset of these survey questions was used in the investigation described in this paper. Additionally, we had one open ended question which asked the following—“Do you have any additional comments on your difficulties with online learning that you would like to share”? A total of 285 survey responses were received. After removing the incomplete responses, 240 responses remained as the final sample for our following data analysis. For the open‐ended question, only 138 responses were received. We will discuss the analysis of the open‐ended question after discussing the analysis and results of the survey data. Table 2 presents the demographic profile and descriptive statistics of the final sample.

TABLE 2

Demographic profile and descriptive statistics of sample

Category	Frequency	Percent (%)
Gender
Male	94	39.2
Female	130	54.2
Other	7	2.9
Prefer not to disclose	9	3.7
Age
Under 18	2	0.8
18–24	189	78.8
25–34	40	16.7
35 or older	8	3.3
Prefer not to disclose	1	0.4
International student
Yes	64	26.7
No	169	70.4
Prefer not to disclose	7	2.9
Number of online courses taken before
0	81	33.8
1–2	95	39.6
3–5	41	17.1
More than 5	22	9.2
Prefer not to disclose	1	0.4
Current number of online courses
1	9	3.8
2	17	7.1
3	25	10.4
4	77	32.1
5 or more	109	45.4
Prefer not to answer	3	1.3
Total hours per week on online courses
Less than 14 hour	34	14.2
15–24 hour	35	14.6
25–34 hour	71	29.6
35–44 hour	62	25.8
45 hour or more	37	15.4
Prefer not to answer	1	0.4

RESULTS

Quantitative model

We tested the proposed model using structural equation modelling (SEM). IBM SPSS AMOS 26 was used for model estimation. Since our measurements were adapted from various sources, we first conducted a confirmatory factor analysis (CFA) to evaluate the reliability and validity of the measurements. The results indicate that, at the global level, the measurement model has a good fit (CFI = 0.983; TLI = 0.974; RMSEA = 0.049). We then checked the reliability and convergent validity of the measurement model by examining the factor loading, composite reliability (CR), Cronbach's alpha, and the average variance extracted (AVE) (see Table 3).

TABLE 3

Reliability and convergent validity testing results

Factor	Items	Std. factor loading	Cronbach's α	CR	AVE
Information overload	IFO1	0.879	0.904	0.906	0.763
	IFO2	0.911
	IFO3	0.828
Communication overload	COM1	0.827	0.872	0.872	0.872
	COM2	0.803
	COM3	0.868
Perceived tech skill requirement	PTR1	0.773	0.666	0.677	0.514
	PTR2	0.656
Perceived difficulty	DIF1	0.890	0.887	0.889	0.728
	DIF2	0.810
	DIF3	0.857
Satisfaction	SAT1	0.953	0.907	0.912	0.778
	SAT2	0.908
	SAT3	0.775

As shown in Table 3, all standardized factor loadings were greater than 0.6, CR and Cronbach's alpha were greater than 0.6, and AVEs were greater than 0.5 (Fornell & Larcker, 1981; Hess et al., 2009). These indices show that our measurement has good composite reliability and convergent validity. For discriminant validity, we compared the square root values of AVE to the inter‐construct correlations in Table 4. As shown in Table 4, the squared root of the AVE for each construct was greater than all the related inter‐construct correlations, and thus the discriminant validity of all scales was established (Fornell & Larcker, 1981).

TABLE 4

Descriptive statistics, correlation matrix, and squared root of the AVE

	Mean	SD	1	2	3	4	5
Satisfaction	4.22	1.70	0.882
Information overload	2.35	1.45	−0.536	0.873
Perceived tech skill requirement	3.37	1.28	−0.363	0.556	0.717
Perceived difficulty	2.58	1.50	−0.790	0.762	0.596	0.853
Communication overload	3.74	1.65	−0.236	0.566	0.469	0.446	0.833

To test the hypotheses, we used the path coefficients of independent variables and their statistical significance in the structural model. We included two control variables, namely, prior online learning experience (whether a student has taken online course previously) and international status (whether a student is international or domestic student), in the structural model because previous research found they can affect students' online learning satisfaction (Chen et al., 2012; Jan, 2015; Ke & Kwak, 2013). Since no significant effect of prior online learning experience was found, we removed it from our model and only included international status as a control variable in our final structural model. At the global level, the structural model showed a good fit (χ 2 = 137.185; χ 2/df = 1.737; TLI = 0.962; CFI = 0.975; RMSEA = 0.056). The results are presented in Figure 2.

FIGURE 2

Structural model results

As expected, the results showed that information overload has a positive effect on perceived difficulty (t = 7.880, p < 0.001), thus Hypothesis 1 is supported. Furthermore, we found that perceived tech skill requirement positively impacts perceived difficulty (t = 2.803, p < 0.01), therefore, Hypothesis 3 is supported. Different from our expectation, the effect of communication overload on perceived difficulty is not significant (t = −0.765, p = 0.444). Thus, Hypothesis 2 is not supported. One possible explanation of this non‐significant result is that, while communication overload may increase the students' difficulty in managing and tracking the communication activities, it also indicates that the instructor has a high level of social presence and more interactions with the students. Research has found that student–instructor interactions can help lower students' anxiety by reducing the uncertainty towards learning materials and encourage students to commit themselves to the course, thus could ease students' online learning experience (Abrantes et al., 2007; Jaggars & Xu, 2016). Lastly, we found that perceived difficulty has a negative effect on online learning satisfaction (t = −11.229, p < 0.001), supporting Hypothesis 4. Regarding the effect of control variable, we found that international status has no significant effect on perceived difficulty (t = −0.514, p = 0.607), but has a negative effect on satisfaction (β = −0.119, t = −2.541, p < 0.05); that is, despite international students did not find online learning more difficult, they still had a lower level of satisfaction towards online learning than domestic students.

Qualitative findings

As we had indicated previously, the survey had one open‐ended question, which asked if “participants would like to add any other comments pertaining to their experience with the online learning”. There were 138 responses to this question out of the 240 participants who completed the survey. Guided by the three dimensions in the quantitative hypotheses model described in previous sections, we ran the 138 responses through the QDAMiner software, generating a list of frequently occurring keywords. Figure 3 shows the pie chart of the “Included Keywords” and Figure 4 shows the word cloud of the “Leftover Words”.

FIGURE 3

Frequency of included words

FIGURE 4

Frequency of leftover word

Using the descriptors from the hypotheses model, Information Overload, Communication Overload, and Perceived Technical Skill Requirement, three researchers on the team developed code categories described in Table 5. We grouped “unclear communication of expectations” and “too much information” as codes under the category of Communication Overload. The “Perceived Technical Skills Required” found itself in two code categories, “Required Skills” and “Support”, as the Technical Skills code and Tech Support code respectively. Since the grounded theory approach allows for the theoretical basis of social constructivism (Charmaz, 2006) and objectivism (Glaser & Strauss, 1967) to be invoked, we continued to develop code categories and codes for the 138 responses and the included/leftover keywords. The units of analysis were phrases and sentence fragments as provided in Appendix A. Once this was done, the three researchers independently coded the 138 responses and achieved an intercoder reliability Krippendorff Alpha score of 0.87.

TABLE 5

Code categories and codes—open‐ended question

Code categories and codes

Usage ◦ Positive experience ◦ Negative experience Time ◦ Time zone Support ◦ Prof support ◦ TA support ◦ Teammate support ◦ Tech support ◦ Emotional psych support ◦ General support Performance ◦ Course performance ◦ Program performance Lack of Social Interaction ◦ With friends ◦ With classmates ◦ Interaction with prof ◦ Missing studying on campus ◦ Making friends Responsibilities ◦ Added responsibilities ◦ Workload ◦ Course requirements Motivation ◦ Lack of motivation ◦ Engagement ◦ Escaping from reality Physical impacts ◦ Mental health ◦ Physical fatigue ◦ Screen time ◦ Health Issues

Information and communication overload ◦ In‐person communications ◦ Sympathy and help ◦ Second language ◦ Unclear communication of expectations ◦ Online discussions ◦ Too much information ◦ Feedback Courses ◦ Course materials ◦ Course e‐tools ◦ Difficulty level ◦ Mandatory participation ◦ Group work ◦ Grading and feedback ◦ Course limitations ◦ Course organization ◦ Lack of live lectures/synchronous sessions ◦ Self‐teaching ◦ Flexibility Organization ◦ Schedule and calendar ◦ Ability to organize oneself Teaching ◦ Quality of teaching Required Skills ◦ Technical skills Internet ◦ Internet issues Learning from Home ◦ Distractions

Since the focus of the open‐ended question was on gauging the experience of student participants, the Usage Code Category has two codes—positive experience and negative experience. During the coding process, several additional factors emerged around fatigue, mental and physical health, support and feedback, how courses were organized, delivered, and the modes of delivery, interactions with others (peers, instructors, teaching assistants etc.) and motivation. Since social constructivism at its core is about the social and peer influence on learning (Vygotsky, 1978), these concepts find room in this grounded theory approach and consequently in our Code Categories (Table 5).

Since the dataset is relatively small, 138 responses, many in the form of short sentences, many of the codes did not appear in great frequency, but nevertheless are included here to demonstrate the evolution of the thematic code category table. The next step was to conduct code sequence analysis between all the codes and the “positive and negative experience” codes. This procedure analyses the occurrence of every coded segment in comparison to the segments coded as “positive experience” or “negative experience”. To simplify our investigation, we describe our findings related to the negative experiences. Table 6 shows only those thematic codes that had z‐scores significant at either p <= 0.05 or p <= 0.10 levels.

TABLE 6

Code sequence analysis between positive and negative experience and other significant codes

Code A	Code B	Freq A	Freq B	Freq (B | A)	% of A	Freq (A | B)	% of B	z	Prob.
Unclear communication of expectations	Negative experience	21	117	11	52.4	8	6.8	−2.04*	0.065
Course organization	Negative experience	10	117	5	50.0	3	2.6	−2.64**	0.035
Feedback	Negative experience	5	117	4	80.0	2	1.7	−2.16*	0.089
Interaction with prof	Negative experience	16	117	7	43.8	5	4.3	−2.01*	0.079
Internet issues	Negative experience	11	117	4	36.4	3	2.6	−2.64**	0.035
Lack of live lectures	Negative experience	11	117	2	18.2	2	1.7	−3.07**	0.019
Mental health	Negative experience	36	117	16	44.4	13	11.1	−1.80*	0.081
Online discussions	Negative experience	8	117	3	37.5	1	0.9	−2.72**	0.049
Quality of teaching	Negative experience	41	117	22	53.7	18	15.4	−2.16**	0.040
Too much information	Negative experience	14	117	8	57.1	6	5.1	−2.48**	0.034
Interaction with classmates	Negative experience	15	117	6	40.0	4	3.4	−3.05**	0.014
Interaction with friends	Negative experience	5	117	2	40.0	1	0.9	−2.72**	0.049

p <= 0.05;

p <= 0.10.

We note from Table 6 the overwhelming responses around the broader dimensions of Information and Communication Overload to be statistically significant with negative experience like so: unclear communication of expectations →negative experience (z = 2.04, p = 0.065); too much information →negative experience (z = 2.48, p = 0.034). Regarding Technical Skills Required (a dimension we hypothesized and tested with our path analysis), we see that internet issues →negative experience was statistically significant with z = 2.64, p = 0.035 and online discussions which have a moderate bearing on the skills of students to deliberate topical issues in an online forum, → negative experience, was statistically significant with z = 2.72, p = 0.049.

We then grouped codes together. We see that interaction with the prof was associated with negative experience (z = 2.01, p = 0.079), interactions with classmates with negative experience (z = 3.05, p = 0.014), and interaction with friends with negative experience (z = 2.72, p = 0.049). The interactions with the prof appear to mirror the students' experience with the quality of teaching with negative experience (z = 2.16, p = 0.040), as do course organization (z = 2.64, p = 0.035) and feedback (z = 2.16, p = 0.089).

Looking at the above results, we note that from the student responses that several factors impacted their negative experiences with online learning. While some of these factors mirror our findings from the quantitative analysis described earlier around Information Overload, Communication Overload, and Technical Skills, several new factors have also emerged, particularly around interactions with professors, classmates, and friends, the inability to make new friends or have social interactions with friends, and also how the course has been organized and delivered and the sparse feedback they received from professors or TAs. When we perform code sequence analysis, it is customary to share a snippet of the coded segments which demonstrate the keywords/codes in context of the actual student responses. Appendix A summarizes a small sample of these coded segments, particularly around Information Overload, Communication Overload, and Technical Skills Required (in the case of the coded segments—Online Discussions and Live/Synch Lectures), the three indicators used in path analysis described earlier.

DISCUSSION

Discussion of the results

Our findings suggest that information overload and perceptions about required technical skills positively influenced perceived online learning difficulty. This in‐turn negatively influenced satisfaction with the online learning environment. Further, our qualitative findings found that participants frequently reported design factors (eg, teaching quality, too many online discussions, too much information, class format) significantly contributed to reported negative experiences. These findings are supported by the theory of cognitive load and related Cognitive Theory of Multimedia Learning (Meyer, 2009; Sweller, 1994) which both posit that limitations on humans' cognitive capabilities in turn limit our ability to process sensory information.

The fact that communication overload was not found to be associated with perceived difficulty was surprising. In the early days of the pandemic, the quantity of communications was identified as a possible negative influence on wellbeing (Ahmed, 2020), and it has been identified as a factor that negatively influences learning performance (Karr‐Wisniewski & Lu, 2010; Mansi & Levy, 2013; McFarlane & Latorella, 2002). However, the qualitative findings suggest instead that lack of or poor communication with professors was a source of negative experience, rather than the quantity of communications in themselves. Taken together with the significant influence of information overload and perceived technical capability, we interpret these findings to suggest that the online teaching format and implementation, rather than communication quantity, generated extraneous cognitive load, which caused the observed association with perceived difficulty (Sweller et al., 1998).

It is interesting that students identified that the lack of synchronous or lectures (live classes) as a source of major concern. Administrators asked instructors at our university to prepare asynchronous content in an effort to increase accessibility for students. However, respondents described this negatively. One possible explanation for this finding is that students did not experience social presence, a factor often associated with positive online learning experiences (Garrison et al., 1999; Kehrwald, 2008), in an asynchronous environment. Though synchronous teaching can be fraught with technical challenges (eg, poor internet connections), students nonetheless retain the experience of interacting with their instructor and colleagues. Furthermore, the qualitative findings are consistent with the importance of social and instructor presence and corroborate other prominent models of e‐learning success, such as that proposed by Garrison et al. (1999), as well as possible design factors. Future work may benefit by extending our model by investigating the role that social and instructor presence may play in such abrupt transitions to online learning. The biggest challenge for synchronous sessions (live classes) was the navigation of different time zones that students were in, in country and in different countries. So while many instructors offered synchronous lectures or sessions, not all students were able to participate and while these were recorded live sessions, it never provides the feeling of being socially present in the same space as other students and the instructor.

Implications for higher education

The COVID‐19 pandemic has posed a significant challenge to higher education, forced academic institutions across the globe to shift to emergency remote teaching, and dramatically changed the global higher education environment (Dwivedi et al., 2020; Ratten & Jones, 2021). Higher education, due to its reliance on international students and the pedagogy of experiential learning, was deeply affected by the pandemic and was forced to undergo a rapid digital transformation (Beech & Anseel, 2020; Brammer & Clark, 2020). The pandemic, together with the resulting changing student demands, the development and proliferation of new learning technologies, and the reduction in international students, have all accelerated the digital transformation of traditional higher education institutions (Brammer & Clark, 2020). Higher education is seeing significant structural changes happening, which are expected to continuously benefit higher education institutions even in the post‐pandemic era (Dwivedi et al., 2020). However, at the same time, these emergent and radical changes resulted in many academics learning new online teaching techniques with little or no training and minimal prior experience. Students also had to adapt to this new way of learning without being prepared for it. Consequently, many academics are overwhelmed by the efforts required for digital transformation, and students are dissatisfied with this unplanned emergency shift to online learning (Tzavara, 2021; VanLeeuwen et al., 2021). Education institutions are also cautious, especially when making decisions about the quality of students' learning experience.

The findings of this paper contribute to higher education by suggesting that shifting to online teaching is not simply duplicating the learning materials in the digital format. Instead, it involves redesigning students' learning experience systematically and re‐evaluating traditional pedagogical approaches in the online context (Tzavara, 2021). For example, delivering a course and communicating clearly and concisely could help students better organize their studies, reduce the perceived difficulty of online learning, and improve their learning experiences. Additionally, designing courses in a format enriched by social interactions (even virtually) may increase students' motivation to learn and reduce their mental stress, thus increasing their satisfaction with online learning. This research also emphasizes the necessity of establishing training programs and providing timely IT supports to help students address the technical challenges in the online learning environment.

Limitations

A limitation of our findings is that both our qualitative and quantitative methods are subject to selection bias. The survey was sent to the entire student body of our university's Faculty of Management, and a subset opted to respond to the survey, and only a subset of which opted to respond to the qualitative question, suggesting that we cannot assume a representative sample. Though this limitation is partially overcome by our choice to analyse the quantitative data with a structural equation model, it is nonetheless possible that our findings do not capture the views of apathetic students.

A second limitation is that our survey could not adequately account for all well‐known factors that are known to influence e‐learning success. For example, social factors (eg, social and instructor presence), which are suggested by the qualitative findings, were not hypotheses that we sought to investigate in our causal model and questionnaire. Additionally, many of these factors could be the result of decisions that individual instructors took when designing their courses, which could not be accounted for in our study. It is possible that social and instructor presence, social well‐being, or individual instructor design decisions are major factors in perceived difficulty during such transitions to online learning environments. This should be accounted for in future research.

CONCLUSION

By studying the transition to online learning environments during the Covid‐19 pandemic, educators in higher education can identify ways that they can improve their online offerings and implementations during future disruptions. This research was motivated by a desire to develop insight from what was otherwise a difficulty period in the history of our practice. By considering design factors that contribute to effective online delivery, educators in higher education can improve students' learning experiences, even during such difficult times.

CONFLICT OF INTEREST

The authors declare the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

ETHICS STATEMENT

This study was reviewed by our institutions research ethics board and the participants provided their explicit and informed consent to participate in this study.

Too much information	Negative experience
I have found that with the large influx of information and notifications due dates are very, very easy to miss. There are some cases where a class has too many notifications and it overshadows the notifications or due dates from other courses	It's not learning anymore, it's just completing tasks on a list without retaining any information
I learned nothing the past months. Too much to process	I feel as though I have learned less than half as much about the course materials than I would have had the classes been in person
Professors send too many messages that it is hard to even keep up with them	I found it difficult to learn online in a class with no lecture videos
There was a large dump of information—over 6 hours a week in readings for 1 class—not including time for assignments or studying for tests (being in 6 classes this is not feasible to manage easily)	Online school has created an environment of information landfill whereby there is too much material being dumped on students, with profs leaving little to no guidance for helping us digest (as we self‐teach majority of the material) what is valuable
I have found that with the large influx of information and notifications due dates are very, very easy to miss. There are some cases where a class has too many notifications and it overshadows the notifications or due dates from other courses	It's not learning anymore; it's just completing tasks on a list without retaining any information
Yes. How can we opt out of receiving messages about social events from student societies, news from the president, department news, the half‐dozen surveys I've received about screentime, etc? The course information is not the main cause of information overload—it's all the other things!	I spend so much time on my screens that if affects my sleep and my mentality and mental health. Having more than 10 hours a day on screens is psychologically diminishing. Professors send too many messages that it is hard to even keep up with them

Unclear communication of expectations	Negative experience
I don't want to say the specific class, but some due dates were not clearly indicated in the syllabus	There are multiple platforms professors use and it becomes difficult to keep track of
They sent emails weekly, touch based often while students were able to complete the work on our own schedules	I hope instructor could think more for students, we are both in a hard situation, not just you guys, we are human, not learning machine. Thank you for your patient
And if profs could post a message on Brightspace for what is due that week it would help keep things organized	It is extremely hard for people with specific learning disabilities such as ADHD
And if profs could post a message on Brightspace for what is due that week it would help keep things organized	It is extremely hard for people with specific learning disabilities such as ADHD
Please have every professor's require in one place and especially the deadline and request about the work we need to deal with	It is so easy to not hand something in simply due to the fact that I did not see it because there is an information overload

Lack of live lectures/synchronous sessions	Negative experience
Asynchronous is NOT the answer—I have learned more in my one synchronous class than in all of my other asynchronous classes combined	Online Learning is stressful and eventually taking away the joy of in‐person learning
Secondly, I truly feel that we should be seeing more from our professors. More in person zoom calls, rather than pre‐recorded lectures. There is a great lack of communication. Although teachers are evidently under some stress as well, it truly feels that us students are completely teaching ourselves right now	The University did a horrible job at responding to COVID and how you decided to implement online course was terrible. DO BETTER

Online discussions	Negative experience
I find making discussion posts for each class weekly more mentally tiring than just having a live class discussion	It's frustrating to feel so isolated from classmates

Internet issues	Negative experience
Internet speeds	The professors do not take into consideration the difficulties of online school on mental wellbeing of students
VPN service make me struggling always	It would be great if there was a way to do online learning without excessive amounts of screen time