Negative emotions, cognitive load, acceptance, and self-perceived learning outcome in emergency remote education during COVID-19.

Learning related emotions (LREs) are determinant for students' achievement both in face-to-face and online education. Research has also shown that LREs tend to affect technology acceptance which in turn affects learning outcomes as well. Today though, the negative psychological impact of the COVID-19 crisis and the sudden transmission to obligatory remote education might yield different functions of emotions and acceptance on learning outcomes. In this context, the current study seeks to model the relations between students' negative emotions, acceptance of (emergency) remote education, and self-perceived knowledge improvement. The suggested model was examined and validated on 116 university students that attended fully remote courses in Greece during the COVID-19 crisis. The results suggested that negative emotions of boredom and cognitive load are significant predictors of students' acceptance of remote learning components: i) online attending a lecture, ii) online communicating with professor, and iii) online collaborating with peers. Anxiety directly affected perceived knowledge improvement, boredom, and cognitive load; Boredom was also affected by cognitive load. In addition, acceptance of remote learning components indirectly affected perceived knowledge improvement mediated by learnability. Boredom was the strongest predictor of online attending a lecture and online collaborating with peers, while online communication with professor was the strongest predictor of learnability. The contribution of this study and the structural findings are further discussed in the paper.

Introduction

In the wake of the fully remote education due to the COVID-19 pandemic, much research has focused on the analysis of the psychological consequences on the students’ well-being and academic achievement. Recent evidence supports the increase of mental health issues among university students, like severe or moderate stress (Husky et al., 2020), depression and decreased emotional self-efficacy (Besser et al., 2020; Chen et al., 2020). In the context of remote learning, it is found that the case of the pandemic crisis and Emergency Remote Education (ERE) has affected some basic learning related emotions (LREs) like boredom and anxiety and caused several mood changes (Irawan et al., 2020). Research has also shown that the perception of negative emotions of boredom and anxiety is stronger in online courses (Stephan et al., 2019) compared to the traditional (face-to-face) mode, impairing students’ academic performance. As a fact, students participating in online courses tend to achieve lower performance scores compared to those participating in face-to-face education (Francis et al., 2019). To this end, investigating the effects of negative emotions towards online academic achievement might be of critical importance in the context of ERE, where education is not only fully remote, but also ‘sudden’ and obligatory (Bozkurt & Sharma, 2020). Moreover, students are challenged to attend online courses and collaborate with their peers and professors in a complex, fragile and emotionally affected educational era.

In the past years, several researchers attempted to model the relationship between learning related emotions like enjoyment, boredom, confusion, anxiety, etc., and student achievement both in traditional (face-to-face or blended) and in online education. In particular, stepping on Pekrun's Control-Value Theory (CTV) of achievement emotions (Pekrun, 2006) several studies have explored the relationship between student characteristics and emotions or the relationship between emotions and learning achievement (e.g., Ding & Zhao, 2020; Tempelaar et al., 2012). It should be noted that this study uses the terms of LREs and achievement emotions interchangeably. As explained in Pekrun’s CVT framework, achievement emotions have an indirect effect on learning achievement, via affecting attention, motivation, and self-regulation (Pekrun, 2006).

As a fact, understanding how emotions impact students’ attitudes toward different learning environments is becoming more and more vital today since educational institutions increasingly focus on student enjoyment and engagement in their learning activities. However, most of the previous studies examine the direct relation of emotions and achievement, whereas the students’ perceived learning gains (learnability) should also be measured (Azevedo & Marques, 2017; Loizzo et al., 2017).

Furthermore, other factors like technology acceptance have also been proved to significantly affect learning outcomes and in parallel be influenced by emotional states (Stephan et al., 2019). Although technology acceptance and attitude towards distance learning (including remote learning) have been extensively analyzed in past literature (e.g., Chung et al., 2020), the relation between learning related emotions, technology acceptance, and academic achievement is under-researched.

Overall researchers agree that the question of how students experience online-learning environments emotionally has not been sufficiently answered so far (Stephan et al., 2019). Although Pekrun’s framework of CVT proved to assign well in the context of online education, researchers point that there is a need to “further examine the assumptions of control-value theory while exploring or developing other theoretical models that might account better for the function of emotions in online learning environments” (Artino, 2012). Since most previous studies were conducted in the generic context of distance education and not in the context of fully (or emergency) remote education, today due to the COVID-19 crisis there is an even greater and emergent need to deeper examine the role of LREs and acceptance in remote education.

Motivated by the aforementioned research gap and the recent ERE challenge, this study seeks to model the relationship between i) negative learning related emotions, ii) acceptance of remote learning components, and iii) perceived achievement outcome in terms of learnability and knowledge improvement.

Although previous studies point that positive emotions are higher perceived in online educational settings than negative emotions (Stephan et al., 2019), this study focuses on negative LREs assuming their critical role in the distinct case of ERE and the emotional impact of the pandemic crisis on the educational community, including students, parents, and teachers. Furthermore, contrary to previous studies that to explore the application of Pekrun's CVT framework in different settings, this study does not analyze the causes that develop negative achievement emotions but focuses on their direct and indirect effects on students’ acceptance of remote education components and students’ learning outcomes.

So, this study examines the following research objectives (RO):

RO1: To examine the effects of students’ negative LREs on students’ acceptance of ERE components and on students’ perceived learning outcomes.

RO2: To examine the effects of students’ perceived acceptance of ERE components on students’ perceived learning outcomes.

The study is expected to contribute to research on remote education in three aspects. First, it can add to knowledge of the role of negative emotions in the context of ERE, assisting to the design of adjusted and appropriate psychological and pedagogical approaches. Such approaches can be used to emotionally train students in need and facilitate the process of transition to remote learning. Second, it can enrich our understanding of the role of negative LREs on the students’ acceptance of remote education and their learning outcomes. Third, by providing a clear insight on the role of student’s negative emotions in ERE, it can assist course designers and educators in applying the appropriate learning theories and online learning tools/media to manage the levels of boredom, anxiety and cognitive overload in ERE activities.

Literature review

Several researchers have shown that emotions are crucial determinants to the learning processes and have an impact on academic achievement. Learning-Related emotions (LREs) also called ‘achievement emotions’ have been emerged in the educational research context mainly after the development of the Control-Value Theory (CVT) (Pekrun, 2006) that categorized them according to valence (positive, negative) and the nature of response (activating or deactivating). The LREs of boredom (deactivating) and anxiety (activating) have been noted among as the basic negative emotions in CVT, while enjoyment (activating) and relief (deactivating) are the most frequently mentioned positive ones, as shown in Fig. 1.

Fig. 1

Two-dimensional taxonomy of learning related – achievement emotions, according to Pekrun’s (2006) Control Value Theory

The most frequently experienced emotions in educational contexts are enjoyment, frustration, boredom, confusion and anxiety. These emotions tend to be experienced in technology-mediated learning activities like for instance educational serious games and intelligent tutoring systems (e.g., D'Mello & Graesser, 2012; Schrader & Nett, 2018).

Interestingly, the factors that affect emotions in distance learning are different than those in the face-to-face learning modes (Regan et al., 2012). Research has shown that the relation between LREs and achievement might vary in different learning contexts, like for instance between online and face-to-face courses (Marchand & Gutier, 2012). Moreover, there are few differences in emotions experienced in online learning environments compared to face-to-face education (Daniels & Stupnisky, 2012a, b). For instance, Stephan et al. (2019) have found that students who attended an online course reported higher levels of boredom, anxiety, and anger, and lower levels of enjoyment compared to students who attend a traditional (face-to-face) course. From an educational perspective, positive LREs usually serve as a determinant of students’ emotional well-being (Hasnain et al., 2019; Phan et al., 2019; Stephan et al., 2019), while negative LREs like anxiety and boredom tend to produce maladaptive practices and negative learning outcomes for students (e.g., Obergriesser & Stoeger, 2020; Valiente et al., 2012).

In the distance learning context, there are mixed theories and results. For instance, Ding and Zhao (2020) have found that negative emotions like boredom and annoyance can negatively affect students’ engagement and self-perceived achievement, while positive emotions like excitement and enjoyment bring positive effects. However, their structural model does not consider anxiety and is targeted only towards self-perceive achievement and not any other learning-related outputs. Contrary to their findings, previous research study (Artino & Jones, 2012) found that the negative activating emotion of frustration can “emerge as a positive predictor of students’ metacognition”. However, most previous works mainly investigate the relationships between emotions, engagement, achievement, student characteristics, and other constructs. Most results indicate significant correlations between the examined constructs, highlighting the crucial role of LREs in students’ academic achievement. However, their research is conducted in the context of MOOCs or online/blended learning and not in the emerged situation of distance or remote education.

Researchers also agree that emotions are tightly linked to the technological context integrated in the learning approach or environment (Stephan et al., 2019). In general, students tend to experience emotions as response to the learning environment and technology itself (Daniels & Stupnisky, 2012a, b). As a fact, technology acceptance and students’ attitude might be an important determinant of LREs in the context of emergency remote education.

The traditional Technology Acceptance Model (Davis, 1989) investigates the effects of perceived ease of use and perceived usefulness on user acceptance in technology enriched environments. The next version of the model (Venkatesh & Davis, 2000) conceptualizes the effects of perceived ease of use and usefulness on the individuals’ generic attitude towards technology and their intention to use a particular technology. What is interesting is that the authors integrate in their mode a set of emotional attributes like perceived enjoyment, and playfulness as well as the factors of control (which is usually examined along with self-regulation in the context of online learning), self-efficacy and computer anxiety.

The TAM model has been broadly used in the context of online learning and distance education, where researchers attempt to quantify and measure students’ acceptance and attitude on their intention to participate or continue online learning (Chen et al., 2017; Joo et al., 2018; Romero Martínez et al., 2020; Ullah, 2017; Zhu et al., 2020). Other studies used TAM in combination to other scales and models (e.g., of digital competence) to examine students’ attitude towards distance education (Romero Martínez et al., 2020) and the factors that affect students’ continuous intention to participate in online learning activities (Zhu et al., 2020). However, most of these studies did not include LREs as factors affecting attitude and acceptance.

Although several works have used TAM to examine the acceptance of online and technology enriched teaching and learning (e.g., Scherer et al., 2019; Wong, 2015) in various educational settings, the integration and investigation of students’ emotional states is under-researched. In one research worth to mention as an example, Saadé and Kira (2006) explained the influential role of emotions on students’ technology acceptance, by showing that anxiety and enjoyment can influence the TAM constructs of perceived ease of use and usefulness in the context of an online learning environment.

In the context of COVID-19 and ERE, some researchers have already attempted to investigate the factors that affect students’ acceptance of remote learning (e.g., Brooks & Grajek, 2020; Chung et al., 2020; Aguilera-Hermida, 2020) or to evaluate the successfulness of the emerged remote course (Manalo et al., 2020; Samueli et al., 2020). Unfortunately, most of these works concluded that students face difficulties to accept remote education compared to previous face-to-face settings. However, similarly to the previously mentioned studies, most of these COVID-19 oriented works lack the inclusion of emotions in their list of examined constructs.

Measures selection and research hypotheses

Knowledge improvement has been measured by previous researchers through either trace data, e.g., quizzes (Butz et al., 2016; Kim et al., 2014) or self-perceived knowledge improvement (Ding & Zhao, 2020). This learning outcome construct has been broadly examined by researchers in the context of learning experience and the factors that affect them. Researchers also suggest that the students’ perceived easiness to learn and learning gains (learnability) needs to be separately studied from overall knowledge achievement, since it constitutes an important part of the online learning experience (Azevedo & Marques, 2017; Loizzo et al., 2017). For this, in this study we examine self-perceived knowledge improvement and learnability, considering them the targeted learning outcomes in distance learning.

Although some research suggests that students tend to perceive positive emotions higher than negative ones in online settings (Stephan, 2019), researchers in the field (e.g., Irawan et al., 2020) have pointed tight links between negative emotions and certain online learning components like course attendance and collaboration (e.g., boredom to attend, frustration to collaborate, etc.). As explained in the literature review, boredom, anxiety, and cognitive load (or mental fatigue) are considered critical negative LREs in the context of ERE during the COVID-19 ERE situation.

Overall, interaction, collaboration and communication with professors have been stressed as important components in online learning compared to ‘traditional’ face-to-face learning approaches (Keskin et al., 2020; Strauß & Rummel, 2020; Tanis, 2020). Based on that and concerning the basic learning activities in remote education, this study classifies remote education into three basic components: i) Online attending lectures, ii) Online communicating with professors, and iii) Online collaborating with peers.

As a conclusion, this study focuses on the following constructs:

Negative LRES: Boredom; Cognitive (over)Load; and Anxiety.

Acceptance of remote education components: Online attending lectures; Online communicating with professor; and Online collaborating with peers;

Perceived learning outcomes: Learnability; and Knowledge Improvement.

Next, we describe the negative LREs that we examine in the current study, providing evidence to form our research hypotheses.

Boredom and academic achievement

Boredom has been extensively analyzed by psychologists and philosophers in the past years, who attempted conceptualize and quantify the term (Sharp et al., 2020). For instance, Greenson (1953) defined it as a conflict between internal and external world, or as “a state of dissatisfaction and disinclination to action”. Other researchers (Barmack, 1939) noted it as “a state of conflict between the tendency to continue and the tendency to get away from a situation which has become unpleasant”.

The development of Pekrun’s Control-Value Theory (CVT; Pekrun, 2006; Pekrun et al., 2011) brought to light the significant role of academic boredom and several researchers proved its influence not only on academic achievement but also on student’s educational involvement, career planning, cultural participation, and emotional autonomy (Mann & Robinson, 2009). According to Pekrun’s theoretical framework, boredom and academic achievement tend to affect each other causally, with increased boredom leading to poorer academic achievement and poor academic achievement leading to increased boredom. As a fact, recent research on undergraduate students has proved that that bored students are the “less effective learners” (Sharp et al., 2020). However, research findings have also proved a positive correlation between boredom and achievement by e.g. motivating students’ creativity and ‘forcing’ them find ways to defend over their boredom and perceived frustration (Bench & Lench, 2018; Elpidorou, 2017).

Overall, boredom has only recently emerged as a critical factor to consider in the educational world (Parker et al., 2021; Knörzer et al., 2016) and despite previous findings its function in academic achievement and students’ behavior is under-researched in the context of distance or remote education.

Based on the aforementioned studies and in combination with the literature review on acceptance in distance education, our first research hypothesis (H1) is formed as follows:

H1: Students’ academic boredom indirectly affects their perceived learning outcomes through its direct effect on ERE acceptance.

Anxiety and academic achievement

Although there are contradictory findings, most studies consider anxiety to be a major predictor of academic performance. Studies have showed that students’ perceived anxiety can directly affect their learning experience and academic achievements (e.g., Ibukun, 2015; Vitasari et al., 2010). However, some other studies show that anxiety does not significantly predict online learning outcomes (Heckel & Ringeisen, 2019; Tempelaar et al., 2012).

Recently, anxiety has become a critical factor to consider in distance education as well, since researchers have pointed several components like online interaction with peers and professor, oral presentation using modern technology, etc. that tend to increase students’ perceived levels of anxiety (Ajmal & Ahmad, 2019). Moreover, recent evidence supports an increase of mental health issues like severe or moderate stress (Husky et al., 2020), depression and decreased emotional self-efficacy (Chen et al., 2020) among teachers and students. In fact, compared to traditional learning approaches, students seem to experience higher levels of anxiety in distance education (Ajmal & Ahmad, 2019). Academic anxiety though is not a single case as it can be reflected in different types of anxiety, like for instance computer anxiety, language anxiety, and social anxiety. Today it is known that all those types of anxiety tend to keep students away from online learning (Ajmal & Ahmad, 2019).

Recent research has shown that, experiencing online learning anxiety significantly affects academic achievement (Ajmal & Ahmad, 2019; Saadé et al., 2017) even though web-based and online learning might benefit some students (especially those with social anxiety or anxiety disorders) by decreasing the levels of perceived stress (Bektaş & Yardımcı, 2018; Keskin et al., 2020). One possible reason is the tight relationship between anxiety and the use of Internet or technology (computer anxiety) as explained in Saadé et al. (2017), as well as the fact that students’ computer related self-efficacy has a significant effect on their experienced anxiety in online learning environments (Heckel & Ringeisen, 2019; Hill et al., 2009; Saadé & Kira, 2009). Online feelings of anxiety have also been proved to be influenced by students’ demographic factors, previous online learning experience, and satisfaction with online learning environment (Abdous, 2019). Other research though has shown that the emotion of anxiety can also yield positive learning outcomes, mainly because it motivates students to avoid failure and put more effort in terms of attention (Knörzer et al., 2016; Pekrun, 2006).

In the times of the pandemic crisis, research has also shown tight links between online anxiety, fear of COVID-19 and problematic Internet use (Seyed et al., 2020). Despite the increased interest in examining the effects of COVID-19 on student’s psychological health including anxiety issues (Besser et al., 2020; Chen et al., 2020), little is known about the relationship between the COVID-19 increased anxiety even in online activities (Abdous, 2019; Elhai et al., 2020; Lee & Lee, 2019) and the students’ experience or academic achievement within the context of emergency remote education. Such knowledge is critical to design instructional material in a way that defuses the students' feelings of anxiety and eases the transition from traditional to remote learning orientations (Abdous, 2019).

Based on the aforementioned studies and in combination with the literature review on acceptance in distance education, our second research hypothesis is:

H2: Students’ online learning anxiety directly affects other negative LREs (boredom and cognitive overload (H2a) and perceived outcome (H2b)

Cognitive load and academic achievement

Cognitive overload happens when students receive a large amount of information that is complicated or difficult in terms of time to be adopted (Sweller et al., 1998). Researchers agree that perceived cognitive overload or else called ‘mental fatigue’ also reflects a learning related emotional state since it can significantly affect engagement and performance however, low levels of cognitive load are shown to cause boredom (Atiomo, 2020). In the context of distance education cognitive overload has been linked to mental fatigue, a phenomenon called ‘zoom fatigue’ (BBC, 2020) that explains how and why we tend to feel mentally tired in video conferences.

Ample research has conducted on the relation between cognitive load and learning outcome. The cognitive load theory is considered as one of the most important educational theories for teachers (Atiomo, 2020) since it provides them with an explanatory framework on how information can be presented to students to optimize their mental performance (Sweller et al., 1998).

The theory explains how new information is partially and progressively stored in short (working) and long-term memory during the learning process. In simple words, if the information provided exceeds the limited capacity of the working memory, then this new information is impossible to be stored in the long-term memory (which is unlimited (Sweller et al., 2019)) and be available for future retrieval. This way, cognitive overload might impair academic performance.

Cognitive overload in online learning activities has been proved several times in the past (e.g., Chen et al., 2011; Vonderwell & Zachariah, 2005) to influence students’ acceptance, achievement, and participation in online learning. Research evidence also support the relation between students’ emotional wellbeing and cognitive load, arguing that anxiety can lead to mental fatigue and impair cognitive performance, including the processes of working memory and allocation of attention resources (Atiomo, 2020; Sweller et al., 2019).

Based on the aforementioned studies and in combination with the literature review on acceptance in distance education, our third research hypothesis is:

H3: Students’ cognitive load directly affects boredom (H3a) and indirectly affects perceived learning outcomes through its direct effect on ERE acceptance (H3b).

Figure 2 shows the conceptual research model on the relations between negative emotions, acceptance, and perceived learning outcomes. As shown, the negative LREs (on the left) bring both direct (the case of anxiety) and indirect effects on perceived learning outcome, through their direct effects on the construct of acceptance. The direct affects between the LREs themselves are also shown, as well as the direct effect of perceived learnability to perceived knowledge improvement.

Fig. 2

Conceptual research model: Relationships among emotions, acceptance, and perceived learning outcomes during emergency remote education

Methodology

Measurement model

A 31-item instrument was developed and validated to measure the examined variables of negative emotions, acceptance, and perceived learning outcomes in ERE. All items were measured in a 7-point Likert scale and bipolar format from “traditional learning” to “remote learning”, where students had to select the level, they agree on the given statement about the emerged remote learning method relative to the previous (traditional) one. All observed items were carefully reviewed by two experts in the field of Affective Computing and Technology Enhanced Learning (TEL) to eliminate any typos, complex and difficulty perceived expressions. The negatively phrased items were reversed during the data processing procedure. Since all constructs are measured in the context of remote education, they are all characterized by their ‘online versions’. For instance, the emotion of boredom reflects ‘online boredom’, anxiety reflects ‘online-anxiety’ and cognitive load reflects ‘online cognitive load’. Similarly, the constructs of perceived learning outcomes can be expressed as ‘online learnability’ and ‘online knowledge improvement’. The questionnaire structure is presented in Table 3 in Appendix.

Table 3

Questionnaire items used

Construct	Item(s)
Negative Learning Related Emotions (Online)		When I attend a lecture, I usually feel that:
Boredom	BR1	I could do something more interesting instead attending the lecture
	BR2	I have better things to do
	BR3	I am bored
	BR4	My attention is easily distracted on other tasks
	BR5	I have difficulty to concentrate on the course
		I could do something else more productive for me
Cognitive Load	CL1	I have difficulty to recognize or find the linkage in significant information
	CL2	I am confused
	CL3	I think too intensively/in an exhausting way
		I have difficulty to remember significant information/concepts/terms
	CL4	I have difficulty to get organised and work on the task that the professor assigns during the lecture
Anxiety	AN1	I am anxious when I attend lectures
	AN2	I am anxious when I communicate with the professor
		I am anxious when I collaborate with peers (co-students)
Acceptance of Remote Education Components		(aggregated items per ERE component)
Online Attending Lectures	OAL1-OAL4	It is easy to attend lectures; It is useful to attend lectures; I enjoy attending lectures; I have control when to attend lectures
Online Communicating with Professor	OCP1-OCP4	It is easy to communicate with professor; It is useful to communicate with professor; I enjoy communicating with professor; I have control when to communicate with professor
Online Collaborating with Peers	OCP1-OCP4	It is easy to communicate with peers; It is useful to communicate with peers; I enjoy communicating with peers; I have control when to communicate with peers
Perceived Learning Outcomes (Online)
Learnability	LR1	I can easily learn when I attend lectures
	LR2	I can easily learn when I communicate with the professor
	LR3	I can easily learn when I collaborate with peers (co-students)
Knowledge Improvement	KI1	I can easily develop/enhance my knowledge
	KI2	I can easily perceive the main concepts/information introduced by the professor during the lecture
	KI3	I try to apply the knowledge I gained in other tasks/projects
	KI4	I want to further develop my skills and learn even more about the course (which I attend)
	KI5	I have difficulty to understand and practically apply important information (on the course content)

Measures of learning related emotions

Boredom was measured via six items, based on the instrument of Van Tilburg and Igou (2012) that was originally written to evaluate individuals' distinctive experiences of boredom, and on the instrument of WHO (2003) that developed a self-report scale for adults to measure attention deficit symptoms/behavior. The items were adjusted to the context of remote learning and their wording was carefully reviewed by the two experts. In particular, the items boredom was “When I attend a lecture, I usually feel that: I could do something more interesting instead attending the lecture; I have better things to do; I am bored; My attention is easily distracted on other tasks; I have difficulty to concentrate on the course; I could do something else more productive for me”.

Cognitive load was measured via five items, according to the Cognitive Load Theory (CLT; Sweller, 2010; Sweller & Chandler, 1991) and the measurement model developed and validated by Klepsch et al. (2017). The items were “When I attend a lecture, I usually feel that: I have difficulty to recognize or find the linkage in significant information; I am confused; I think too intensively/in an exhausting way; I have difficulty to remember significant information/concepts/terms; I have difficulty to get organised and work on the task that the professor assigns during the lecture”.

To measure anxiety in the context of remote education we decided to use a set of single-item questions per component. This would enable our research to be focused on the remote components regardless all types of perceived anxiety and to examine direct influence on academic achievement, according to the literature. Hence, anxiety was divided in three items corresponding to the three components of remote education included in the model: “I am anxious when I attend lectures; I am anxious when I communicate with the professor; I am anxious when I collaborate with peers (co-students)”.

Measures of acceptance

Stepping on previous studies that examined attitude and acceptance towards distance education, as well as the later version of TAM (Venkatesh & Davis, 2000), we decided to measure acceptance using a single TAM construct for every ERE component. Towards this direction we used four single items of perceived ease of use, usefulness, enjoyment, and control for every component, respectively. For instance, for the first component of online attending lectures the items were: It is easy to attend lectures; It is useful to attend lectures; I enjoy attending lectures; I have control when to attend lectures”.

Measures of perceived outcome

Like anxiety, perceived learnability was measured via three items as regards to online attendance, online communication with professor and online collaboration with peers.

Knowledge improvement was measured via five items, based on previous questionnaires on e-learning (Chu et al., 2006) and game-based learning (Fu et al., 2009) perceived knowledge improvement via the following: “I can easily develop/enhance my knowledge; I can easily perceive the main concepts/information introduced by the professor during the lecture; I try to apply the knowledge i gained in other tasks/projects; I want to further develop my skills and learn even more about the course (which I attend); I have difficulty to understand and practically apply important information (on the course content)”.

Context, participants, and procedure

The study was conducted on a sample of 116 (73 female, 43 male) undergraduate and postgraduate university students in a Greek university who attended remote lectures during the COVID-19 pandemic crisis in 2020. By the time of the survey the students had already attended ERE courses for almost two months (February to April).

To invite participants in the survey an online questionnaire in Greek language was sent out via emails and the e-learning platform notification system to the students enrolled in one Bachelor programme of Economics and two Master programmes of Information Systems and Law & Finance. The programme courses were conducted in a fully remote mode. All lectures were conducted synchronously via the Google Meet video conferencing platform and collaboration/communication with peers and professor could take place both synchronously during the lectures, or asynchronously via email and the e-learning platform.

Data was voluntarily and anonymously collected by the students according to the ethical standards of the institutional research committee. Also, participants were informed about the study objectives and the manipulation of their data and were asked to consent for their participation before proceeding to the survey.

Data analysis

After conducting a normal distribution test, we found that the collected data did not follow normality (Shapiro & Wilks, 1965). The partial least squares structural equation modeling approach (PLS-SEM) was selected for multivariate analysis since it is known that PLS-SEM does not require normally distributed data and is suitable for small or complex data models (Hair et al., 2014). According to researchers (e.g., Cin, 1998; Hair et al., 2011; Peng & Lai, 2012) the sample size for a PLS study should be at least ten times larger than the largest number of independent variables impacting a dependent latent variable. Our sample size exceeds the recommended lowest value of 30 since in our model, the largest number of independent variables impacting a dependent variable (Learnability) is three.

In terms of software, we used the SmartPLS 3 to perform the PLS-SEM for the measurement and the conceptual model, while SPSS software was used to estimate the normality of the data.

Results

Evaluating the measurement model

The measurement model was evaluated in terms of reliability of measures, convergent and discriminant validity. The reliability of measures was evaluated by Cronbach alpha with its minimum value of 0.60 (Hair, 2006) and the composite reliability with its minimum value of 0.70 (Gefen et al., 2000). To assess convergent validity, we calculated the average variance extracted (AVE) with its minimum value of 0.50 (Bagozzi & Yi, 1988). The discriminant validity was assessed by the square root of AVE and latent variable correlations (Fornell & Larcker, 1981).

As shown in Table 1, the results prove the reliability and validity of the measurement model since all measures meet the required threshold.

Table 1

Reliability, validity, and internal consistency results for the measurement model

	Cronbach's Alpha	Composite Reliability	Average Variance Extracted (AVE)
Anxiety	0.801	0.883	0.717
Boredom	0.922	0.939	0.72
Cognitive Load	0.88	0.912	0.676
Knowledge Improvement	0.863	0.907	0.709
Learnability	0.889	0.931	0.818
Online Attend Lecture	0.919	0.943	0.806
Online Collaborate with Peers	0.915	0.94	0.796
Online Communicate with Professor	0.907	0.935	0.782

We also examined the discriminant validity of the model. Discriminant validity is the extent to which the construct is empirically distinct from other constructs (Hair et al., 2014). In this study, the discriminant validity is assessed by the square root of AVE and latent variable correlations (Fornell & Larcker, 1981). As shown in Table 2, the discriminant validity is also verified for the suggested measurement model.

Table 2

The discriminant validity of the measurement model

	Anxiety	Boredom	Cognitive Load	Knowledge Improvement	Learnability	Online Attend Lecture	Online Collaborate with Peers	Online Communicate with Professor
Anxiety	0.847
Boredom	−0.469	0.849
Cognitive Load	−0.423	0.749	0.822
Knowledge Improvement	0.690	−0.583	−0.509	0.842
Learnability	0.746	−0.599	−0.538	0.737	0.905
Online Attend Lecture	0.776	−0.567	−0.527	0.718	0.711	0.898
Online Collaborate with Peers	0.734	−0.408	−0.383	0.595	0.675	0.662	0.892
Online Communicate with Professor	0.789	−0.484	−0.481	0.706	0.759	0.768	0.690	0.884

Evaluating the conceptual model

The hypothesized conceptual model showed acceptable goodness of fit (Standardized Root Mean Residual [SRMR] = 0.073, Normed Fit Index [NFI] = 0.682 and chi-square = 1.620.081) according to the model fit acceptance criteria (Hair et al., 2014). Bootstrapping results proved that all relationship paths in the model were significant (p < 0.05), except the path from cognitive load to online collaboration with peers. Results also showed that all indirect relations were significant as well (t > 1.96, p-value < 0.05).

The validated model explained 58% of the variance in perceived knowledge improvement and 63% in perceived learnability. Negative emotions of boredom and cognitive load were found to significantly affect most acceptance components and indirectly affect learnability and perceived knowledge improvement. The two types of learning outcomes (learnability and knowledge improvement) were predicted by all acceptance constructs, while anxiety directly predicted knowledge improvement accounting for 40% of the variances. Cognitive load was a strong predictor for boredom, while anxiety revealed significant effects both on cognitive load and boredom.

Overall, the strongest direct predictors for knowledge improvement were learnability and anxiety. Moreover, positive attitude of online communication with professor proved to influence learnability more than the other two components of online lecture attendance and online collaboration with peers. Another significant result was the strong relation between cognitive load and boredom.

Figure 3 indicates the standardized path coefficients in the direct paths of the model, extracted after the PLS calculation.

Fig. 3

Relations between negative emotions, acceptance, and perceived learning outcome in remote education. Note. Thicker arrows represent stronger relationships

Discussion

The suggested research hypotheses are all confirmed, and a more detailed representation of the relationships has been presented in the results section. With respect to the research hypotheses H1 and H2, this model showed that acceptance of remote education is predicted by the negative LREs of boredom and cognitive load. This finding is in accordance with previous studies, concluding that cognitive load has a direct impact on the students’ sense of boredom (Atiomo, 2020).

The results also showed that experiencing boredom can affect all three components of online attending lectures, online communicating with professor and online collaborating with peers. This finding aligns well with previous literature where researchers explained how academic boredom affects educational involvement (Mann & Robinson, 2009) and disengages students from active participation or collaboration activities. The reason for this disengagement is also reflected on the primary terms assigned to boredom describing it as a “state of disinclination to action” or a “desire not to get involved in unpleasant situations” (Barmack, 1939; Greenson, 1953). Overall, design courses that are not ‘unpleasant, require effort or attention’ and yield students’ feelings of enjoyment will be expected to reduce their sense of boredom in remote learning.

Cognitive load proved to predict acceptance of only two components, without affecting online collaboration with peers. This might be explained by the fact that cognitive load is mainly correlated with individual learning (Kirschner et al., 2011) and collaborative learning tends to reduce cognitive load. This probably brings no significant effects on students’ acceptance and learning outcomes in the remote educational context. Researchers explain that during collaborative learning the cognitive load of the learning task can be subdivided across peers and information is provided exactly on the time it is asked without causing overload or mental fatigue to the students (Kirschner et al., 2011, 2018). This finding is important to be considered in the context of ERE and the COVID-19 crisis, when video conference meetings are shown to increase individuals’ mental fatigue, a phenomenon called ‘zoom fatigue’ (BBC, 2020). According to the findings of the current study, encouraging students to collaborate or design approaches of remote collaborative learning could assist students in efficiently managing their learning tasks without being cognitively affected.

Regarding H3, results proved that anxiety affects cognitive load and cognitive load is an important predictor of boredom also in the case of ERE, similarly to other learning contexts (e.g., face to face or blended). This finding is important to be considered by educational institutions and remote course designers, since students tend to experience higher levels of boredom and anxiety in online learning than traditional learning, (Stephan et al., 2019) negatively affecting their academic achievement.

Finally, as assumed, anxiety brings a strong and direct effect on the learning outcome, and particularly on perceived knowledge improvement. This aligns well with previous literature (Ibukun, 2015; Vitasari et al., 2010) noting that anxiety is a major and direct predictor of academic achievement in most of the learning contexts. What is interesting is the positive correlation between anxiety and knowledge improvement, implying that the sense of anxiety in this context of ERE by motivating students to increase their effort to avoid failure (Knörzer et al., 2016; Pekrun, 2006). This finding is reinforced by the negative relationship between anxiety and boredom, implying that low levels of anxiety might cause boredom increment, negatively affecting acceptance and hence learning outcomes in remote education. An explanation of these relations can be the fact that students’ fear of loss of academic year has significantly increased during the COVID-19 situation (Hasan & Bao, 2020), and this fear seems to motivate them to study harder not to fail. Another explanation is the possible medium level of perceived anxiety.

The positive relationship between anxiety and knowledge improvement can also be explained by the findings of Artino and Jones (2012) who stated that a negative activating emotion (for-instance frustration) can “emerge as a positive predictor of students’ metacognition”. In our case, that negative activating emotion is anxiety. Not surprisingly, research in distance education has showed that low or moderate anxiety is positively correlated with academic achievement (Shibli, 2015). Factors that tend to cause serious academic anxiety yielding negative performance usually are of family, society, or political reasons (Rehman, 2016). Due to the social dimension of the COVID-19, we assume that serious academic anxiety should be considered and measured. For this, we recommend that further research needs to be conducted since the current study did not consider different levels of perceived anxiety.

Finally, this study also found that acceptance of remote education components affects perceived knowledge improvement through learnability. To be specific, learnability was positively affected by all three components of remote education acceptance, and in turn learnability strongly and directly affected perceived knowledge improvement. Considering that one basic reasons of student’s low performance in distance education is the lack of direct communication with professor and the instant provision of help during their learning tasks (Francis et al., 2019), it seems that the component of online communication with professor brought the strongest effect on learnability.

Practical implications and limitations

Overall, the findings of the present study have implications for designing and building more effective remote education components and pedagogical strategies, by considering the effects of the negative emotions that students experience during ERE and COVID-19. Stepping on recent researchers’ statement that there is no empirical data on which to base any strategies to improve academic attainment, by reducing cognitive load and negative emotions during the COVID-19 pandemic (Atiomo, 2020), this study attempted to provide with some meaningful insights that could be considered by designers of courses and teaching strategies.

For instance, as already suggested, approaches of collaborative learning and techniques to reduce boredom, and increase effort/attention should be implemented. A strategy to increase students’ engagement and collaboration during remote education is the application of constructive learning theories (e.g., Bates, 2019). For instance, encouraging students to participate in constructive activities and forum discussions can possibly yield positive emotions and active participation in ERE.

Also, game-based learning has been proved to positively affect students’ emotions and increase engagement. Several researchers have studied the effects of game on learners’ emotions (e.g., Plass, 2020), while Hernández-Ramos and Belmonte (2020) showed that when game-based learning approaches are implemented in DE, students tend to achieve higher performance scores than students attending a face-to-face class.

Cognitive overload can easily happen in the context of remote education, due to the variety of teaching and learning media (videos, text, activities, blogs, etc.) that are available online. For this reason, educators and course designers should carefully design the online courses and choose only the appropriate kinds of media to transfer knowledge and avoid the use of unnecessary rich media that might cause cognitive overload to remote students. As Bates (2019) suggests, rich media might be inappropriate for some teaching subjects, and using rich media in teaching might negatively affect the students’ attention and concentration.

Moreover, course designers should consider the basic components of remote learning, attempting to individually design each one of them according to the students’ emotional needs and the expected learning outcomes. Considering the basic TAM constructs, the remote education components should be designed in a way to achieve students’ acceptance, e.g., by being perceived as easy, useful, controllable, and pleasant.

The construct of learnability should be highly considered as a mediator between emotions, acceptance, and knowledge improvement and course designers should apply principles to enhance the learnability provide by every component separately.

Finally, the positive effect of anxiety on perceived learning outcomes brings to light important (re)considerations regarding the severity of academic anxiety in students’ during COVID-19, recommending reevaluation in different populations.

It should be mentioned that this study brings some limitations. First, data was collected via self-reported measures and this technique might be prone to bias and validity of the research. Second, all participants were of homogenous profile in terms of their socio-demographic characteristics and the experimental study was conducted in the context of remote courses. Future research that uses concrete data (e.g., grades, performance scores) or more robust data collection methods and conducted in different populations of students need to be conducted in the same context of ERE to verify the findings. Moreover, future research could further examine the positive association between anxiety and learning outcome by for instance examining the levels of perceived anxiety or incorporating more variables in the model like the students’ motivation to learn, their studying effort and their fear of failure in the context of ERE.

Conclusions

In this study we modeled the relationship between negative learning related emotions, acceptance and perceived outcomes of learnability and knowledge improvement, in ERE. Students’ feedback was collected during the COVID-29 pandemic crisis, when all students had to obligatory attend fully remote university courses. As previous research on emotions and achievement did not include acceptance items and did not distinguish between perceived learnability and knowledge improvement, we hypothesized that negative LREs and acceptance items would both predict self-perceived learning outcomes in the context of distance education.

The findings of this study confirmed previous research but also highlighted the main aspects that need to be considered in the context of ERE. Overall, our findings revealed that negative LREs of boredom, cognitive overload and anxiety bring direct effects on acceptance and indirect effects on learning outcomes. However, perceived anxiety proved to directly affect knowledge improvement as well. This study also showed the inter-relationships between the examined LREs. For instance, cognitive load was proved to significantly affect boredom in distance learning, and boredom proved a strong predictor of acceptance across all the three ERE components of online attending lectures, online communicating with professor, and online collaborating with peers. Interestingly, the students’ anxiety revealed a positive relationship to perceived knowledge improvement and a negative effect of cognitive load. This can be explained in future research where the levels of perceived anxiety can be associated to learning outcomes to reveal whereas online learning anxiety is of medium levels or it tends to motivate students to increase their effort. For this reason, it might yield productive results and positively affects the learning outcomes in the context of COVID-19 ERE.

Finally, the results of this study revealed the essential role of perceived learnability in the context of ERE since it proved to be affected by all the measured acceptance components and directly affect knowledge improvement. As proved, learnability was affected by acceptance and had a direct positive impact on knowledge improvement.

Overall, the strengths of the examined relationships imply that emphasis should be given on the emotions of boredom and online communication with professor, since they seem to significantly affect the students’ perceived learning outcome.

The main limitation of this study is the self-reported measurement it used, and hence future research is encouraged. Overall, this study contributes to the better understanding of the role of negative emotions and acceptance on the students’ learning outcomes of learnability and knowledge improvement during the COVID-19 emergency remote education.