Higher education students' achievement emotions and their antecedents in e-learning amid COVID-19 pandemic: A multi-country survey.

The outbreak of the COVID-19 pandemic has had a wide range of negative consequences for higher education students. We explored the generalizability of the control-value theory of achievement emotions for e-learning, focusing on their antecedents. We involved 17019 higher education students from 13 countries, who completed an online survey during the first wave of the pandemic. A structural equation model revealed that proximal antecedents (e-learning self-efficacy, computer self-efficacy) mediated the relation between environmental antecedents (cognitive and motivational quality of the task) and positive and negative achievement emotions, with some exceptions. The model was invariant across country, area of study, and gender. The rates of achievement emotions varied according to these same factors. Beyond their theoretical relevance, these findings could be the basis for policy recommendations to support stakeholders in coping with the challenges of e-learning and the current and future sequelae of the pandemic.

Introduction

Pandemics, and the subsequent closure of educational institutions, have the potential to affect higher education (HE) students physically, academically, financially, and psychologically (Kecojevic et al., 2020). It is likely, therefore, that the COVID-19 pandemic has impacted their emotions related to learning, i.e., their achievement emotions (Pekrun, 2006, 2018; Pekrun & Perry, 2014). In parallel, the pandemic has radically changed the instructional context, forcing most of the students worldwide to attend lessons through e-learning. Accordingly, understanding this impact on students' achievement emotions can inform the design of e-learning environments such that they support students’ emotional wellbeing and performance (Mayer, 2020).

Using the control-value theory (CVT) of achievement emotions as the main theoretical framework (Pekrun, 2006, 2018; Pekrun & Perry, 2014), we investigated how a sample of more than 17000 HE students from 13 countries felt in relation to academic e-learning during the first wave of the COVID-19 pandemic. Such situation has forced academic institutions to abruptly accelerate the implementation of e-learning worldwide, offering the exceptional opportunity to study how a very large amount of students feel while learning in online contexts. The knowledge about correlates of emotions in these contexts is the base for guiding future informed choices to shift from on-site to online modalities and vice versa, taking into account their benefits and shortcomings.

Impact of the COVID-19 pandemic on higher education students

Pandemics change life in general not only through their direct effects on physical health but also by impacting an individual's sense of vulnerability, loss, fear, and stress (Aristovnik et al., 2020b; Cao et al., 2020; Horesh & Brown, 2020). In the case of COVID-19, the general stress can be broken into various components. First and foremost is the risk of death from infection, be it the death of the individual or a loved person (Son et al., 2020). Moreover, changes in the psycho-social milieu are a source of stress. Academic routines are essential coping mechanisms for young people; hence, a change means losing an anchor that had previously helped them deal with everyday stressors (Cao et al., 2020). Variations and delays in academic activities are positively associated with anxiety symptoms (Lee, 2020). Some findings revealed that after the outbreak of the COVID-19 pandemic 25% of students suffered from severe anxiety (Cao et al., 2020), 83% experienced dire situations, and 26% were unable to access mental health support (Lee, 2020). Others indicated that 44% of the students had depressive thoughts and 5% suicidal thoughts; besides, many experienced difficulties in concentrating or had disrupted eating and sleeping patterns (Son et al., 2020). Mental health issues can significantly impair students' academic success and social interactions, thus affecting both their future career and personal life. However, also for those students who are more resilient and do not show impairments in their mental health, normal reactions to the pandemic can include more frequent and more intense negative emotions, also concerning their daily activities as those related to studying and learning. Therefore, the impact of the pandemic on students’ achievement emotions related to e-learning is a phenomenon worth further investigation, to inform policies and practice to better support students while learning through technological environments. Hence the study of the socioemotional implications of online learning and related coping strategies has become an important area of research (Katzman & Stanton, 2020).

During the first wave of the pandemic, a substantial number of descriptive and cross-sectional studies focused on remote teaching and learning processes concerning HE. Only a few of these studies were conducted using a cross-cultural approach involving a large number of different countries. These studied mainly the perceptions of the teachers’ rather than those of the students (Bond et al., 2021). Nevertheless, to our knowledge, issues concerning achievement emotions have been largely neglected.

The control-value theory of achievement emotions

While the study of emotions has a long history, research on how they affect performance in a variety of learning contexts have flourished only in the last two decades (Pekrun, 2018), with the two notable exceptions of studies of attributional theory (Weiner, 2018) and test anxiety (Zeidner, 1998). It is imperative to explore the links between e-learning and students' emotions to explain the cognitive-affective processes of e-learning (Mayer, 2020). In light of the increased use of technology for learning, investigating which antecedents foster emotions, in turn associated with performance, enables to identify viable ways to better support students’ learning and wellbeing in that specific context. Therefore, recent research is paying increasing attention to this (Loderer et al., 2020).

Pekrun’s (2006, 2018; Pekrun & Perry, 2014) CVT of achievement emotions offers an integrative approach for describing the interrelations between emotions, their antecedents, and their outcomes. The relative universality of these links among different contexts, which constitutes one of the basic assumptions of the CVT, makes it a privileged perspective for studying emotions in both traditional and e-learning environments. Achievement emotions are defined as those emotions which are felt in relation to learning activities and outcomes. They can be characterised by two underlying dimensions: valence (positive, negative emotions) and activation (activating, deactivating emotions). Thus, emotions can be grouped into four categories, i.e., positive activating emotions such as enjoyment, positive deactivating emotions such as relief, negative activating emotions such as anxiety, and negative deactivating emotions such as boredom. Research has revealed that positive activating, negative activating, and negative deactivating emotions are particularly salient for technology-based learning contexts (Duffy et al., 2020; Harley et al., 2020; Jarvenoja et al., 2020; Loderer et al., 2020; Mayer, 2020). It is worth noting that the CVT also provides a focus for examining the reciprocal links between constructs that can be considered as the antecedents or outcomes of the emotions (Pekrun et al., 2017).

Antecedents of achievement emotions

The CVT differentiates between proximal individual antecedents of achievement emotions, which impact them directly, and more distal individual or environmental antecedents, which impact them through the mediation of the proximal antecedents (Pekrun, 2006, 2018; Pekrun & Perry, 2014). Among proximal antecedents, an important role is played by individuals' beliefs about control, which include “appraisals of control over actions and outcomes” (Pekrun & Perry, 2014, p. 124). Self-efficacy expectations, related to a person's level of conviction that he or she can perform a task successfully (Bandura, 1997), can be conceptualized as prospective control appraisals (Pekrun & Perry, 2014). When students feel in control of their achievements, and the results of their efforts are positive, the sense of control is an additional source of positive emotion and promotes increased interest in further learning (Moors et al., 2013). Empirical evidence indicates that high control determines high levels of positive achievement emotions associated with technology-based learning environments, while low levels induce negative emotions (Hall et al., 2006; Lehman et al., 2012; Loderer et al., 2020; Moos & Azevedo, 2009; Niculescu et al., 2015; Perry et al., 2001; You & Kang, 2014). Specifically, computer self-efficacy, which concerns students' confidence in their abilities to use computers and similar devices (Sharma et al., 2007), is associated with decreases in related anxiety (Saadé & Kira, 2009) and is positively linked to students' emotional and cognitive engagement (Pellas, 2014). It differs from e-learning self-efficacy, which relates to students' confidence about their abilities to learn through e-learning (Sharma et al., 2007). Moreover, also e-learning self-efficacy seems to be positively related with perceived satisfaction, for example in online learning management platform, and performance (Liaw & Huang, 2013; Sharma et al., 2007).

Distal antecedents comprise individual antecedents such as gender, and environmental antecedents, which include a diversified group of factors characterizing the learning tasks and situations, among which are the cognitive and motivational qualities of the tasks. Cognitive quality refers to factors such as task structure and clarity, or task demands; Among other indicators, the motivational quality can refer to messages about the learning context conveyed by relevant people such as teachers or peers. These antecedents have rarely been examined for e-learning, but there are exceptions. For example, some findings indicated that instructional support and task characteristics influence emotions related to online learning (Artino, 2009). Moreover, focusing on web-based learning, Chen et al. (2013) found that both the perceived quality of the systems and the social influence (i.e., the perceived peers and teachers' esteem) were associated with increased self-efficacy; positively with achievement emotions such as enjoyment and negatively with anxiety. When investigating these constructs, it is particularly relevant to consider students' perceptions (rather than more objective factors), given their role in influencing students’ approaches to learning (Kember & Leung, 1998).

Consequences of achievement emotions

According to CVT, achievement emotions impact performance through the mediation of motivational, cognitive, and regulatory factors. Empirical data from cross-sectional and longitudinal studies worldwide suggest that, overall, positive emotions are associated with performance increases, while negative emotions diminish performance, albeit there are exceptions (Loderer et al., 2020; Pekrun, 2006, 2017, 2018; Pekrun et al., 2017; Pekrun & Perry, 2014; Shao et al., 2020). Positive activating emotions, such as enjoyment, hope, and pride, usually affect students and their performance positively; strengthening attention, interest, motivation, effort, self-regulation of learning, the use of flexible learning strategies, and the availability of cognitive resources for assessment purposes. Negative deactivating emotions, such as hopelessness and boredom, typically have an opposite impact on achievement; compromising attention, motivation to learn, and learning strategies. Positive deactivating emotions, such as relief, and negative activating emotions, such as frustration, anger, anxiety, and shame, can undermine attention, but sometimes they also have beneficial effects, for example, when frustration is associated with an internal rather than an external locus of control (e.g., as in the case of a bug) in technological contexts and therefore can prompt people's efforts to solve a problem (Mentis, 2007; Pekrun, 2018).

Loderer et al. (2020) conducted a meta-analysis of studies exploring achievement emotions in e-learning contexts and confirmed positive and negative relations, respectively, between enjoyment and anxiety on the one hand and learning processes (e.g., engagement, strategy use) and achievement outcomes on the other hand. Single studies focusing on other emotions in technology-based learning environments also documented the detrimental effects on performance of negative emotions such as anger/frustration and boredom (Bosch et al., 2013; D’Mello & Graesser, 2011).

Achievement emotions across genders, areas of study, and cultures

Resorting to the principle of parsimony, CVT proposes the relative universality of the connections between achievement emotions and their correlates (Pekrun, 2006, 2018; Pekrun & Perry, 2014; Pekrun & Stephens, 2010). The functional mechanisms which link antecedents, achievement emotions, and outcomes are assumed to be the same in different individuals, and across genders, areas of study, and cultures. Nevertheless, achievement emotions can differ in their rates, contents, or parameters according to individual dispositions (Frenzel, Pekrun et al., 2007; Pelch, 2018) and cultural contexts (Frenzel, Thrash, 2007).

Gender stereotypes held by students, for example that females are better at languages and males are better at math or technology, can impact their beliefs about domain-specific competence, in turn influencing achievement emotions. Nevertheless, recent findings suggest that for technology this gap is reducing (Loderer et al., 2020). CVT also posits that both achievement emotions and their antecedents are organized in ways that vary in different areas of study. In addition, while large-scale studies including many different countries have rarely been conducted, it seems that emotion concepts are characterized by a high cross-cultural similarity (Loderer et al., 2020), a prerequisite for valid cross-cultural comparisons. Finally, it is worth noting that, for traditional learning, positive achievement emotions usually prevail over negative achievement emotions (Raccanello et al., 2022).

The current study

This study was part of a large-scale global online student survey entitled “Impact of the COVID-19 Pandemic on Life of Higher Education Students” (http://www.covidsoclab.org), promoted by the University of Ljubljana, Slovenia, and adapted from the European Students' Union Survey (2020). It was aimed at examining HE students' perceptions on the impact of the first wave of COVID-19 pandemic in the first half of 2020. It consisted of 39 questions exploring sociodemographic, educational, and other aspects of the life of HE students – these included academic online work, social life, emotional life, changes in habits, the role of institutions, as well as personal reflections on COVID-19 (Aristovnik et al., 2020b). Students’ participation exceeded the milestone of 30000 responses involving more than 130 countries of all the six continents (Aristovnik et al., 2020a, 2020b).

Because many traditional HE courses worldwide were forced by the pandemic to transition to e-learning, we had the opportunity to explore the applicability of CVT for this domain. We focused on the achievement emotions and on their antecedents (investigating also gender, area of study, and country differences for achievement emotions), felt by HE students from 13 countries during the first wave of the pandemic. Notwithstanding the importance of this topic, researchers have rarely addressed this issue using cross-national samples. From a theoretical perspective, beyond having the rare chance to assess the universality of the assumptions of CVT, we could explore possible differences in the rates of achievement emotions according to parameters such as gender, area of study, and country. From an applied perspective, the knowledge on the correlates of achievement emotions in e-learning can give valuable indications for ameliorating online environments and reflecting on the advantages and limitations of online compared to on-site contexts.

Aim 1

We tested a model based on the CVT in which environmental and proximal antecedents were linked to achievement emotions.

We hypothesized that environmental antecedents such as cognitive (structure and clarity of the task, task demands) and motivational quality (peer support, teacher support) of the e-learning task were associated with the proximal antecedents (e-learning self-efficacy, computer self-efficacy) of achievement emotions. We expected that students who perceived a better structure and clarity of the task, in terms of a higher organization of classes (e.g., lectures, tutorials, seminars, and practical classes) and teachers' work, and who received more frequently peers and teachers' support, had in turn a higher self-efficacy. Vice versa, we expected that those students who perceived increases in the task demands in terms of workload would have had a lower self-efficacy.

We hypothesized that e-learning and computer self-efficacy were associated with achievement emotions: Students with a higher self-efficacy were expected to feel more frequently positive achievement emotions and less frequently negative achievement emotions.

We hypothesized that self-efficacy (e-learning self-efficacy, computer self-efficacy) mediated the relation between environmental antecedents (i.e., structure and clarity of the task, task demands, peer support, teacher support) and achievement emotions (i.e., positive and negative achievement emotions). We expected that all the environmental antecedents impacted achievement emotions through the mediation of self-efficacy.

Aim 2

We examined the invariance of the model on the relations between the environmental antecedents, self-efficacy, and achievement emotions, based on the assumptions about the universality of the CVT.

We expected that the tested model was invariant across gender, area of study (social sciences, applied sciences, natural and life sciences, arts and humanities), and country (Bangladesh, Chile, Croatia, Ecuador, India, Indonesia, Italy, Mexico, Poland, Portugal, Romania, Slovenia, Turkey), i.e., that the relations between the variables did not change.

Aim 3

We focused on achievement emotions. We explored differences in their mean scores according to gender, area of study, country, and valence (positive, negative). This aim was exploratory in nature and we did not formulate any specific hypotheses.

Method

Participants

Our target population was constituted by HE students, at least 18 years old. We involved a convenience sample of 17019 students enrolled in a HE institution (M = 23.51 years, SD = 5.04, Mdn = 26, interquartile range = 3 years; 66% females, 33% males, 1% other gender identities), of which 91% were studying full-time. Most of them were bachelor's students (80%), followed by master's students (17%), with 3% doctorate students. The most frequent area of study was social sciences (37%), followed by applied sciences (30%), natural and life sciences (22%), and arts and humanities (11%). See Table A of the Supplementary Materials for the descriptive statistics by country.

Since it was not compulsory to complete the whole questionnaire, response rate varied across questions. Accordingly, we applied a complete case analysis approach to mitigate missing data issues, with the assumption of “missing completely at random” (Little & Rubin, 2019). To derive a more robust analysis and perform reliable comparisons at the national level, this study focused on the 13 countries (Bangladesh, Chile, Croatia, Ecuador, India, Indonesia, Italy, Mexico, Poland, Portugal, Romania, Slovenia, Turkey) for which there were at least 500 submitted answers on achievement emotions and their antecedents.

Procedure

The study complies with the Declaration of Helsinki regarding research on human participants. The ethical committees of several of the involved HE institutions approved it, i.e., University of Verona (protocol number: 152951), ISPA–Instituto Universitário (ethical clearance number: I/035/05/2020), University of Arkansas (IRB protocol number: 2005267431), Walter Sisulu University (ethical clearance number: REC/ST01/2020), and Fiji National University (CHREC ID: 252.20).

We recruited the respondents by advertising on HE communication systems around the world and on social media. The students gave their informed consent before participating and they confirmed their status as HE students at the beginning of the survey. The questionnaire was designed in English and later translated into Italian, North Macedonian, Portuguese, Romanian, Spanish, and Turkish. The web-based survey was launched via the open-source web application 1 KA (One Click Survey; www.1ka.si) on May 5, 2020 and remained open until June 15, 2020, that is, in a period in which most of the nations experienced the arduous restrictions imposed by the lockdown. The participants confirmed that their on-site classes (which usually take place in the location/campus of their institution) had been cancelled due to the pandemic. Here we included only the data from the parts of the questionnaire on achievement emotions and their antecedents.

Measures

We examined 8 questions (out of the 39 questions of the questionnaire), related to environmental antecedents of achievement emotions (i.e., concerning structure and clarity of the task – three questions, comprising 15 items; task demands – one question, comprising one item; and motivational quality of the task – one question, comprising two items), proximal antecedents (i.e., e-learning self-efficacy – one question, comprising four items; computer self-efficacy – one question, comprising seven items), and achievement emotions (one question, comprising 10 items), for a total of 39 items (see Table 1 for McDonald's Omega concerning reliability of the measures, and Table B of the Supplementary Materials for the list of the items).

Table 1

Intercorrelations, Means (M), Standard Deviations (SD), 95% Confidence Intervals (CI), and McDonald's Omega (ω) for the Variables of the SEM.

Variable	1	2	3	4	5	6	7	8	9
1. Class organization	–
2. Teacher organization	.63***	–
3. Workload	.05***	.14***	–
4. Peer support	.14***	.18***	.13***	–
5. Teacher support	.18***	.22***	.14***	.47***	–
6. E-learning self-efficacy	.55***	.80***	.11***	.17***	.16***	–
7. Computer self-efficacy	.46***	.73***	.05***	.18***	.12***	.86***	–
8. Positive achievement emotions	.05***	.30***	-.02***	.13***	.13***	.44***	.49***	–
9. Negative achievement emotions	-.03***	.27***	.11***	.03***	.01	.42***	.48***	.63***	–
M	1.82	2.30	3.46	3.41	2.58	2.04	2.48	2.18	2.27
SD	1.29	1.72	1.21	1.61	1.39	1.61	1.91	1.38	1.41
95% CI	[1.80, 1.84]	[2.28, 2.33]	[3.44, 3.48]	[3.39, 3.43]	[2.56, 2.60]	[2.02, 2.06]	[2.46, 2.51]	[2.16, 2.20]	[2.25, 2.29]
ω1	.882	.811	–	–	–	.748	.876	.800	.815

*p < .05, **p < .01, ***p < .001.1 For these variables, McDonald's omega is preferable than Cronbach's alpha (Deng & Chan, 2017). The omega does not require tau-equivalent items (i.e., items with the same difficulties, variances, and means) and it is a centred estimator, while alpha is a lower-bound estimate of reliability because heterogeneous test items would violate the assumptions of the tau-equivalent model. The omega values are interpreted similarly to the alpha values. For single-item measures (i.e., workload, peer support, teacher support) it is not possible to calculate the omega values.

Environmental antecedents

Cognitive Quality of the Task. That is, the structure and clarity of the task and task demands (Pekrun, 2006; Raccanello et al., 2018).

We assessed students' satisfaction (1 = very dissatisfied and 5 = very satisfied) with: (a) class organization, operationalized as organization of lectures (5 items; i.e., Since on-site classes were cancelled, the organization of lectures has changed. Below, several different forms of online lectures are listed. Please assess your level of satisfaction with each form) and tutorials, seminars, and practical classes (5 items; i.e., Since on-site classes were cancelled, the organization of tutorials, seminars, and practical classes has changed. Below, several different forms of online tutorials, seminars, and practical classes are listed. Please assess your level of satisfaction with each form). For both, the students evaluated their satisfaction concerning the following learning forms (Hassanzadeh et al., 2012): online in real-time (videoconference), online with a video recording (not in real-time), online with an audio recording (not in real-time), online by sending presentations to students, written communication (forums, chat, etc.); (b) teacher/lecturer organization (5 items; i.e., Please rate your agreement with the following statements. Since on-site classes were cancelled, my lecturers...), focusing on actions such as having provided course assignments (e.g., readings, homework, quizzes) on a regular basis, having provided feedback on students' performance on given assignments, having responded to students' questions in a timely manner, having been open to students’ suggestions and adjustments of online classes, and having informed the students on -how the exams will look like in the new situation (Al-Fraihat et al., 2020; Chopra et al., 2019).

We used one item (adapted from Kember & Leung, 1998) on the perceived extent of the changes in the workload during the period in which the on-site classes had been cancelled, compared to the previous period (i.e., On average, compared to the workload before on-site classes were cancelled, would you say that your study workload over the last weeks has been…). The students responded on a 5-point scale (1 = significantly smaller and 5 = significantly larger).

Motivational Quality of the Task. We operationalized this in terms of social relationships, using two items developed ad-hoc assessing peer support and teacher support (based on Raccanello et al., 2018). The students rated the frequency with which they had communicated with peers and teachers since the outbreak of the pandemic (i.e., How often have you communicated with the following people online since the COVID-19 pandemic? Colleague from my course, Lecturer) on a 6-point scale (1 = not at all and 6 = several times a day).

Proximal antecedents

The students evaluated self-efficacy on a 5-point scale (1 = strongly disagree and 5 = strongly agree).

E-Learning Self-Efficacy. We used four items (e.g., I can master the skills taught in class this year even if on-site classes were cancelled), adapted from the validated Patterns of Adapted Learning Scale (PALS, Midgley et al., 2000). The first item was reversed.

Computer Self-Efficacy. We used seven items about confidence at using computers and similar devices relevant for e-learning, adapted to be aligned with the technical processes of the current technological devices, software, and applications for academic learning (Lu et al., 2016). We asked the students how confident they were (i.e., To what extent do you agree with the following statements about your computer skills. I am confident in…) in browsing online information, sharing digital content, using online teaching platforms (BigBlueButton, Moodle, Blackboard, GoToMeeting, etc.), using online collaboration platforms (Zoom, MS Teams, Skype, etc.), using online communication platforms (e-mail, messaging, etc.), using software and programmes required for their studies, applying advanced settings to some software and programmes.

Achievement emotions

We utilized a 10-item adapted brief version of the validated Achievement Emotions Adjective List (AEAL, Raccanello et al., 2022). The respondents rated the frequency with which they had felt three positive activating (joyful, hopeful, proud), one positive deactivating (relieved), four negative activating (frustrated, angry, anxious, ashamed), and two negative deactivating (hopeless, bored) emotions while attending classes and studying since the outbreak of the pandemic (i.e., Please rate to what extent have you felt the following emotions while attending your classes and studying and preparing for them since the outbreak of COVID-19 in your country) on a 5-point scale (1 = never and 5 = always). In contrast to the original AEAL, we used one adjective for each emotion instead of three to limit the cognitive workload of the survey (Raccanello et al., 2020); we assessed only one emotion for positive deactivating emotions (i.e., relief), given their peripheral role for technological contexts; and we added frustration, particularly salient in such contexts (Mayer, 2020).

Data analysis

We used the R software and its packages (Version 4.0.4, R Core Team, 2021; see Paragraph A of the Supplementary Materials for the list of packages). First, we conducted a structural equation model (SEM) testing the relations between environmental antecedents (i.e., class organization, teacher organization, workload, peer support, teacher support) and achievement emotions, examining the mediation of the proximal antecedents (e-learning self-efficacy, computer self-efficacy). We included all the links between the variables. We applied a two-step process using the robust maximum likelihood estimator (lavaan package), examining the measurement model and then the structural model. The indices for model fit were the comparative fit index (CFI), Tucker Lewis index (TLI), root mean square error of approximation (RMSEA), and square root mean residual (SRMR), with CFI and TLI > .90, and RMSEA and SRMR < 0.05, as thresholds (Bentler, 1990; Marôco, 2014; Marsh et al., 2005). We also calculated McDonald's (1999) omega (ω; cutoff for good reliability: ω > .70; semTools package).

Second, we conducted the invariance analysis by comparing the difference in the fit of a series of sequentially constrained models concerning configural, loadings, intercepts, means, and regression coefficients (Vandenberg & Lance, 2000). Since Χ is likely to be overly sensitive for large sample sizes (Meade et al., 2008), we examined the differences between CFI values (ΔCFI) for two sequentially constrained models (cutoff value: ΔCFI < −0.01; Cheung & Rensvold, 2002).

Third, we examined the differences in achievement emotions conducting a linear mixed model (LMM; lme4, emmeans, effects, and car R packages). We considered gender (males, females – the category "other gender identities" was excluded given its low occurrence), area of study (social sciences, applied sciences, natural and life sciences, arts and humanities), and country (Bangladesh, Chile, Croatia, Ecuador, India, Indonesia, Italy, Mexico, Poland, Portugal, Romania, Slovenia, Turkey) as between-subject fixed effects; valence of emotions (positive, negative) as within-subject fixed effect; participants as random effect; and the score of positive and negative emotions as dependent variables. We performed an analysis of the deviance table (Wald chi-square tests) and we used the Bonferroni correction for post-hoc tests (level of significance: p < .05).

Results

Structural equation model

The SEM diagram is shown in Fig. 1 ; the intercorrelations and the descriptive statistics for the variables are shown in Table 1; the factor loadings are shown in Table C of the Supplementary Materials. The loadings were statistically significant at p < .001, and larger than the 0.50 cutoff value (with one exception). Reliability was adequate, with ω ranging from .748 (e-learning self-efficacy) to .882 (class organization). The SEM had a good fit, Χ (662, N = 17019) = 6310.465, p < .001, CFI = 0.932, TLI = 0.924, RMSEA = 0.045, SRMR = 0.041, with most of the structural paths significant at p < .001. It explained 30% of variance for positive and 24% for negative emotions.

Fig. 1

SEM for the Relations Between Environmental Antecedents, Self-Efficacy, and Achievement Emotions. For Parsimony, we Presented Only Significant Structural Paths. *p < .01, **p < .001, ***p < .001.

Concerning the relation between environmental antecedents and self-efficacy, we found that class organization, teacher organization, and peer support were positively linked to e-learning self-efficacy (β = 0.52, p < .001; β = 0.21, p < .001; β = 0.04, p = .008, respectively) and computer self-efficacy (β = 0.19, p < .001; β = 0.27, p < .001; β = 0.19, p < .001). Moreover, workload was significantly linked to self-efficacy, negatively for e-learning self-efficacy (β = −0.06, p < .001) and positively for computer self-efficacy (β = 0.04, p = .021). These findings partially supported Hypothesis 1a.

As for the relation between self-efficacy and emotions, we found that both e-learning self-efficacy (β = 0.33, p < .001) and computer self-efficacy (β = 0.10, p < .001) were significantly and positively linked to positive emotions, while only e-learning self-efficacy (β = −0.38, p < .001) was significantly and negatively linked to negative emotions, partially corroborating Hypothesis 1b.

Regarding the role of self-efficacy as a mediator, e-learning self-efficacy mediated the relation of environmental antecedents with positive emotions, partially for class organization (indirect effect: 0.17, p < .001), teacher organization (0.07, p < .001), peer support (0.01, p = .010), and workload (−0.02, p < .001); and with negative emotions, partially for class organization (−0.20, p < .001) and workload (0.02, p < .001), and totally for teacher organization (−0.08, p < .001) and peer support (−0.02, p < .001). Computer self-efficacy mediated the relation with positive emotions, partially for class organization (0.02, p < .001), teacher organization (0.03, p < .001), peer support (0.02, p < .001), and workload (0.01, p = .037). Finally, self-efficacy did not mediate the relation between teacher support and emotions: Teacher support was significantly (and positively) linked only to positive emotions (β = 0.05, p = .003). These findings partially supported Hypothesis 1c.

Invariance analysis

The model was characterized by strong metric invariance for both gender and area of study (ΔCFImetric-config < 0.001/< 0.001; ΔCFIscalar- metric = −0.002/-0.003, respectively); invariance was observed both for factor means (ΔCFImeans-scalar = −0.002/-0.001) and for structural regression coefficients (ΔCFIregress-means < 0.001/< 0.001). However, across countries, only configural invariance was satisfied (ΔCFImetric-config = −0.008), and there was no scalar invariance (ΔCFIscalar- metric = −0.033). Thus, only the basic factor structure of the model was invariant across countries. These findings partially supported Hypothesis 2. We reported in Table D of the Supplementary Materials the structural standardized coefficients by country. The models explained from 13% (Indonesia) to 52% (Italy) of the variance for positive emotions, and from 7% (Ecuador) to 42% (Chile) for negative emotions. Considering the most relevant environmental antecedents of self-efficacy as revealed by the original SEM (class and teacher organization), we highlight that, for all the countries except Romania, at least one indicator of organization was significantly and positively linked to at least one indicator of self-efficacy. As for e-learning self-efficacy (which, in the original SEM, resulted as the main proximal antecedent), it was significantly related to positive and/or negative emotions, in the expected direction, for most of the countries (except Ecuador, Indonesia, and Romania).

Linear mixed models

We excluded countries from the LMM because the invariance across country was only configural. We compared three LMM which examined: (1) fixed effects (Akaike's Information Criterion, AIC = 62017.289, Bayesian Information Criterion, BIC = 62082.552); (2) fixed effects and interactions between the between-subject factors and the within-subject factor (AIC = 61852.496, BIC = 61956.203); (3) fixed effects and all interactions (AIC = 61858.309, BIC = 61999.338). We selected model 2, due to lower AIC and BIC (Fabozzi et al., 2014). This model was significantly different from the others (model 2 vs. 1, Χ (4, N = 16517) = 166.980, p < .001; model 2 vs. 3, Χ (6, N = 16517) = 17.813, p < .001; Satorra & Bentler, 2001).

The LMM revealed a significant effect of valence, Χ (1, N = 16517) = 268.784, p < .001: Positive emotions (M = 2.81, SD = 0.82, 95% CI [2.80, 2.83]) were lower than negative emotions (M = 2.98, SD = 0.79, 95% CI [2.96, 2.99]). Also gender, Χ (1, N = 16517) = 19.543, p < .001, had a significant effect. Such effects were in turn moderated by two significant two-ways interactions, gender × valence, Χ (1, N = 16517) = 99.830, p < .001 (Fig. 2 a), and area of study × valence, Χ (3, N = 16517) = 53.863, p < .001 (Fig. 2b). We reported in Table E of the Supplementary Materials the descriptive statistics of achievement emotions by gender and area of study, and the post-hoc tests. Examining them (all p < .001), we found that positive emotions were lower than negative emotions for females but not for males; moreover, they were lower than negative emotions for natural and life sciences and for arts and humanities.

Fig. 2

Positive and Negative Achievement Emotions, According to (a) Gender and (b) Area of Study. The Bars Represent the 95% CI.

Discussion

We extended the generalizability of the CVT to the context of e-learning during the first wave of COVID-19 pandemic in 2020, focusing on achievement emotions and their antecedents in a sample of over 17000 students from 13 countries. Our findings represent an important milestone for the CVT, as they confirm its relevance not only for traditional learning environments, as documented by previous research, but also for e-learning processes during the emergency phase of a global disaster.

Achievement emotions and their antecedents across gender, area of study, and country

Concerning the relation between environmental and proximal antecedents (Aim 1), our data mainly confirmed Hypothesis 1a. Class and teacher organization were positively related to e-learning and computer self-efficacy. Surprisingly, workload was related negatively to e-learning self-efficacy and positively to computer self-efficacy. The quasi-instant transition to online education in March, 2020 probably led to higher workloads, and those more confident in their use of technology probably adapted more easily; at the same time, a greater use of technology could have resulted in better technical abilities due to increased familiarity with equipment and processes. Increases in workload could have caused a diminishing of confidence in mastering e-learning tasks, which, even considering their pervasiveness for any academic task, had suddenly been perceived as particularly demanding. Long-term designs could explore whether these effects persist in the subsequent phases of the pandemic or in phases after the pandemic.

The effect of the cognitive quality was stronger on e-learning than computer self-efficacy. This suggests that improvements or disruptions in structural/organizational facilities impact students’ beliefs on their abilities to master e-learning in its complexity (including sub-processes such as goal setting, learning and task-related strategies, ways of seeking help, etc.; Littlejohn et al., 2016) more than their individual confidence in their technical abilities (Saadé & Kira, 2009).

With respect to the motivational quality of the task, only peers support was positively related to both types of self-efficacy. This reveals the greater importance of peers compared to teachers for self-efficacy, as shown by students turning to their colleagues and less to their teachers for support during the first wave of the pandemic. This shows the differential role of social support sources (Ahmed et al., 2010), as well as the influence peers play on students’ performance in e-learning (Arguel et al., 2019).

Turning to the links between proximal antecedents and achievement emotions, our data mainly supported Hypothesis 1b. Higher e-learning self-efficacy was associated with more frequent positive achievement emotions and less frequent negative achievement emotions. However, computer self-efficacy was positively related only to positive achievement emotions. In other words, while any change in e-learning self-efficacy impact any type of achievement emotions, the perception of improvements of technological abilities have beneficial effects only on positive emotions. These results could guide teachers' planning for e-learning strategies or practices that can influences students' e-learning self-efficacy, given its pervasive effect on their emotions. The findings also support the implementation of permanent actions to increase students’ technological abilities.

Our data also demonstrated the mediating role of self-efficacy in the relation between environmental antecedents and achievement emotions, corroborating Hypothesis 1c and confirming the CVT as regards the influence that distal constructs have on the latter – that is, a mediation through proximal antecedents (Pekrun, 2006, 2018; Pekrun & Perry, 2014).

Moreover, we showed (Aim 2) that the relations between the constructs of our model were stable across genders, four areas of study, and 13 countries, confirming Hypothesis 2. Acknowledging that the invariance across countries was only configural, we can nevertheless infer that even if in different countries the level of some factors varied, the overall model was confirmed. Therefore, we supported the relative universality of the CVT concerning the relations between achievement emotions and their antecedents for e-learning (Pekrun, 2006, 2018; Pekrun & Perry, 2014; Pekrun & Stephens, 2010). Consistent with the CVT we also found that the mean rates of achievement emotions varied according to gender and area of study (Aim 3). In contrast to patterns typical of the pre-pandemic phase, negative achievement emotions were more frequent than or equal to positive ones; a further indicator of the traumatic impact of the pandemic on students’ life (Aristovnik et al., 2020b; Cao et al., 2020; Horesh & Brown, 2020; Kecojevic et al., 2020; Lee, 2020; Son et al., 2020). However, this effect was more apparent for females than males (consistent with other research indicating that females can be more inclined to learning-related anxiety; Loderer et al., 2020), and for two areas of study (natural and life sciences, arts and humanities).

Limitations and directions for future research

This study suffers from several limitations. First, our convenience sample was not representative at the country level since it excluded students with difficulties in accessing the internet or not having electronic devices. In addition, we did not distinguish undergraduate from graduate students, nor part-time from full-time students. Future research should use more representative samples and consider alternative ways of data collection beyond online surveys. Second, the gender distribution of respondents was not balanced, with more females responding to the survey than males; however, the model was invariant across genders. Moreover, we could not consider the category concerning other gender identities given to its very low occurrence. Third, the survey was based on self-report measures, characterized by desirability biases; nevertheless, they are still among the most privileged ways to access individuals' inner states (Pekrun & Bühner, 2014). Future research could also include objective measures (i.e., administrative/official data on academic performance). Fourth, we did not distinguish activating and deactivating emotions, nor did we measure another key proximal appraisal, i.e., value; future studies should consider these aspects. Moreover, there was a different number of positive and negative emotions; further studies could balance this. Fifth, the survey assessed general emotions rather than emotions specific by domain; however, this was due to the need to involve students attending courses involving different areas of study. To compare how students from various areas of study adapted to the online learning environment, future research should include a higher number of students from different areas, as well as specific questions (e.g., for medical students, science, mathematics, amongst others). Finally, we did not examine the effect of the COVID-19 impact in different countries due to the lack of valid indicators for comparisons. In May–June 2020, different countries were in different stages of the pandemic and the situation was continuously changing. Future research could incorporate pandemic-specific indicators (e.g., severity of impact, degree of school openness, amongst others), whilst pointing out to how the subsequent waves of the pandemic affected students’ emotions, by using longitudinal research. Through such designs, future studies could examine whether the differences in achievement emotions that we detected were characteristics of the initial emergency phase and whether and how they changed as the students gradually became more familiar with the restraints due to the prolonged emergency phase of the pandemic.

Implications for practice and policy

The negative impact of the pandemic on students’ emotions, and consequently on performance and wellbeing, calls for psychological/emotional preparedness and support measures dedicated to students since online education is still a dominant means of educational delivery. Our data can be the basis for evidence-based policy recommendations to support stakeholders at the international, national, and local level. Their actions can aim at influencing the antecedents to emotions such as self-efficacy thus improving students emotional balance (see Tomaževič et al., 2021, for further policy suggestions on e-learning).

Moreover, as the access to technology and students' skills using e-learning devices is unequal, policy measures are needed for students coming from/living in economically disadvantaged families/backgrounds to provide them with the necessary equipment and training in the use of e-learning platforms. In terms of types of support for students, the more important role of peers compared to teachers suggests that HE institutions need to find ways to maintain communication and collaboration between students, whilst at the same time increasing involvement of teachers in supporting students’ e-learning process.

Conclusions

The outbreak of the COVID-19 pandemic has resulted in devastating consequences for many students, which can include negative impacts on their emotional functioning. The current study extends the generalizability of the CVT of achievement emotions to the context of e-learning that was imposed to compensate for the absence of on-site classes cancelled or reduced in response to the ongoing disaster. Despite the difficulties of using the online mode in these circumstances, technological advances gave the students the possibility of accessing instruction while physically isolated. Our data, involving a very large sample of HE students from 13 countries, revealed that both environmental and proximal antecedents of achievement emotions are linked to how the students felt in relation to their learning activities. Beyond their theoretical relevance, these findings play a crucial role for the development of interventions aiming at sustaining students’ wellbeing and performance in such critical periods.

CRediT author contribution statement

Conceptualization: Aleksander Aristovnik, Nina Tomaževič, Daniela Raccanello; Data curation: Lan Umek, Dejan Ravšelj; Formal analysis: Lan Umek, Daniela Raccanello, Giada Vicentini, Roberto Burro, Özlem Yorulmaz, João Marôco, Adeniyi Francis Fagbamigbe; Funding acquisition: Aleksander Aristovnik; Investigation: Aleksander Aristovnik, Damijana Keržič, Nina Tomaževič, Dejan Ravšelj, Lan Umek, Daniela Raccanello, Giada Vicentini, Roberto Burro, Alpana Mishra, Sanja Tatalović Vorkapić, João Marôco, Fany Inasius, Maria Cheraghi, Md. Mamun-ur-Rashid, Silvia Mariela Méndez Prado, Beata Dobrowolska, Thais França, Roxana Balbontín-Alvarado; Methodology: Daniela Raccanello, Giada Vicentini, Roberto Burro, João Marôco; Project administration: Aleksander Aristovnik, Nina Tomaževič, Dejan Ravšelj; Resources: Lan Umek; Supervision: Aleksander Aristovnik, Nina Tomaževič; Validation: Lan Umek; Visualization: João Marôco, Özlem Yorulmaz, Daniela Raccanello, Giada Vicentini, Roberto Burro; Roles/Writing - original draft: Kornélia Lazányi, Daniela Raccanello, Giada Vicentini, Roberto Burro, Sujita Kumar Kar, Bertil P. Marques, Kornélia Lazányi, Florin Lazăr, Sanja Tatalović Vorkapić, Silvia Mariela Méndez Prado, Ana-Maria Zamfir; Writing - review and editing: Aleksander Aristovnik, Damijana Keržič, Nina Tomaževič, Dejan Ravšelj, Lan Umek, Kornélia Lazányi, Daniela Raccanello, Giada Vicentini, Roberto Burro, Sujita Kumar Kar, Bertil P. Marques, Kornélia Lazányi, Alpana Mishra, Florin Lazăr, Özlem Yorulmaz, João Marôco, Md. Mamun-ur-Rashid, Denilson da Silva Bezerra, Roxana Balbontín-Alvarado, Alka Obadić, Silvia Mariela Méndez Prado, Sujita Kumar Kar, Fany Inasius, Cristina Mollica, Juan D. Machin-Mastromatteo, Belinka González-Fernández, Oliva Mejía-Rodríguez, Adeniyi Francis Fagbamigbe, Beata Dobrowolska, Thais França, Bertil Maria Pires Marques, Florin Lazăr, Chinaza Uleanya, MoezAlIslam Ezzat Faris, Silvana G. Navarro Jiménez, Cristina Mollica, Rob Hall

Declaration of competing interest

None.