Emotional outcomes of e-learning adoption during compulsory online education.

Information on the emotional outcomes of e-learning system use and emotional aspects of user experience in higher education is quite limited. Accordingly, the aim of the study is to identify the factors that influence university students' intention to continue using e-learning systems and to examine the emotional outcomes of the continuance intention. The core constructs of the Technology Acceptance Model formed the basis of the proposed model, and the model was extended with a framework of emotions (challenge, achievement, deterrence, loss) and external variables. Data were collected online from 19,530 university students of a state university. For the analysis, Partial Least Squares-Structural Equation Modeling was employed. The proposed model explained 73.5% of continuance intention, 50.3% of achievement, and 52.2% of challenge emotions. In addition, 23 of the 25 tested hypotheses were supported. The findings indicate that perceived usefulness is a decisive factor in creating user experiences that generate emotions such as enjoyment, playfulness and satisfaction. In addition, the results showed that personal innovativeness strongly influenced the core constructs of technology acceptance model and the positive aspects of emotions (achievement and challenge). Accordingly, it can be stated that these findings lead us to the fact that students' value perceptions regarding e-learning systems have a critical role in terms of emotional outcomes. In addition, the findings suggest that both intrinsic-extrinsic motivators, innovativeness characteristics and emotional outcomes should be taken into account in design and development process in order to improve the quality of the user experience. In this direction, implications for research and practice are discussed.

Introduction

New practices that need to be implemented due to the COVID-19 pandemic have led to extensive changes worldwide. Education has been one of the vital areas in which these practices have had a profound effect. Education during the pandemic, which could not be delivered with traditional methods, had to be transformed into online education so that more than 1 billion students whose education was disrupted (International Association of Universities, 2020) could continue their education (Lowenthal et al., 2020). Higher education institutions have made large-scale investments to ensure that education can continue without interruption and that education can be delivered in a quality manner. At this point, the vital role to be played by e-learning systems, whose potential has been recognized for many years, has clearly emerged. In parallel, the importance of the effective use of information technologies such as e-learning systems for the success of online education is emphasized (Şahin et al., 2021).

E-learning systems have significant potential to facilitate teaching and learning experiences (Lee et al., 2011), improve access to educational resources and programs (Lwoga, 2014), and enable education to be delivered effectively and efficiently (Samsudeen & Mohamed, 2019). Despite large-scale investments and the mandatory transformation of education, it is a clear prerequisite that students accept and effectively use e-learning systems. Additionally, even if students start to use e-learning systems, ensuring its continuity comes to the fore as a very critical role. Problems experienced in terms of continued use may generally occur in the form of the ineffectiveness of the adoption process to a significant extent or the cessation of use after the first use (Adelé & Brangier, 2013). Although the acceptance of e-learning is an important first step for the success of these systems, continued use is essential to ensure real success (Lee, 2010). In other words, it can be stated that the uses that consist of a use that only covers the beginning of the process and that do not show continuity in the long term will be far from providing the desired results during and after the pandemic. To benefit the potential of e-learning systems effectively, the acceptance of these technologies by students is the first step that cannot be compromised (Abdullah & Ward, 2016), and ensuring that students continue to use them plays a vital role (Lee, 2010).

It is emphasized that determining the factors that influence the intention to continue using will help increase the use of these technologies by supporting both users such as students and instructors and system designers (Lee, 2010). Furthermore, studies emphasize the significant role of emotional variables in technology acceptance and reveal the need for more detailed and comprehensive processing of emotions in technology acceptance (Beaudry & Pinsonneault, 2010; Lu et al., 2019; Şahin et al., 2021; Şahin, 2021). In line with this, the absence of an up-to-date technology acceptance study that employs a comprehensive emotional framework as an outcome and focuses on student experiences in the context of emotions suggests that there is a significant gap in the field. Accordingly, it is anticipated that an inclusive model involving institutional, social, individual external variables and emotional outcomes will provide valuable contributions to the effective and continuous use of e-learning systems.

The technology acceptance model (TAM) approach was adopted as a tool for examining the continuance intention and influential variables. Technology acceptance models can be expressed as effective tools that can offer a solid roadmap, which has been used in many different fields for many years. Among the acceptance models, TAM is the most prominent one, especially in the field of education. TAM is expressed as a robust (Taylor & Todd, 1995; Venkatesh et al., 2003), reliable, and effective model that allows extending the models planned to be improved without complicating (Davis, 1989; Mathieson, 1991). TAM was selected as the basis of the research model, based on its simple structure allowing for successful expansion (King & He, 2006) and providing a suitable starting point for the complex model of the study. With regard to emotions, a framework for the classification of emotions (loss, deterrence, challenge, achievement) based on theoretical foundations (Beaudry & Pinsonneault, 2010) and external variables from different theories were employed to extend TAM. Accordingly, the aim of the study is to determine the variables that affect university students’ intention to continue using e-learning systems with an extended TAM and to examine the emotional outcomes of their intention to continue using e-learning systems.

Theoretical background

A framework for classification of emotions

In the context of emotions and behaviors, it is stated that individuals’ emotions and adaptation behaviors serve to bridge the gap between the moment their routines are interrupted and the time when new routines are created or old routines are re-established (Beaudry & Pinsonneault, 2010). It can be said that emotions and adaptation behaviors, which also play a role in an individual’s awareness of a future interruption or deviation, are also related to non-standard practices that emerge as a result of technological changes. Accordingly, this suggests that the rapid and radical change in learning processes (Şahin et al., 2021) may be related to the emotional reactions of students, their intention to use e-learning systems, and possible variables that affect their intention. Moreover, considering the foundations of theories of emotion such as appraisal tendency indicating that different categories of emotions will reveal different adaptation behaviors and affect technology-related behaviors in different ways (Bagozzi, 1992), employing a structure that comprehensively addresses emotions, has a basis, and groups them in a consistent way can help fill a gap in the field in terms of technology use and emotional reactions. Accordingly, a categorization as emotions of challenge, achievement, deterrence, and loss has been adopted to group emotional reactions toward information technologies (Beaudry & Pinsonneault, 2010; Lu et al., 2019).

Challenge and achievement

Emotions belonging to the challenge class are explained as an individual regarding information technology as an opportunity that is likely to yield positive results and feeling that he/she has some control over the outcomes. It is indicated that a situation evaluated in the context of overcoming obstacles can arouse emotions such as excitement, enjoyment and flow, and strengthen positive attitudes toward technologies (Beaudry & Pinsonneault, 2010; Folkman & Lazarus, 1985; Lee et al., 2012). The emotions of achievement are explained as the pleasant feelings that emerge as a result of evaluating the use of information technologies as a situation that will yield positive outcomes and the individual’s achievement of a goal using technology (Beaudry & Pinsonneault, 2010; Lee et al., 2012; Lu et al., 2019). Accordingly, it can be said that the factors that may affect the individual’s evaluation of technology use as an opportunity, the effect of technology use perceived as an opportunity on individuals’ positive emotions, and the power of potential variables related to these in the context of motivating users play a critical role. This important role suggests that university students’ intention to continue using e-learning systems and potential factors associated with their intention to continue using may be related to the emotional outcomes experienced by students.

Deterrence and loss

Emotions in the loss class are expressed as the perception that the use of an information technology is a threat and that the individual does not have sufficient control over the consequences of this use (Beaudry & Pinsonneault, 2010). It is emphasized that as a result of experiencing such a situation, there is a possibility of emotions such as anger, disappointment, and frustration to emerge (Bagozzi, 1992). Emotions in the deterrence class are explained as the perception of technology as a threat, but the individual’s feeling that he/she has some control over the consequences of use, and feelings such as anxiety, worry, and fear are shown as examples of emotions to be experienced under these conditions (Bagozzi, 1992; Beaudry & Pinsonneault, 2010). Accordingly, an inference can be made that the factors that may prevent or reduce the individual’s perception of an information technology as a threat or a factor that may cause negative consequences are related to these classes of emotions. The points emphasized indicate that university students’ intentions to continue using e-learning systems and the factors that influence their intentions may be related to their emotional reactions to these technologies.

Technology acceptance model

TAM is expressed as a model that describes how individuals accept and use information systems (Davis et al., 1992). TAM, designed to explain adoption processes based on the benefits that technology can provide and the ease of use it offers (Zarafshani et al., 2020), is employed to examine the effects of potential external variables on individuals’ attitudes, beliefs, and intentions (Davis, 1989; Zheng & Li, 2020). Within the scope of this study, the three core constructs included in TAM, perceived ease of use (PEU), perceived usefulness (PU), and intention (INT), constitute the basis of the model proposal. PEU refers to the degree of the individual’s perception of how little effort the technology will require, PU refers to the degree of belief in the performance increase that can be obtained using technology, and INT denotes the individual’s intention to use the technology (Davis, 1989). In this study, students currently using e-learning systems were selected as the participant group, and accordingly, the continuance intention (CI) was employed instead of intention. PEU and PU, are expressed as two key variables in the context of explaining the technology acceptance of users (Zheng & Li, 2020), and it is indicated that these constructs are the most effective determinants of the intention (Venkatesh et al., 2003). Accordingly, the following hypotheses were proposed.

H1. Perceived ease of use positively influences (a) perceived usefulness and (b) continuance intention.

H2. Perceived usefulness positively influences continuance intention.

H3. Perceived usefulness positively influences the emotions of (a) challenge and (b) achievement.

H4.Continuance intention positively influences the emotions of (a) challenge and (b) achievement while negatively influences the emotions of (c) deterrence and (d) loss.

Compatibility

It is stated that compatibility (CMP) (Venkatesh & Davis, 2000), which is expressed as the compatibility of the technology to be used with the individual’s task or job, has various effects on acceptance (Sanchez-Prieto et al., 2019; Ursavaş, 2014). Furthermore, studies indicate that if an individual finds a technology suitable for his/her own style, he/she will prefer to use that technology (Ursavaş, 2014), but if there is no such suitability, a significant obstacle will emerge. In addition, past studies on educational technologies have shown that CMP has strong effects on all core acceptance constructs (Şahin et al., 2021; Şahin & Şahin, 2021; Ursavaş, 2014). Based on these, the hypotheses below were formulated.

H5. Compatibility positively influences (a) perceived usefulness (b) perceived ease of use and (c) continuance intention.

Facilitating conditions

Facilitating conditions (FC), which can serve as a guide in the context of support and training (Garone et al., 2019), are expressed as the user’s perception of the elements that may influence his/her fulfillment of a task and the effect of technical support (Teo, 2009; Venkatesh et al., 2003). Considering the emphasis made on the importance of resources such as technical support, training, and infrastructure in the context of FC (Şahin et al., 2021; Ursavaş, 2014), it can be stated that FC have an crucial role in students’ continuance intention. Accordingly, the following hypotheses were proposed.

H6. Facilitating conditions positively influences (a) perceived ease of use and (b) continuance intention.

Self-efficacy

Self-efficacy (SE), which is among the most studied variables in the context of e-learning (Abdullah & Ward, 2016), is explained as an individual’s judgment of the capacity he/she has to fulfill a task (Bandura, 1982). Within the scope of technologies, SE, which is expressed as the belief in the ability to fulfill a task with the use of technology (Compeau & Higgins 1995), has been investigated in many acceptance studies conducted in the field of education, and its various effects on acceptance have been identified (e.g. Chang et al., 2017; Joo et al., 2018; Revythi & Tselios, 2019; Salloum et al., 2019; Şahin et al., 2021; Zheng & Li, 2020). Considering the connection of self-efficacy with the competence and self-confidence of users, and its effects on ease of use and intention (Şahin et al., 2021; Şahin, 2021), it can be stated that SE plays a role in both the tendency of students to continue using the e-learning system and the emotional reactions arising from the use of technology. Accordingly, the hypotheses below were proposed.

H7. Self-efficacy positively influences (a) perceived ease of use and (b) continuance intention.

H8. Self-efficacy negatively influences the emotions of (a) deterrence and (b) loss.

Innovativeness

Innovativeness (PI) is explained as the degree to which an individual in a social system adopts an innovation relatively faster and easier than other individuals (Rogers & Shoemaker, 1971). It is stated in the previous studies that innovativeness is an important trait in the context of technology (Kılıçer & Odabaşı, 2010), university students’ innovativeness affects their technology adoption processes (Şahin, 2016), and innovative individuals exhibit more positive attitudes toward technologies (Ali, 2019). Moreover, the fact that the innovativeness characteristic includes personality traits such as resistance to change, openness (Şahin et al., 2021), opinion leadership and risk-taking (Kılıçer & Odabaşı, 2010; Şahin, 2016) indicates that it may be associated with both positive and negative emotions, and emotional experiences that can arise in uncertainty situations. Accordingly, the following hypotheses were proposed.

H9. Personal innovativeness positively influences (a) perceived usefulness (b) perceived ease of use and (c) continuance intention.

H10. Personal innovativeness positively influences the emotions of (a) challenge and (b) achievement while negatively influences the emotions of (c) deterrence and (d) loss.

The research model is presented in Fig. 1.

Fig. 1

Research model

Method

Participants

The participants of this study consist of university students receiving education through the e-learning systems of a state university with live lectures, course recordings, summary videos, question solution videos and various written, visual and audio learning materials (textbooks, audiobooks, unit summaries, tests consisting of multiple-choice and open-ended questions, etc.). 20,190 university students were reached online, and data were collected with a digital form. The data of 660 students who gave the same response to all or most of the items were removed, and the final version of the data set was clarified with 19,530 participants. The descriptive statistics (min, max, mean, f and %) performed on the age, gender and graduation level data of the 19,530 students in order to create the demographic profiles. The demographic information of the participating students is summarized in Table 1.

Table 1

Profile of the participants

Age		Min	Max	Mean
		18	76	30.08
		f		%
Gender	Female	8255		42.3
	Male	11,275		57.7
Graduation Degree	High School	5910		30.3
	Undergraduate	12,566		64.3
	Master	940		4.8
	PhD	114		0.6

Data collection

Prior to the data collection, the necessary permission for the study was obtained from the ethics committee of the university. In the process executed without any influence of the researchers, the data collection tool, which was prepared in a Google form, was delivered to the students online, and it was clearly stated that the participation was completely voluntary. The data collection tool consists of the first part, in which the students’ demographic information is obtained, and the second part, which includes the items for the variables. The second part consists of forty-one 5-point Likert-type (1=strongly disagree; 5=strongly agree) items.

The 41-item scale was developed by the authors. During the scale development process, items for TAM constructs in the measurement tools used in the field of education (e.g. Chang et al., 2017; Teo, 2009; Teo et al., 2012; Ursavaş, 2014), the variables added for extension purpose form other theories and models (e.g. Baydaş & Göktaş, 2017; Salloum et al., 2019; Ursavaş et al., 2019), and the classifications for emotions (e.g. Beaudry & Pinsonneault, 2005; Beaudry & Pinsonneault, 2010) were examined in detail. Accordingly, the factors determined according to the literature and theoretical foundations are perceived ease of use, perceived usefulness, continuance intention, compatibility, facilitating conditions, self-efficacy, personal innovativeness, and emotions of challenge, achievement, deterrence and loss. After the factors were determined, an item pool for the scale was created. In the next stage, the scale was pilot-tested before proceeding to the data collection phase. The pilot test of the study was carried out with the participation of 25 university students in order to make the necessary adjustments by testing the clarity of the items, and to determine the average completion time of the measurement tool. After the pilot test, two separate data collection processes were carried out for exploratory and confirmatory factor analyses, and the scale was finalized as 41 items. The 41-item measurement tool of the study is presented in Table 2.

Table 2

Constructs, items & definitions

Items	Definitions
CI1. I anticipate that e-learning systems will also meet my educational expectations for the future.	The intention to continue using e-learning systems.
CI2. I intend to continue learning through e-learning systems.
CI3. I plan to use e-learning systems for my future lessons as well.
CI4. I predict that I will continue to benefit from e-learning systems.
PU1. Using e-learning systems for my courses increased my performance.	The degree of belief in the performance increase that can be obtained using e-learning systems.
PU2. Learning with e-learning systems increased my efficiency.
PU3.Taking my lessons through e-learning technologies increased my productivity.
PU4.Attending classes through the e-learning systems increased my effectiveness.
PEU1. It was easy for me to use e-learning technologies for my lessons.	The degree of the individual’s perception of how little effort the e-learning system will require.
PEU2. I was able to benefit from e-learning systems without much effort.
PEU3. I think that learning with e-learning systems was simple for me. PEU4. Using e-learning technologies for education required very little effort.
FC1. It is important that I have the necessary hardware and software so that I can easily use e-learning systems.	The user’s perception of the factors that affect the fulfillment of a task through the e-learning system and the impact of technical support.
FC2. Providing the necessary infrastructure and resources is important for me to access my courses through e-learning systems.
FC3. Technical support have an important factor for me to benefit from e-learning systems.
FC4. Whether e-learning systems are user-friendly or not is decisive for me.
SE1. I have the capacity to use e-learning technologies for learning purposes.	An individual’s judgment of his/her capacity to perform a learning task through e-learning systems. The belief in the ability to fulfill an educational task with the use of e-learning systems.
SE2. I have the knowledge and skills to use e-learning systems effectively.
SE3. I am confident in performing learning activities with e-learning systems.
SE4. I think that I have the ability to efficiently participate in my classes by using e-learning systems.
INV1. It is important to stay up-to-date on e-learning technologies.	The tendency of individuals to use new information technologies and the degree to which an individual in a social system adopts an innovation relatively faster than other individuals.
INV2. I am eager to try new e-learning technologies.
INV3. I am generally curious about e-learning technologies.
INV4. It’s a good idea to quickly get acquainted with new technologies.
INV5. I am interested in e-learning technologies that are new to me.
INV6. I would like to be one of the first to try emerging e-learning technologies.
CMP1. The relevance of e-learning systems to my courses is a decisive factor.	The compatibility of the e-learning system to be used with the individual’s educational task or job.
CMP2. It is vital that e-learning systems meet my expectations regarding the quality of learning.
CMP3. I believe that e-learning systems have a high relevance to my courses.
CMP4. It is crucial for me that e-learning systems are compatible with my courses.
ACH1. The education I received through e-learning systems pleased me.	The pleasant emotions that emerge as a result of evaluating the use of information technologies as a situation that will yield positive outcomes.
ACH2. Learning through e-learning systems was fun for me.
ACH3. I felt relieved after learning through e-learning systems.
CHL1. It was exciting for me to study using e-learning systems.	Emotions associated with feeling some control over the consequences of using technology by perceiving information technology as an opportunity with a high probability of producing positive results.
CHL2. My experience with e-learning systems met my expectations.
CHL3. Learning through e-learning systems was an interesting experience.
LSS1. Having to attend classes through e-learning systems frustrated me.	Emotions associated with the perception that an information technology use is a threat and that the individual does not have adequate control over the consequences of that use.
LSS2. The contribution of e-learning systems to education has disappointed me.
LSS3. Learning through e-learning systems was annoying.
DTR1. Having to study online worried me.	Emotions associated with the individual’s perception that an information technology use is a threat but that he/she has some control over the consequences of technology use.
DTR2. My learning experiences with e-learning systems aroused my anxiety.
DTR3. Having to study online was a frightening experience for me.

Data analysis

SmartPLS 3 software and the Partial Least Squares Structural Equation Modeling (PLS-SEM) technique were used for the analysis of the study. PLS was preferred for the path modeling based on the explanatory nature of the study, the complex structure of the model proposal, and its suitability in determining the relationships in the model. At the analysis stage, convergent and discriminant validity were primarily evaluated. In terms of the convergent validity, item loadings, cronbach’s alpha (α), composite reliability (CR) and average variance extracted (AVE) of each construct were examined. As for the discriminant validity, Fornell-Larcker criterion and HTMT ratio of the constructs were tested (Hair et al., 2011; Hair et al., 2017). In addition, the constructs were evaluated in terms of multi-collinearity. Whether there is any problem in terms of linearity was evaluated by examining the variance inflation factor (VIF) of the predictive variables. After the reliability and validity testing, the results were investigated whether the relationships tested were significant or not through the path modeling and bootstrapping analysis.

Results

Measurement mode

Before investigating the structural model and testing the hypotheses, the convergent and discriminant validity of the measurement model were examined. The fact that all of the external loadings examined within the scope of convergent validity were above 0.7 revealed that reliability was achieved at the item level (Hair et al., 2017). Additionally, the fact that Cronbach’s alpha (α) values were between 0.875 and 931, composite reliability (CR) values were between 0.914 and 0.945, and the average variance extracted (AVE) was higher than 0.5 for each construct indicated that convergent validity was established (Hair et al., 2017). The convergent validity results are presented in Table 3.

Table 3

Convergent validity

Factor	Item	Item Loading	α	CR	AVE
Continuance Intention	CI1	0.908	0.897	0.936	0.829
	CI2	0.920
	CI3	0.905
	CI4	0.908
Perceived Usefulness	PU1	0.871	0.910	0.937	0.787
	PU2	0.906
	PU3	0.893
	PU4	0.877
Perceived Ease of Use	PEU1	0.863	0.875	0.914	0.727
	PEU2	0.886
	PEU3	0.799
	PEU4	0.860
Self-Efficacy	SE1	0.882	0.916	0.941	0.799
	SE3	0.907
	SE3	0.903
	SE4	0.884
Facilitating Conditions	FC1	0.871	0.906	0.934	0.780
	FC2	0.899
	FC3	0.893
	FC4	0.871
Compatibility	CMP1	0.875	0.905	0.934	0.778
	CMP2	0.887
	CMP3	0.895
	CMP4	0.872
Innovativeness	INV1	0.825	0.931	0.945	0.743
	INV2	0.884
	INV3	0.881
	INV4	0.877
	INV5	0.874
	INV6	0.828
Challenge Emotions	CHL1	0.913	0.905	0.941	0.841
	CHL2	0.913
	CHL3	0.925
Achievement Emotions	ACH1	0.905	0.898	0.936	0.831
	ACH2	0.921
	ACH3	0.909
Loss Emotions	LSS1	0.884	0.885	0.929	0.813
	LSS2	0.926
	LSS3	0.894
Deterrence Emotions	DTR1	0.891	0.893	0.933	0.823
	DTR2	0.908
	DTR3	0.922

α: Cronbach’s alpha, CR: Composite reliability, AVE: Average variance extracted

The Fornell-Larcker criterion and HTMT ratio were evaluated to test discriminant validity. As seen in Table 4, the square root values ​​of AVE are larger than the inter-structure correlation coefficients. Accordingly, it was determined that there was no problem in terms of the Fornell-Larcker criterion. In terms of the HTMT ratio, as presented in Table 5, all except three indices (CHL-ACH, INT-CMP, LSS-DTR) were at appropriate values ​​(HTMT90). However, since these three indices were marginal and the Fornell-Larcker criterion results were good, discriminant validity was established (Fornell & Larcker, 1981; Hair et al., 2017).

Table 4

Discriminant validity (Fornell-Larcker Criterion)

Factor	ACH	CHL	CMP	DTR	FC	CI	LSS	PEU	PI	PU	SE
ACH	0.912
CHL	0.850	0.917
CMP	0.606	0.614	0.882
DTR	-0.241	-0.210	-0.220	0.907
FC	0.562	0.559	0.722	-0.162	0.883
CI	0.604	0.606	0.836	-0.239	0.685	0.911
LSS	-0.229	-0.192	-0.224	0.821	-0.166	-0.238	0.901
PEU	0.633	0.642	0.700	-0.169	0.743	0.679	-0.162	0.853
PI	0.597	0.602	0.848	-0.209	0.763	0.791	-0.206	0.718	0.862
PU	0.676	0.694	0.688	-0.186	0.674	0.675	-0.189	0.769	0.687	0.887
SE	0.554	0.551	0.754	-0.200	0.784	0.709	-0.192	0.705	0.830	0.639	0.894

Values in bold represent the square root of the AVE (average variance extracted); The values below the diagonal represent correlations between constructs. ACH: Achievement emotions, CHL: Challenge emotions, CMP: Compatibility, DTR: Deterrence emotions, FC: Facilitating conditions: CI: Continuance intention, LSS: Loss emotions, PEU: Perceived ease of use, PI: Innovativeness, PU: Perceived usefulness, SE: Self−Efficacy

Table 5

Discriminant validity (HTMT Ratio)

Factor	ACH	CHL	CMP	DTR	FC	CI	LSS	PEU	PI	PU	SE
ACH
CHL	0.913
CMP	0.672	0.678
DTR	0.267	0.232	0.244
FC	0.623	0.617	0.797	0.179
CI	0.672	0.672	0.907	0.266	0.759
LSS	0.255	0.211	0.249	0.893	0.184	0.266
PEU	0.709	0.716	0.781	0.184	0.832	0.762	0.175
PI	0.653	0.655	0.914	0.228	0.831	0.866	0.226	0.792
PU	0.748	0.765	0.758	0.206	0.742	0.747	0.209	0.856	0.747
SE	0.611	0.605	0.827	0.220	0.861	0.782	0.212	0.784	0.899	0.699

ACH: Achievement emotions, CHL: Challenge emotions, CMP: Compatibility, DTR: Deterrence emotions, FC: Facilitating conditions: CI: Continuance intention, LSS: Loss emotions, PEU: Perceived ease of use, PI: Innovativeness, PU: Perceived usefulness, SE: Self−Efficacy

After establishing the validity, the constructs were evaluated in terms of multi-collinearity. According to the variance inflation factor (VIF) (Table 5), VIF values ​​of all the predictive constructs tested were within appropriate ranges (Hair et al., 2011). Accordingly, the consistency of the prediction coefficients is good, and there is no problem in terms of linearity. Furthermore, the standardized root mean square residual (SRMR) value of 0.032 indicates a very good model fit.

Structural model

According to the PLS-SEM results, the proposed model explains 62.3% of PEU, 64% of PU, 73.5% of CI, 50.3% of ACH, and 52.2% of CHL. The hypothesis results demonstrate that 23 of the 25 tested hypotheses are supported. The results of the structural model evaluation are summarized in Table 6. As seen in Table 6, all hypotheses regarding PEU, PU, CI, CMP, FC, and SE were supported. The only unsupported hypotheses are PI->DTR (H10c) and PI->LSS (H10d).

Table 6

Hypothesis testing

Path	Coefficient	t-Value	p-Value	f2	VIF	Results
CI -> ACH	0.191	15.903***	0.000	0.025 c	2.927	Supported
CI -> CHL	0.176	14.463***	0.000	0.022 c	2.927	Supported
CI -> DTR	-0.191	13.567***	0.000	0.015 c	2.738	Supported
CI -> LSS	-0.195	14.482***	0.000	0.015 c	2.738	Supported
PEU -> CI	0.054	5.471***	0.008	0.003 c	3.357	Supported
PEU -> PU	0.533	54.399***	0.008	0.356 a	3.995	Supported
PU -> ACH	0.457	43.231***	0.000	0.202 a	2.076	Supported
PU -> CHL	0.487	46.488***	0.000	0.239 a	2.076	Supported
PU -> CI	0.106	11.692***	0.000	0.015 c	2.801	Supported
CMP -> CI	0.518	40.689***	0.000	0.254 a	3.974	Supported
CMP -> PEU	0.197	14.760***	0.000	0.027 c	3.768	Supported
CMP -> PU	0.202	15.905***	0.000	0.030 c	3.789	Supported
FC -> CI	0.027	2.569**	0.010	0.001 c	3.404	Supported
FC -> PEU	0.384	29.651***	0.000	0.131 b	2.983	Supported
SE -> DTR	-0.051	3.433***	0.001	0.002 c	3.291	Supported
SE -> CI	0.032	2.643**	0.008	0.001 c	3.976	Supported
SE -> LSS	-0.033	2.154*	0.031	0.001 c	3.291	Supported
SE -> PEU	0.132	9.826***	0.000	0.012 c	3.926	Supported
PI -> PEU	0.149	10.018***	0.008	0.011 c	5.237	Supported
PI -> PU	0.133	10.550***	0.002	0.012 c	3.995	Supported
PI -> CI	0.194	13.892***	0.000	0.027 c	5.335	Supported
PI -> ACH	0.132	11.682***	0.000	0.012 c	3.020	Supported
PI -> CHL	0.127	10.869***	0.000	0.011 c	3.020	Supported
PI -> DTR	-0.016	0.922(ns)	0.357	0.000	4.368	Not Supported
PI -> LSS	-0.025	1.433(ns)	0.152	0.000	4.368	Not Supported

p: ns ≥ 0.05; ∗ < 0.05; ∗∗ < 0.01; ∗∗∗ < 0.001. a Large effect size. b Medium effect size. c Small effect size

CI: Continuance intention, PEU: Perceived ease of use, PU: Perceived usefulness, CMP: Compatibility, FC: Facilitating conditions, SE: Self−Efficacy, PI: Innovativeness, ACH: Achievement emotions, CHL: Challenge emotions, DTR: Deterrence emotions, LSS: Loss emotions

All relationships between CI and emotional constructs were significant. Hence, H4a, H4b, H4c and H4d were supported. On the other hand, CI relationships of PEU, PU, ​​CMP, FC, SE, and PI were also significant. Thus, H1b, H3, H5c, H6b, H7b, H9c were supported. All the relationships with the core TAM constructs were significant in terms of CMP. In terms of SE, all the relationships with negative emotional outcomes were significant. Hence, H5a, H5b and H8a, H8b were supported. With regard to PI, all the relationships with the core acceptance constructs and positive emotional outcomes were significant. Thus, H9a, H9b and H10a, H10b were supported. The strongest relationships in the model are between PEU->PU (H1a), PU->CHL (H3a), and PU->ACH (H3b), respectively. As for mediator effects, the results showed that PU and PI indirectly affected ACH and CHL, while SE indirectly affected DTR and LSS through CI. Finally, effect sizes are large for PEU->PU, PU->ACH, PU->CHL, and CMP->CI, medium for FC->PEU, and small for other significant relationships.

Discussion

The model validated in this study, which aims to determine the factors that influence university students’ intention to continue using e-learning systems and examine the emotional outcomes of the continuance intention and external variables, explains PEU by 62.3%, PU by 64%, CI by 73.5%, ACH by 50.3.%, and CHL by 52.2% (Fig. 2). The results indicate that the model explains both the core acceptance constructs, and especially the positive categories of emotional outcomes at a high level. Additionally, the results suggest that the robustness and explanatory power of TAM are further improved, contributing to a better understanding of the relationships between users’ experiences with regard to using information technologies and their emotional responses. In line with this, it can be stated that the study helped fill a gap in the field and provided valuable findings for the improvement of online education both during and after the pandemic.

Fig. 2

PLS path modeling. CI: Continuance intention, PEU: Perceived ease of use, PU: Perceived usefulness, CMP: Compatibility, FC: Facilitating conditions, SE: Self−Efficacy, PI: Innovativeness, ACH: Achievement emotions, CHL: Challenge emotions, DTR: Deterrence emotions, LSS: Loss emotions

In terms of emotional outcomes, the results indicate that students’ intention to continue using the e-learning system affects ACH and CHL emotions positively, and DTR and LSS emotions negatively. Furthermore, all the relationships between CI and emotional outcomes are strong. CI-related relationships suggest that university students’ use of the e-learning system is beneficial and that the both systems in use provide successful user experiences that will trigger positive emotions.

The findings also suggest that the use of e-learning systems supports the experiences of challenge and achievement, including emotions such as enjoyment, playfulness and satisfaction, and weakens the negative outcomes of loss and deterrence, which include emotions such as anxiety, fear, frustration and disappointment. Although there is no current study that deals with the relationship between technology use and emotional outcomes in the field of education in a similar way, there are studies that reported similar results in the context of information technologies (Lu et al., 2019). Furthermore, the fact that positive and negative emotions are among the most effective experiences in the context of the success of integration is in line with the results of the study (Partala & Saari, 2015; Şahin, 2021).

With regard to the relationships of PU with ACH and CHL, findings indicate that students’ perceptions of the increase in performance that they can achieve using the e-learning system affect their emotions of achievement and challenge. The positive relationships between these constructs suggest that students are more likely to experience emotions of achievement and challenge if they have strong perceptions of the benefit that the use of e-learning systems can provide. Additionally, considering that PU is expressed as one of the strongest motivators in the context of technology acceptance (Şahin et al., 2021; Şahin & Şahin, 2021; Şahin, 2021), the effect of motivation of the fulfillment of tasks, and the connection of positive emotions to task completion using technology (Beaudry & Pinsonneault, 2010; Lee et al., 2012; Lu et al., 2019), an inference can be made that the relationships of PU with ACH and CHL are supported. In addition, based on the mediating effects, it can be stated that students’ perceptions of the increase in performance that can be obtained with the use of e-learning systems also contribute to the emergence of positive emotional outcomes by supporting the tendency to continue using them.

In the context of SE, it is observed that university students’ self-efficacy has a reverse effect on negative emotions. The results indicate that university students are less likely to experience DTR and LSS emotions if their self-efficacy is high. The direct relationship of SE with technology-related knowledge and skills (Compeau & Higgins, 1995), its connection with users’ beliefs about the difficulty of using technology, and their tendency to use technology if they regard themselves as competent in technology use (Şahin et al., 2021; Şahin, 2021) suggest that university students’ belief in their skills can help reduce their negative experiences. The results indicate that university students who are competent in using e-learning systems will have more positive user experiences and less negative emotional outcomes. On the other hand, the findings related to the fact that efficacy also indirectly affects the emotions of deterrence and loss by helping to strengthen students’ continuation intention to use e-learning systems are also in line with this inference.

The relationships of PI with ACH and CHL were significant, but not with negative DTR and LSS. The insignificant PI->DTR and PI->LSS relationships also represent the only hypotheses not supported in the model. The results indicate that university students who are more curious about new technologies and more willing to try new technologies are more likely to have positive experiences in using e-learning systems. Moreover, the facts that innovativeness includes personality traits such as adapting to uncertainties and risk-taking (Baylor & Ritchie, 2002; Kirton & De Ciantis, 1986) and non-innovative university students are not open to changes and show resistance (Şahin, 2016) suggest that the effects of PI on emotional outcomes triggering positive experiences are supported. In parallel, the indirect effects of PI on ACH and CHL through CI indicate that innovativeness also positively affects the emotions of achievement and challenge categories by strengthening the continuance intention.

In terms of relationships between PEU, PU, and CI, the findings indicate that university students’ perceptions of the increase in performance they will achieve using e-learning systems and the effort required for effective use of the system are influential on continuance intention. The study findings with regard to PEU and PU, suggest that these two motivational core factors are strongly related to university students’ continuance intention. However, although the findings of previous studies are mostly parallel (e.g. Chang et al., 2017; Rafiee & Abbasian-Naghneh, 2021; Salloum et al., 2019; Tarhini et al., 2014; Tarhini et al., 2017), there are also findings indicating that the effect of PEU and PU is weakened for instructors and students in higher education due to the mandatory use of technology during the pandemic (Şahin et al., 2021; Şahin & Şahin, 2021). Accordingly, the findings lead us to the fact that users can have different experiences even if the same technology is used on both the teaching and learning sides of online education. From this point of view, it can be stated that there is still a gap regarding the user experience and there is a need for up-to-date studies.

With regard to SE, the findings indicate that students with high self-efficacy perceive the use of an e-learning system as easier and will be more likely to continue using. These findings, which generally overlap with the literature (e.g. Moreno et al., 2017; Rafiee & Abbasian-Naghneh, 2021; Valencia-Arias et al., 2019), suggest that it is important for students to be technologically competent for the continued use. On the other hand, FC->CI indicate that university students with resources such as technical support and infrastructure will tend to continue using. Considering the importance of the resources and supports that users will have and that will facilitate the use and process in the context of technology use in education (Garone et al., 2019; Şahin et al., 2021), it can be stated that the study findings are supported. With regard to FC->PEU, which represents one of the strong relationships, the results suggest that the conditions that facilitate the use of the system, and therefore learning, help students perceive the use of the e-learning system as a less demanding task. The results also points out that facilitating conditions influence perceived usefulness through ease of use, and emphasizes the role of the facilitators->ease of use->usefulness chain.

In terms of CMP, the findings indicate that compatibility of e-learning systems with the education is determinative for university students in terms of the potential benefit to be obtained from these technologies, the effort required for effective use, and continuance intention. It is emphasized that meeting the expectations of university students from e-learning systems and their learning needs through e-learning is a critical factor for the intention to use and the compatibility of the education with the e-learning system functions as an important motivator (Al-Rahmi et al., 2019; Chen, 2011). Moreover, it is stated in the literature that the compatibility is a priority for university students (Şahin & Şahin, 2021; Şahin et al., 2021).

In the context of PI, it can be stated that innovativeness affects the perception of performance increase and ease of use and the intention to continue using the e-learning system. Considering that innovativeness represents the willingness to try information technologies, it can be said that the findings are supported in terms of intention. Furthermore, the fact that innovativeness has a broad framework that includes characteristics such as mental curiosity, open-mindedness, imagination, originality, knowledge-seeking, risk-taking, having vision, and leadership (Baylor & Ritchie, 2002; Bozionelos et al., 2014; Kılıçer & Odabaşı, 2010; Şahin, 2016; Şahin, 2021) suggests that innovative students will find e-learning systems useful and easy to use. Based on this information overlapping with the findings, it can be stated that university students will tend to continue using e-learning systems if they are innovative and that students can be triggered more easily with extrinsic motivation characteristics of e-learning systems such as usefulness and ease of use.

Conclusion and implications

In this study, the factors that influence the intention to continue using e-learning systems of university students were identified and the emotional outcomes of system use were examined. To the authors’ knowledge, this study is one of the rare studies conducted during the mandatory online education that comprehensively address emotional outcomes of online technology use from both positive and negative aspects. In this respect, it is one of the main contributions of this study to provide information that will help fill a gap in the field with regard to emotional user experience. In addition, examining the influence of innovativeness, which is an important personal trait in terms of technology use (Agarwal & Prasad, 1998; Şahin, 2016) but not adequately addressed with regard to emotional experience in path modeling studies in the field of education, can be expressed as another important contribution of this study. On the other hand, the fact that the model, which offers high explanatory power (e.g. CI=73.5%, ACH=50.3%, CHL=52.2%.), provides a solid theoretical infrastructure in terms of user experience and emotional outcomes regarding the continuation of the use of educational technologies can be counted among the contributions to the field.

One of the major conclusions of the study is the significance of the perception of the effort required for the effective use and perception of the performance increase that can be achieved with the use of the e-learning system, and the vital role of the system’s potential to meet the educational expectations of the students. This suggests that compatibility of the e-learning system with the online education is a priority for the students. Another major conclusion is the crucial role of the beliefs of students in their skills, tendency to try new technologies and resources such as technical support and training. Accordingly, these conclusions emphasize that both intrinsic and extrinsic motivators and personality traits should be taken into account by system designers, instructional technologists and policy makers to maintain the quality of online education during the COVID-19 pandemic (Lu et al., 2019; Şahin et al., 2021; Şahin, 2021).

An important result of the study showed that self-efficacy influences emotions of deterrence and loss, and students’ beliefs in their skills affect emotional user experience. Therefore, students’ perceptions of their competencies with regard to e-learning systems have an significant role in whether their negative emotional experiences such as anxiety and frustration become dominant. From this point of view, instilling students with knowledge and skills about online educational technologies in order to ensure continued use and positive user experiences should be regarded by both instructors and instructional designers as a determining factor for online education not to fail.

It is a striking finding that among the relationships tested in the study, by far the strongest ones (PEU->PU, PU->CHL, PU->ACH) were all related to perceived usefulness. These findings lead us to the conclusion that perceived usefulness is a decisive factor in creating user experiences that generate emotions of both challenge and achievement (e.g. enjoyment, playfulness, satisfaction). In light of the findings regarding the critical role of emotions in technology use, it is important that future studies employ emotions not only as output variables but also as antecedents and determinants, and focus on relatively under-studied emotions such as anticipation, relief, frustration and disappointment rather than examining certain well-studied emotions (e.g. enjoyment, anxiety). In this way, the theoretical background of emotions in the acceptance of technology can be developed further and vital contributions can be made to the emerging body of literature. In practice, policy makers’ implementation of practices that will ensure regular assessment of students’ emotions and evaluation of the personality traits of students (Lu et al., 2019; Şahin, 2021) can serve as effective tools for more efficient results of approaches that have proven their potential in education, such as e-learning and mobile learning (Abdullah & Ward, 2016; Al-Emran et al., 2018). On the other hand, the findings also lead us to the conclusion that the link between perceived ease of use and perceived usefulness is crucial. At this point, program, instructional and educational technology designers have a vital role in designing e-learning systems to include a wide variety of audio and written materials that are compatible with the content of the courses, provide synchronous and asynchronous access, and provide materials that facilitate learning such as summary videos and solved questions. In this way, both students’ perceptions of the benefit and performance increase they will gain by using these systems will be more positive, and a significant performance support will be provided in real terms. In addition, it should be considered crucial to strengthen students’ perceptions that they can use the e-learning system more effectively with less effort by supporting students in terms of self-confidence and competence by providing a clear and understandable user-friendly interface.

In the study, it was concluded that innovativeness is influential on both core acceptance constructs and positive emotional outcomes. Although these represent one of the major concslusions of the study, the inadequacy of up-to-date studies examining innovativeness in the context of user experience points to a gap in the field of education. In addition, to the authors’ knowledge, there are no recent studies in the field of education focusing on the effects of innovativeness on the emotional outcomes of technology use. From this point of view, future research can provide valuable contributions to the field by comprehensively addressing the factors influencing user experience and examining the roles of personality traits such as innovativeness in terms of outcomes of technology use. Acquiring knowledge about innovativeness categories (Rogers, 1995, 2003) of the students that can be used to improve students’ user experiences can be presented as another important implication. Finally, in addition to the 23 direct effects determined within the scope of the study, many indirect effects were also identified. Although this manuscript has focused only on the major ones, it can be stated that the importance of mediation effects in the context of educational technology use is evident. In this direction, focusing on mediation effects in future studies and examining mediation models in depth can provide important theoretical contributions to the field.