Adolescent emotion scale for online lessons: A study from Turkey.

People's day-to-day routines have changed drastically since the outbreak of the COVID-19 pandemic. One of the changes to take place has been the transition to online learning due to the changing conditions in learning environments. One of the factors that guide students through learning environments is their emotions. The few existing scales that measure the emotions of adolescents in learning environments have been developed with consideration of face-to-face learning environments and their items do not adequately express the state of online environments. For this reason, this study aimed to develop a scale which reveals the emotions of adolescents that may affect their academic success with regard to this transition of learning environments as they attend online lessons. A total of 3,655 middle and high school students living in Istanbul, Turkey, participated in the study. The scoring system was designed with regard to the frequency with which each student reading an item experiences the relevant emotion for the item in the online learning environment. Exploratory and confirmatory factor analyses conducted show the scale to possess a four-dimensional structure. These dimensions are enjoyment-pride, boredom-hopelessness, loneliness and anxiety. For all dimensions of the scale, Cronbach alpha values range between .93 and .83. 'Positive and Negative Affects Schedule-Children' was used to confirm the scale's criterion-related validity, and correlations between dimensions were examined. After validity and reliability studies, the Adolescent Emotion Scale for Online Lessons was found to be sufficient. The scale can be used by experts and teachers who wish to determine adolescents' emotions during a period of online learning.

Introduction

Individuals are guided by their impressions, beliefs and emotions when interpreting their experiences in unexpected circumstances, such as the COVID-19 pandemic. Emotions have an impact on how we act and think, as well as what we learn and how we learn it (Chemi et al., 2007). Emotional engagement is necessary for students to maintain interest and achieve significant achievement in a subject (Bhansali & Sharma, 2019). Strong emotions carry with them large amounts of energy and provide important, life-sustaining motivation for the occurrence of goal-oriented behavior (Çeçen, 2002).

Parallel to these radical changes in daily life, many governments around the world have had to temporarily shut down their educational institutions for certain periods depending on the spread of the virus in order to better control the spread of the COVID-19 disease. In order to cope with education being put on pause, many countries have begun to use distance or hybrid learning methods. These nationwide shutdowns affect over 90% of the world’s student population, and their academic emotions (UNICEF, 2021). This experience, independent of the COVID-19 virus, indicates that the online education approach will now be used as an approach to continue the education process for younger students as well.

Online learning, by its nature, is an educational model that puts most of the onus of learning on the learner and expects self-regulated and self-directed learning skills (Knowles, 1975 cited in Bozkurt, 2020) from them. However, environments that require these skills are more suited to adult learners. For applications at the K-12 level, in terms of the cultivation of positive and effective learning experiences, it is important for learners to be supported with respect to these skills and given guidance.

In this context, the emotions exhibited by students while gaining new experiences, such as attending lessons, doing homework or studying as part of distance education, now hold great significance. Negative influences on individuals’ emotions affect learning greatly and lead to a decrease in student persistence (Rovai, 2002) through lower individual motivation (Osguthorpe & Graham, 2003). If emotions are considered to be an important component of how we understand and make sense of events, they will have an impact on whether our current practices are perpetuated or changed (Chemi et al., 2007). For this reason, it is significant to understand students’ emotions in online environments.

Emotions and learning

Despite there being a large number of studies in the literature concerning emotions, the issue has been discussed since William James made serious attempts at finding answers in 1884, and it is quite difficult to come up with a definition of emotions that is unanimously agreed upon (Cabanac, 2002; Kleinginna & Kleinginna, 1981; Plutchik, 1965; Scherer, 2005). The reason for this variance in definition could be that emotions are neurobiological constructs from one point of view and contain social and psychological dimensions from another.

Personal experiences guide the reactive ways in which individuals demonstrate their emotions. Central neural mechanisms are discussed that lead to partially identical emotional reactions being expressed by all people and, in parallel to this, it is known that the seven basic emotions are universally expressed by the same facial expressions or body language (Ekman & Cordaro, 2011); anger, fear, surprise, sadness, disgust, contempt and happiness. On the other hand, individuals experience emotions specific to the academic environments in which they exist as students.

Previous research on adolescent students' academic emotions

For each of the three basic types of academic situations: attending class, studying, and taking tests and exams, there are a variety of different emotions (Pekrun & Bühner, 2014). In academic contexts, achievement-related emotions such as joy of learning, hope, pride, anger, anxiety, or boredom are widespread, persistent, diverse, and often intense (Pekrun, 2016). Students may feel pride or joy when they receive high marks at school; similarly, they may feel anxious or bored when they are unable to attend a class. In their study conducted with 7th to 10th grade students, Goetz et al. (2006) reached the conclusion that academic emotions (emotions directly pertaining to learning, class education and success) are correlated with students’ cognition and the attributes of their social environment. Pekrun et al. (2002b) show that students experience many positive academic and social emotions resulting from class duties when they take exams or lectures, that negative emotions such as test anxiety are felt at least as intensely as positive emotions and that these emotions are caused by personal evaluations or educational and social antecedents.

It is as hard to measure emotions as it is to define them, because in order for a realistic measurement to be possible, individuals must be conscious of the emotions they experience. Emotional awareness is defined as the ability to identify emotions felt by themselves and others and, separately from the experience and expression of an emotion, it involves the presence of knowledge on the emotion being felt at a particular moment (Kuyumcu, 2012). Since the 1930s, when test anxiety research began, self-report methods for measuring students' and teachers' emotions have made significant development. Except for the Academic Emotions Questionnaire and Epistemic Emotion Scales, there are few self-report assessments of students' positive academic emotions. Students' anger and boredom, in contrast to positive academic feelings, have piqued experts' curiosity (Pekrun & Bühner, 2014).

In one study on the measurement of emotions in academic environments (during lessons, before exams, during exams and suchlike) the taxonomies of different academic emotions are studied and the Academic Emotions Questionnaire developed to understand students’ emotions (enjoyment, hope, pride, relief, anger, anxiety, shame, hopelessness, and boredom (Pekrun et al., 2002a) in learning environments. In another study, the Achievement Emotions Questionnaire was developed, and designed in order to evaluate the various achievement emotions felt by students during lessons, while studying, before exams and while taking exams; enjoyment, hope, pride, relief, anger, anxiety, shame, hopelessness and boredom (Pekrun et al., 2011). Further, the Achievement Emotions Questionnaire-Elementary School was developed which evaluates the achievement emotions (enjoyment, anxiety and boredom) of primary school students in Germany and the United States of America, with the differences between cultures being highlighted (Lichtenfeld et al., 2012). In a recent study, Bieleke et al. (2021) conducted research on the short form of this scale.

A number of the relevant studies focus on emotions related to mathematics and physics lessons. The Academic Emotions Questionnaire—Mathematics, designed to measure emotions for mathematics (enjoyment, pride, anxiety, anger and shame), was applied to eighth grade students in Germany and China with a high similarity being observed between cultures (Frenzel et al., 2007). In another similar study, the Achievement Emotions Questionnaire was applied to students in order to evaluate their emotions (enjoyment, relief, hope, pride, anger, hopelessness, anxiety, boredom, shame) regarding their classrooms and tests for mathematics (Peixoto et al., 2015). Another study attempts to study students’ emotional experiences, and examines the emotions experienced (joy, pride, contentment, worry, shame, hopelessness) before and after the solving of mathematics problems. It finds a decrease in joy and contentment after problem-solving, performance only causing hope to decrease, and that the feeling of achievement is effective in increasing joy, pride and shame (Tornare et al., 2015). In another study, Bhansali and Sharma (2019) studied the reliability and internal validity of the adapted Achievement Emotions Questionnaire (AEQ-PhysicsPrac).

The Current study

As seen in the example given above, the emotions felt by students during learning, with their separate dimensions, have only been analyzed with regard to face-to-face environments. In this study, the emotions of students who mandatorily switched to online learning during the COVID-19 period are focused on. As stated by Moore (1997) in transactional distance theory, distances are observed for student–student, student-instructor and student-content interactions in online learning. These distances are not purely geographical, but also express educational and psychological space. These perceived distances are determined by interaction and dialogue between student and instructor, capability of distance learning curriculum to meet students’ needs, and students’ autonomy and ability to govern themselves (Moore, 1997). The rapid switch to online learning with inadequate preparation due to the COVID-19 pandemic has caused the educational and psychological distance in question to increase.

On the K12 level, since online learning is not a widely preferred distance learning environment, the emotions adolescents feel related to their learning processes in this environment constitute an understudied field. Accordingly, the purpose of the current study has been deemed the development of a scale for understanding adolescents’ emotions in online learning environments, which are seeing widespread use in the current academic period and are very likely to see applications in future academic periods. There are three main reasons for this study:

Firstly, emotions are affected by the domain or the atmosphere of the classroom (Boekaerts, 2007). Pekrun and Bühner (2014) state the existence of emotional traits and states. While individual inclinations to regularly experience a specific type of emotion are referred to as emotional traits, state emotions are described as feelings felt at a certain time and place, such as a student's nervousness before an exam begins (Pekrun & Bühner, 2014). Thus, existing measures provide separate scales for class-related, learning-related, and test-related emotions in face-to-face environments. However, the concept of “class” that students encounter in online learning environments during the pandemic is different from the one they have become familiar with in face-to-face learning environments. For instance, the phrase “I enjoy being in class” (Pekrun et al., 2011) does not fully express the learning environment that the student attends through a monitor. For this reason, it will be appropriate for a scale to be developed that will aid in understanding adolescents’ emotions pertaining to their experiences in the online learning state, which differs significantly from the face-to-face environments they have become used to in terms of time and space.

Secondly, the existing scales have been devised with face-to-face school environments in mind, with wording and expressions chosen to that end. Therefore some of the expressions in these scales are not found to be appropriate for distance learning. Examples of these items include the following: “I am confident when I go to class” (Pekrun et al., 2011), “I take pride in being able to keep up with the material in my math class” (Peixoto et al., 2015), “After I have said something in class I wish I could crawl into a hole and hide” (Bieleke et al., 2021), “I think about what else I might be doing rather than sitting in this boring class” (Bieleke et al., 2021). Items such as these include expressions that imply physical existence within the class environment. It is not possible for these items to be utilized in understanding emotions in online learning environments. Rewriting these items to express the state of attending lectures through a monitor may distort their meanings or not adequately reflect the situation.

Lastly, expressions appearing in similar scales, such as “I enjoy math class” (Lichtenfeld et al., 2012) have been developed with regard to a specific subject. However, the scale developed in this study aims to understand the emotions of adolescents while attending online classes generally, instead of their emotions pertaining to particular subjects. In order for the scale to serve this purpose, the items of the scale have to be written for lessons in online environments in general, as opposed to being written for particular classes.

In fact, schools not only support academic development, but also provide significant psychological and social outcomes for children. Despite having their own unique talents, students may not strive to learn the subjects of their lessons or to succeed in them. In this case, emotions such as interest, enjoyment, boredom and others can be accepted as important forces behind student learning. In many cases, boredom is a product of situational meaninglessness (Chan et al., 2018). It is important to understand what is being done right and what is being done wrong throughout this process. The purpose of this study is the development of an Online Lesson Adolescent Emotions (OLAE) scale to reveal the process-related emotions of middle/high school students experiencing distance education and attending online lessons.

Method

Study group

Upon acquiring Ethical Council approval and the necessary formal permission for the final form of the development of the scale, the scale was sent electronically within a space of fifteen days, in June 2020, to students who were also asked to complete it electronically. Participants of the study consist entirely of volunteer students who agreed to fill out the scale form electronically. All of the students stated that they were continuing to participate in online lessons. Analyses were conducted on a dataset consisting of 3,655 people after necessary controls were undertaken on the data acquired through data collection, and missing/erroneous data was identified with 491 being removed.

The participants of this scale development study were 3,655 (2215 female, 1440 male) middle and high school students. 2,224 of the students attended public schools, while 1,429 attended private schools. Distribution of the students in the study group according to school years is presented in Table 1:

Table 1

Distribution of students in the study group

	Frequency	Percent
6th Grade	567	15,5
7th Grade	483	13,2
8th Grade	466	12,7
9th Grade	718	19,6
10th Grade	619	16,9
11th Grade	802	21,9
Total	3655	100,0

As can be seen in Table 1, 11th grade (21.9%) is the most common school year among students in the study group, while 8th grade (12.7%) is the least common. Due to the COVID-19 pandemic, all of the students had been attending online classes for 2.5 months prior to the date of the study.

Development of scale

For the development of the scale, the factors to be considered were decided initially. For this purpose, the researchers have evaluated existing scales and collaborated with an expert from the field of psychological counselling in order to discuss and determine which of the fundamental emotions studied in literature would be observed in online learning environments. For this purpose, the affective elements that influence the facilitation of learning in the online education period were attempted to be determined, and the emotions that affect learning were comparatively analysed so that a consensus could be reached on the six fundamental emotions that will have an especially significant effect on the adolescent group. At this stage, similar research and scales from the literature (Çivitci, 2007; Frenzel et al., 2007; Lichtenfeld et al., 2012; Peixoto et al., 2015; Pekrun et al., , 2002a, 2002b, 2011; Siyez & Kaya, 2008; Tornare et al., 2015; Yıldız, 2014) were evaluated with regard to their items and factors.

Accordingly, a minimum of four items were written for each emotion with consideration of the related literature and similar scales. The items developed were shared with four separate experts in the field of psychological counselling and distance learning, and their opinions and approval were sought on the matters of the appropriacy of the emotions on the scale for the distance learning environment and the relevancy of each item with the emotion they were placed under. The necessary changes were implemented according to the experts' feedback. Following this, the items acquired were also sent to two academics from two different universities as well as to two experts working in the field of psychological counselling in secondary education bodies. They were asked to grade items in the scale from 1 to 5 in terms of suitability for purpose and comprehensibility in terms of adolescents' emotions in an online learning environment. Necessary changes were made in item expressions with consideration of feedback from the experts.

In this way, twenty-nine items expected to be distributed across six factors were acquired with respect to the expected emotions. The item pool acquired before validity and reliability studies contains twenty-nine items (seven for enjoyment, four for pride, four for boredom, four for hopelessness, four for anxiety, and six for the loneliness factor). The final scale was sent for a final evaluation to an academic in the field of scale development.

The items related to emotions in the acquired draft scale, consisting of a total of 29 items in 6 dimensions, are elaborated below.

Enjoyment: Seven items for the scale were developed with respect to students’ enjoyment of the online learning environment. One instance of this factor being, ‘I enjoy studying using a screen’.

Pride: Four items were developed for the scale in accordance with this factor. One of the items being, ‘I feel as though I have accomplished something important when I turn off the screen at the end of a class’.

Boredom: Four items in the scale were developed in consideration of this factor. An example item being, ‘I find taking online lessons boring’.

Hopelessness: Four items in the scale are directed towards the hopelessness felt by students in online learning environments. An example item being, ‘I have no expectations from online lessons’.

Anxiety: Four items were developed for this factor in the scale. One instance of these items being, ‘I feel anxious during online lessons’.

Loneliness: Six items for the scale were developed with respect to students’ loneliness in the online learning environment. An example of this factor is, ‘Online lessons make me feel lonely’.

The scale items were graded in accordance with a 5-point Likert scale. Grading was designed to be handled according to the frequency with which the student reading the item felt the relevant emotion in the online learning environment. The feeling frequencies for each item were arranged as, ‘Always’, ‘Most times’, ‘Sometimes’, ‘Rarely’ and ‘Never’, with grading as follows: ‘Always’ = 5, ‘Most times’ = 4, ‘Sometimes’ = 3, ‘Rarely’ = 2, ‘Never’ = 1. A total of the grades is not calculated, since each emotion is measured separately. The magnitude of the sum total for each factor is interpreted as the degree to which the relevant emotion is felt.

Data collection tools

Positive and Negative Affects Schedule-Children (PANAS-C)

The PANAS-C positive and negative emotion scale was applied to the entire study group as part of criterion-related validity. The PANAS-C scale was created by Laurent et al. (1999) and adapted to Turkish by Yıldız (2014). Confirmatory factor analysis was applied to a data set collected from 414 adolescents, with 262 girls (63.6%) and 152 boys (36.7%) aged 14–19, in the Turkish adaptation of the PANAS-C scale. A short form made up of fourteen items, seven positives and seven negatives, were acquired as a result of the analysis. Fit indices acquired at the end of analysis were found to be RMSEA = 0.06, SRMR = 0.04, GFI = 0.94, AGFI = 0.91, CFI = 0.98, and Critical N (CN) = 234.82 (Yıldız, 2014).

Analysis of data

Exploratory and confirmatory factor analyses were conducted on the acquired data in order to develop the scale. For two separate analyses, the data set was divided into two files of 1,827 and 1,828 lines, using SPSS’s random file separation function. Prior to these analyses, Kaiser–Meyer–Olkin and Bartlett tests were conducted in order to identify the suitability of the data for factor analysis. The findings showing that these values are statistically meaningful was accepted as an indication that exploratory and confirmatory factor analyses could be conducted on the model. In exploratory factor analysis, the Varimax axis rotation technique, being an oblique rotation technique, was used with the thought that the dimensions are correlated (Pallant, 2010). This rotation technique was preferred on account of being more appropriate for future use relative to Oblimin, being another oblique technique (Büyüköztürk, 2002).

Normality analysis was conducted for all of the items in the data set, with skewness and kurtosis values and Shapiro Wilks normality test significance values considered, and normality conditions observed to be secure for all items. Additionally, Pearson correlation analyses for inter-dimensional relationships to be understood, Variance inflation factor for testing confirmatory factor analysis conditions, and Cronbach Alpha, Average Variance Extracted and Composite Reliability calculations for measuring scale reliability were conducted.

Findings

Construct validity

Exploratory Factor Analysis (EFA)

For the first phase of factor analysis, the twenty-nine items on the scale were worked on and the Varimax rotation technique was applied on the data set comprising 1,827 data points. At this point, three items (10, 13 and 20) were removed and analysis repeated due to being distributed among multiple factors with a load value above 0.30. In the new analysis, the Kaiser–Meyer–Olkin (KMO) value was found to be 0.95 and the Bartlett test of sphericity result value was seen to be statistically meaningful [X2 = 31,985.582, Sd = 325, p < 0.000). These values show that the data set is suitable for factor analysis (Pallant, 2010). In the second phase it was found that the sub-dimensions on the scale are collected under four factors with eigenvalues greater than one. All items explain 66.3% of the total variance, with the acquired factor loads ranging between 0.875 and 0.533. The distribution of the items into factors after application of the Varimax rotation technique is shown in Table 2 and the scree plot in Fig. 1.

Table 2

Second phase EFA results of the Adolescent Emotion Scale for Online Lessons (with Kaiser normalization after Varimax orthogonal rotation)

Item #	Factor	F1	F2	F3	F4
24	Pleasure	0.821
25	Pleasure	0.816
12	Pleasure	0.782
2	Pleasure	0.766
18	Pleasure	0.753
10	Pleasure	0.735
23	Pride	0.732
26	Pleasure	0.728
9	Pride	0.722
4	Pride	0.658
5	Boredom		0.802
6	Hopelessness		0.777
15	Hopelessness		0.719
17	Boredom		0.715
11	Boredom		0.711
13	Hopelessness		0.700
19	Boredom		0.661
16	Hopelessness		0.533
20	Loneliness			0.875
21	Loneliness			0.802
22	Loneliness			0.777
7	Loneliness			0.739
14	Anxiety				0.791
8	Anxiety				0.780
1	Anxiety				0.745
3	Anxiety				0.678
	Explained Variance % (Total = % 66.281)	24.777	18.732	11.616	11.156

Fig. 1

Scree plot

The findings acquired and presented in Table 2 alongside the scree plot given in Fig. 1 reveal that the Adolescent Emotion Scale for Online Lessons contains a four-dimensional structure related to middle and high school students’ emotions. Contrary to expectations, it can be seen that enjoyment and pride related items are collected in the same dimension, explaining 24.8% of the variance. Factor loads in this dimension range between 0.821 and 0.658. The second factor explaining 18.7% of variance, encompasses boredom and hopelessness items. Once again, contrary to expectations, two emotions are grouped together in this dimension, with factor loads ranging between 0.802 and 0.533. It can also be seen that the third factor, which explains 11.6% of variance, includes items related to loneliness, and the last factor explaining 11.2% of variance includes items relating to anxiety. Factor loads for these items range between 0.791 and 0.678. The acquired structure does not show a total agreement with studies conducted across the world on middle and high school students’ emotions. This situation may be explained by cultural differences.

For the acquired four-dimensional structure comprised of twenty-six items, there are no issues related to overlap in item loads or less than three items in one dimension, so the scale can be proclaimed sufficiently valid in this state. A Confirmatory Factor Analysis is conducted on this structure in the next phase.

Confirmatory Factor Analysis (CFA)

DFA operations are performed on the portion made up of 1,828 items from the previously divided dataset. Prior to the analyses, mean and standard deviation calculations for three dimensions of the scale, alongside correlation coefficients between dimensions and Variance Inflation Factor (VIF) values, were examined in order to reveal relationships between predictor variables and, therefore, to determine whether or not a multi-collinearity issue is present. Less than 0.80 correlation between implicit variables (Kline, 2011) and VIF values of less than five show that connection multiplexing issues do not exist (Montgomery, 2001), and mean that the data set is suitable for regression analyses. The values in question are given in Table 3.

Table 3

Means and Standard Deviations, Variance Inflation Factor (VIF) and Interdimensional Correlation Values for the Adolescent Emotion Scale for Online Lessons

Factor	X_	Sd	VGF				Correlation
			[1]	[2]	[3]	[4]	[1]	[2]	[3]	[4]
Pleasure and Pride [1]	2.62	1.01	-	1.49	1.22	1.45	1	-0.633**	-0.182**	-0.283**
Boredom and Hopelessness [2]	2.68	1.06	1.10	-	1.15	1.21	-	1	0.386**	0.531**
Loneliness [3]	2.96	1.19	1.7	2.16	-	1.40	-	-	1	0.353**
Anxiety [4]	2.18	0.99	1.7	1.91	1.18	-	-	-	-	1

According to Table 3, the average values of points gathered from the 5-point Likert scale range from 2.96 to 2.18, while standard deviations vary from 1.19 to 0.99. Cross-dimensional correlations in the scale range between -0.63 and -0.18, and are meaningful at a level of 0.001 for all values. According to these findings, there is a strong (r = -0.63) inverse significant correlation between the scale’s boredom and hopelessness, enjoyment and pride measuring dimensions. The boredom and hopelessness dimension also showed at a medium level of significant correlation (r = 0.53) with the anxiety-related dimension. Correlation coefficients being smaller than 0.80 and VIF values presented being smaller than five show that the data set is suitable for CFA.

Accordingly, the structural equality modelling approach and partial least squares fit hypothesis of the model were tested and, therefore, the first stage CFA was conducted. For this purpose, in compliance with the model presented in the original scale article, the model was tested with CFA on the implicit variables of enjoyment and pride, boredom and hopelessness, loneliness and anxiety, and Fit Indices were calculated for the model. The model containing coefficients from the conducted confirmatory factor analysis is presented in Fig. 2.

Fig. 2

Acquired CFA Model

As can be seen in Fig. 2, correlations between observed variables and implicit variables for the model acquired through the CFA vary between 0.88 and 0.53. For an examination of this model’s suitability, χ2/sd, GFI (Goodness-of-Fit index), AGFI (Adjusted Goodness-of-Fit index), CFI (Comparative Fit Index), RMSEA (the Root Mean Square Error of Approximation) and SRMR (Standardized Root Mean Square Residual) values were accepted as critical criteria. Fit Indices for the model are presented in Table 4.

Table 4

Fit Indices and Recommended Cut-off Values for Fit Indices for the Adolescent Emotion Scale for Online Lessons

Fit indices	Adolescent Emotion Scale for Online Lessons Scale	Good fit	Acceptable fit
Χ2 / sd	4.84	≤ 2	≤ 5
GFI	0.95	≥ 0.95	≥ 0.85
AGFI	0.92	≥ 0.95	≥ 0.85
CFI	0.97	≥ 0.95	≥ 0.80
RMR	0.080	≤ 0.05	≤ 0.08
SRMR	0.050	≤ 0.05	≤ 0.08

As can be seen in Table 4, all of the fit indices are within the recommended cut-off values and are close to a good fit (GFI = 0.95; AGFI = 0.92; CFI = 0.97; RMR = 0.08; SRMR = 0.062; RMSEA = 0.050). Looking at the structure acquired as a result of the CFA and the calculated fit index values, the scale can be said to meet the necessary structure validity criteria.

Criterion-related validity

Within the scope of criterion-related validity, the PANAS-C Positive and Negative Affect scale was also applied to the entirety of the study group. Scale studies were conducted with 14–19 age group (M = 16) and it consists of fourteen items including seven positive and seven negative affects (Yıldız, 2014). Correlation values between scale dimensions and PANAS positive and negative affects for all of the students in the study group (N = 3655) are given in Table 5.

Table 5

PANAS Scale correlation of the Adolescent Emotion Scale for Online Lessons dimensions

	PANAS-C Positive	Pleasure and Pride	PANAS-C Negative	Boredom and Hopelessness	Loneliness	Anxiety
PANAS-C_Positive	1	0.70*	-0.39**	-0.49**	-0.12**	-0.28**
Pleasure and pride		1	-0.42**	-0.62**	-0.17**	-0.28**
PANAS-C_Negative			1	0.61**	0.36**	0.59**
Boredom and Hopelessness				1	0.39**	0.54**
Loneliness					1	0.37**
Anxiety						1

As can be seen in Table 5, correlation between the enjoyment and pride dimension of the Adolescent Emotion Scale for Online Lessons and the PANAS-C Positive Affects scores is significantly and positively high (r = 0.70). Middle and low level statistically significant correlations are present with PANAS-C Negative Affects scores. Similarly, PANAS-C Negative Affects scores show a positively high statistically significant correlation with the boredom and hopelessness dimensions of the Adolescent Emotion Scale for Online Lessons, medium correlation with anxiety and low correlation with loneliness. Additionally, correlations with positive and negative affect-oriented dimensions are statistically significant in the expected directions. These findings can be considered as evidence of the validity of the Adolescent Emotion Scale for Online Lessons.

Reliability

Cronbach Alpha, Average Variance Extracted (AVE) and Composite Reliability values were calculated and are presented in Table 6 according to EFA and CFA findings in order to determine the reliability of the scale structure. Average variance extracted and composite reliability are calculated with factor loads and error ratios acquired through exploratory factor analysis (Hair et al., 1997).

Table 6

Cronbach Alpha coefficients and Average Variance Extracted (AVE) of the Adolescent Emotion Scale for Online Lessons

	Cronbach Alpha Value	Average Variance Extracted (AVE)	Composite Reliability
Pleasure and Pride	0.93	0.71	0.93
Boredom and Hopelessness	0.92	0.50	0.89
Loneliness	0.86	0.64	0.88
Anxiety	0.83	0.57	0.84

The findings related to scores acquired via separate summation of scale items, given in Table 6, show that Cronbach Alpha values range between 0.93 and 0.83. AVE and composite reliability values for all of the dimensions are above threshold values of 0.5 (Jörg et al., 2016) and 0.7 (Hair et al., 1997), respectively. These values show that the scale has sufficient internal consistency. The findings resulting from conducted exploratory and confirmatory factor analyses and internal consistency analyses show that the Adolescent Emotion Scale for Online Lessons is a valid and reliable scale.

Scale scoring

The scale is made up of twenty-six items. All of the items are positively keyed without any opposite items. Possible scores for the enjoyment and pride dimensions containing positive emotions of a 5-point Likert scale range from 10 to 50. For boredom and hopelessness, the scale containing negative emotions have possible scores ranging from 8 to 40, while the loneliness and anxiety dimensions can be scored between 4 and 20.

Discussion and conclusion

The purpose of this study is to understand the emotions of middle and high school students related to online education and the development of an Online Lesson Adolescent Emotions scale. Exploratory factor analysis and confirmatory factor analysis were carried out in order to examine the structural validity of the scale. PANAS-C was used to determine the criterion-related validity of the scale and correlations between dimensions were examined. The reliability of the scale is demonstrated by the Cronbach Alpha internal consistency coefficient, average variance extracted values and composite reliability values.

As mentioned before, state emotions are feelings experienced at a specific time and place, and to attend online classes is a novel state for adolescents. Because of this, the wording of the items on the scale has been chosen to express the experience of online classes, and the emotions were attempted to be evaluated with items that reflect this state in which adolescents undergo their learning. Additionally, the scale contains items intended to facilitate the understanding of academic emotions that are experienced in the online learning environment in general, rather than emotions felt towards particular subjects. The results of this study show that the developed scale may be used as a valid and reliable measuring tool to measure adolescents' emotions in an online learning ‘state’. It is thought that this scale can be used by experts and teachers who wish to determine adolescents’ emotions during a period of online learning and who pay attention to students’ emotions while designing learning environments.

The fundamental reason for school education to continue from a distance is for the ‘learning’ process to be perpetuated. However, the cultivation of learning is dependent on cognitive and emotional processes being considered with the same care. It is known that students exhibit a rich variety of emotions in academic environments (Pekrun et al., 2002a). Therefore, the emotions felt by students in a learning environment can stimulate them in terms of achievement, competition, cooperation, alienation or discouragement. The findings of this study, in turn, show that different emotions manifest in the online environment. In order to avoid creating a process in which students’ emotions are neglected in attempting to perpetuate the learning process, it is highly important to recognize what emotions students feel in order to include them in the learning process in a healthy manner and to protect their emotional well-being. It is hoped that data collected using this scale will satisfy this need.

The subject of this study, the Online Lesson Adolescent Emotions Scale, is a Turkish scale and was developed with data gathered from Turkish student participants. It is important to understand what emotions adolescents experience during online lessons in different cultures, whether due to national conditions or cultural differences. The expression of emotion may differ from culture to culture (Matsumoto & Kupperbusch, 2001). Individuals form different definitions for themselves and others in a social context in Eastern and Western cultures, and this perspective creates culture-specific forms for psychological processes, such as attention, perception, knowledge, emotion and motivation (Markus & Kitayama, 2010). While the expression of emotions is more widely-accepted in Western cultures, it is preferred that emotions are not spoken about in Eastern cultures.

Indeed, a scale was developed that examined the achievement emotions (enjoyment, anxiety and boredom) of primary school students in Germany and the United States of America, and cultural differences were emphasized throughout its development (Lichtenfeld et al., 2012). On the other hand, the Academic Emotions Scale, developed to understand emotions for mathematics lessons, was applied to eighth grade students in Germany and China where a high amount of similarity was observed (Frenzel et al., 2007). This situation may be the reason that the scale ended up with a four dimensional structure rather than the six dimensional structure that was expected at the beginning. For this reason, conducting similar studies among different cultures and comparison of the results in terms of factors may provide a road map for getting to know adolescents’ emotions in the online learning environment.

In preliminary analyses, no significant difference was observed for scale items between different age groups. For this reason, use of the scale can be stated as valid and reliable for the determined age range. Similarly, preliminary analyses reveal that scale items do not differ for male and female students. Even so, validity and reliability of the scale could be researched for different groups according to age and sex.