Technostress causes cognitive overload in high-stress people: Eye tracking analysis in a virtual kiosk test.

In the midst of the COVID-19 pandemic, the use of non-face-to-face information and communication technology (ICT) such as kiosks has increased. While kiosks are useful overall, those who do not adapt well to these technologies experience technostress. The two most serious technostressors are inclusion and overload issues, which indicate a sense of inferiority due to a perceived inability to use ICT well and a sense of being overwhelmed by too much information, respectively. This study investigated the different effects of hybrid technostress-induced by both inclusion and overload issues-on the cognitive load among low-stress and high-stress people when using kiosks to complete daily life tasks. We developed a 'virtual kiosk test' to evaluate participants' cognitive load with eye tracking features and performance features when ordering burgers, sides, and drinks using the kiosk. Twelve low-stress participants and 13 high-stress participants performed the virtual kiosk test. As a result, regarding eye tracking features, high-stress participants generated a larger number of blinks, a longer scanpath length, a more distracted heatmap, and a more complex gaze plot than low-stress participants. Regarding performance features, high-stress participants took significantly longer to order and made more errors than low-stress participants. A support-vector machine (SVM) using both eye tracking features (i.e., number of blinks, scanpath length) and a performance feature (i.e., time to completion) best differentiated between low-stress and high-stress participants (89% accuracy, 100% sensitivity, 83.3% specificity, 75% precision, 85.7% F1 score). Overall, under technostress, high-stress participants experienced cognitive overload and consequently decreased performance; whereas, low-stress participants felt moderate arousal and improved performance. These varying effects of technostress can be interpreted through the Yerkes-Dodson law. Based on our findings, we proposed an adaptive interface, multimodal interaction, and virtual reality training as three implications for technostress relief in non-face-to-face ICT.

Introduction

The COVID-19 pandemic is rapidly changing our daily life in a non-face-to-face manner using information and communication technology (ICT), such as smartphones, tablets, and kiosks. Although non-face-to-face ICT was implemented long before the pandemic, it has been implemented much more rapidly since the COVID-19 pandemic, centered on education (Ali, 2020; Rahiem, 2020), workplace (Rachmawati, Choirunnisa, Pambagyo, Syarafina, & Ghiffari, 2021), and daily life settings (Király et al., 2020; Yang et al., 2020). Stores like restaurants and coffee shops are now actively exploring the use of non-face-to-face ordering kiosks as an eventual replacement of or complement to human employees. A kiosk, a touch screen system with network capabilities, can provide a hygienic and efficient ordering service to the public. The global kiosk market is expected to grow to about $30 billion by 2025, driven by increasing adoption of kiosks (Markets & Markets, 2022).

Although the introduction of the kiosk seems beneficial to the general public, it can put stress on vulnerable populations, such as the elderly, the disabled, and the homeless, who are not familiar with non-face-to-face ICTs (Seifert, Cotten, & Xie, 2021). This phenomenon, known as ‘technostress’, is caused by an inability to adapt or cope with new ICTs in a healthy manner (Weil & Rosen, 1997). The two most problematic technostressors are inclusion and overload issues, referring to a sense of inferiority due to a perceived inability to use ICT well and a sense of being overwhelmed by too much information provided by ICT, respectively (Nimrod, 2018). In particular, hybrid technostress—induced by both inclusion and overload issues—could be exacerbated in the current COVID-19 situation (Nimrod, 2022). Failure to respond well to technostress can negatively impact the user experience (Sharma & Gupta, 2022) and productivity (Rachmawati et al., 2021). Various errors and failures related to the use of kiosks by the elderly have already been reported (Ryou, Kim, Hong, & Heo, 2019). Even ordering a hamburger using a kiosk at a fast food restaurant is a huge challenge for the elderly (Jiajia & Lee, 2021). In order to introduce non-face-to-face ICT services without alienating the vulnerable in the context of COVID-19, it is necessary to properly evaluate the impact of hybrid technostress.

Cognitive load theory (CLT) provides a useful theoretical framework for understanding the impact of ICT-induced stress (i.e., technostress) in terms of cognitive load (Sweller, 1988; 2011). Studies using CLT (Block, Hancock, & Zakay, 2010; Phillips-Wren & Adya, 2020) show that humans have limited working memory capacity, and as a result, information processing is negatively affected when cognitive load exceeds these limits. Plass and Kalyuga (2019) demonstrated that stress acts as an extraneous cognitive load that competes for limited working memory capacity by forcing it to process information that is not related to goals. Based on their findings, they called for more research on the relation of stress and cognitive load. There have been studies on the negative effects of stress on cognitive functions, such as learning (Conway, Dick, Li, Wang, & Chen, 2013), memory (Quaedflieg & Schwabe, 2018), and decision making (Phillips-Wren & Adya, 2020). However, there is little systematic research trying to understand the impact of stress on cognitive load when using non-face-to-face ICTs (e.g., ordering a hamburger using a kiosk), which is an important issue in the current COVID-19 situation (Jiajia & Lee, 2021; Ryou, Kim, Hong, & Heo, 2019). Given the recent rapid adoption of non-face-to-face ICTs, the use of ICTs is no longer an option but rather an expectation. It is therefore important to evaluate the impact of hybrid technostress experienced when using a kiosk in terms of cognitive load.

Recently, La Torre and colleagues (2020) emphasized that some people are more susceptible to technostress associated with ICT use (i.e., high-stress people). For instance, people who are more sensitive to the presence of observers behind them are more likely to be stressed (Bong, Fraser, & Oriot, 2016). Similarly, people who fear being judged or ridiculed by others for not using kiosks skillfully are more likely to be stressed than those who do not (Ha & Kim, 2021). What differences exist in the cognitive load experienced by low-stress and high-stress people when using non-face-to-face ICT (i.e., a kiosk system) is still an open question.

Considering the findings in the literature and the areas for further research, our research question seeks to investigate the different effects of hybrid technostress—induced by both inclusion and overload issues—on the cognitive load among low-stress and high-stress people when using a kiosk system in daily life. To this end, we utilized virtual reality (VR) technology that can realistically simulate a kiosk-related stressful situation (Bong, Fraser, & Oriot, 2016; Ryu & Seo, 2021). VR enables immersive simulation of post-traumatic stress disorder situations (Kothgassner et al., 2019), phobias (Maples-Keller, Yasinski, Manjin, & Rothbaum, 2017), and stressful work environments (Naylor, Ridout, & Campbell, 2020). We developed a ‘virtual kiosk test’ that simulates a stressful situation associated with kiosk use (e.g., when ordering a hamburger using a kiosk at a fast food restaurant, the people waiting behind the user complain that the order is slow). We quantified cognitive load in this stressful situation by analyzing eye tracking data collected during a virtual kiosk test (Seo, Kim, Oh, Ryu, & Choi, 2017; Zhang et al., 2017).

The study offers several unique contributions. First, in terms of methodology, we developed a virtual kiosk test that can be used to facilitate further studies that quantitatively measure cognitive load for different non-face-to-face ICT usage scenarios. Second, this study showed that in a stressful situation, high-stress people experienced cognitive overload and consequently decreased performance; whereas, low-stress people perceived it as an appropriate cognitive load and improved their performance. Last, by linking our anecdotal findings with recent studies, here we propose three implications for technostress relief in non-face-to-face ICT usage scenarios: (1) an adaptive interface that adapts layouts and elements to the needs of the user or context, (2) multimodal interaction that enables users to interact both visually and verbally, and (3) VR training to help users become accustomed to and better tolerate stressful situations in a VR environment.

This paper is organized as follows. Section 2 provides the theoretical framework and background of this research by describing the impact of technostress on cognitive load and related eye tracking analysis techniques. Section 3 describes the materials and methods involved in this study, including the participants, the virtual kiosk test used for data collection, the experimental procedures, and the statistical analyses performed. Section 4 presents key findings on the effects of technostress on cognitive load. Finally, Section 5 provides an overview of the study's conclusions, practical implications for technostress relief, limitations, and future research.

Related work

In this section, we describe the conceptual foundations and derive the hypothesis for this study. First, we explained the theoretical concept and factors that induce technostress, and introduced how technostress negatively affects cognitive load and related performance. In particular, we focused on how the cognitive load differs between low-stress and high-stress people in stressful situations. We then reviewed the latest research on eye tracking technology to quantitatively and qualitatively assess cognitive load in virtual environments and suggested key features.

The impact of technostress on cognitive load

Technostress is defined as “stress created by ICT use,” which is “one of the fallouts of an individual's attempts and struggles to deal with constantly evolving ICTs and the changing cognitive and social requirements related to their use” (p. 303; Tarafdar, Tu, Ragu-Nathan, & Ragu-Nathan, 2007). The two constructs most often confused with technostress are technophobia (e.g., computer anxiety; Brooks, 2015) and technoaddiction (e.g., problematic social media, internet, or smartphone use; Fu, Li, Liu, Pirkkalainen, & Salo, 2020; Elhai, Levine, Dvorak, & Hall, 2016). While technophobia refers to people's worry or fear about the impact of technology on society (Osiceanu, 2015), technostress refers to the problem of adaptation experienced by individuals when they cannot cope with or adapt to ICT (Weil & Rosen, 1997). Technoaddiction refers to problematic experiences, such as cognitive preoccupation with technology, compulsive use, and consequent neglect of daily activities and essential requirements (Lee, Chang, Lin, & Cheng, 2014); whereas, technostress is an emotional reaction that ICT users can commonly experience (Ayyagari, Grover, & Purvis, 2011). In sum, in this study, technostress refers to a stressful experience that anyone can directly or indirectly receive due to the use of ICT in daily life (Lee, Son, & Kim, 2016).

Previous research on technostress has focused on the difficulties experienced while using ICT in the workplace, so the main research subjects were academics (Jena, 2015), government officers (Fuglseth & Sørebø, 2014), communication professionals (Bucher, Fieseler, & Suphan, 2013), teachers (Al-Fudail & Mellar, 2008), and general workers (Ayyagari, Grover, & Purvis, 2011). Recently, as the use of non-face-to-face ICT has increased in daily life due to the COVID-19 outbreak, research on the effect of technostress in daily life is attracting attention. For example, Jiajia and Lee (2021) examined technostress caused by the use of kiosks in the daily life of the elderly and emphasized the need for an inclusive design approach to reduce technostress-related factors. Nimrod (2020) investigated the experiences of elderly internet users during the COVID-19 pandemic and concluded that research is needed to understand and alleviate technostress in daily life. Overall, both studies called for further research into the key factors that induce technostress in daily life and how to reduce it.

Recent studies explored ICT-related factors, events, and circumstances that lead to technostress (i.e., technostressors; Ayyagari, Grover, & Purvis, 2011; Ragu-Nathan, Tarafdar, Ragu-Nathan, & Tu, 2008). Among others, Nimrod (2018) developed a useful lens for interpreting five factors that induce technostress (see Table 1 ). These five factors are consistent with the major ICT-related stress-generating factors already identified in the previous literature (D'Arcy, Herath, & Shoss, 2014; Maier, Laumer, Weinert, & Weitzel, 2015).

Table 1

Five factors that induce technostress, adapted from Nimrod (2018).

Factor that induces technostress	Description
Overload	Too much information provided by ICT overwhelms people and consequently makes them perform tasks more slowly
Invasion	The application of ICT to personal contexts in daily life makes people feel uncomfortable as if they are being imposed upon
Complexity	The ever-changing nature of ICT makes it complicated and difficult for people to learn, use, and master
Privacy	The possibility that ICT use can be traced, documented, and exploited by others makes people feel threatened with the potential overstepping of their personal boundary
Inclusion	The pressure to be able to use ICT like everyone else makes people feel inferior and stressed

Nimrod (2022) further demonstrated that in the current COVID-19 situation, people are most stressed by inclusion (e.g., a sense of inferiority due to a perceived inability to use ICT well compared to other users) and overload issues (e.g., a sense of being overwhelmed by too much information provided by ICT) compared to other technostressors (i.e., invasion, complexity, and privacy). For instance, people experience technostress when they feel that they are not as well-adjusted cognitively and socially as others in the ever-changing ICT environment (Friemel, 2016). Technostress caused by inclusion issues can be exacerbated by the presence of observers (Bong, Fraser, & Oriot, 2016). To illustrate how this works in a specific case, this concept can explain elderly people's stress when ordering a hamburger using a kiosk as stemming from observers behind them (Jiajia & Lee, 2021). As the digital divide (i.e., the gap between those who benefit from the ICT and those who do not) deepens due to the COVID-19 crisis, such inclusion issues are expected to occur more frequently (Lai & Widmar, 2021; Mehra, Sikes, & Singh, 2020). In addition, excessive information provided by a kiosk (e.g., menus, buttons, and complex layouts) creates overload issues for ICT users. Although other technostressors (i.e., invasion, complexity, and privacy) can be addressed with technical modifications, hybrid technostress caused by both inclusion and overload issues should be considered from the perspectives of information processing and management. Therefore, in this study, we focused on a stressful situation associated with both inclusion and overload issues (i.e., observers in the back complaining while a user orders a hamburger at a fast food restaurant using a kiosk with excessive information) and developed this stressful situation as a virtual kiosk test (see Section 3.2).

Technostress exceeding a certain intensity negatively affects cognitive load and related performance outcomes (Conway, Dick, Li, Wang, & Chen, 2013; Phillips-Wren & Adya, 2020; Quaedflieg & Schwabe, 2018). Interestingly, the effects of technostress on cognitive load can differ depending on the individual's sensitivity to factors that induce technostress (see Table 1; La Torre, De Leonardis, & Chiappetta, 2020). Nimrod (2018) developed a questionnaire to differentiate between low-stress and high-stress people based on how sensitive they are to technostress. High-stress people experience technostress more easily in the process of using ICT, and the effect of stress is negative. Under the same stressful situation, compared to low-stress people, high-stress people consume more cognitive resources (Bong, Fraser, & Oriot, 2016) and, therefore, make more cognitive errors (Baradell & Klein, 1993) and risky decisions (Wickens, 2014), both of which negatively impact performance and productivity. For instance, high-stress people are more easily distracted by stimuli that are not related to their goals (Kelley & Lavie, 2011). The difference in the way low-stress and high-stress people experience cognitive load in stressful situations can be theoretically explained by the Yerkes-Dodson law. The Yerkes-Dodson law shows that low-stress people feel moderate arousal in stressful situations and their performance improves; whereas, high-stress people feel cognitive overload and perform much worse in stressful situations (Yerkes & Dodson, 1908). Applying the Yerkes-Dodson law to the results of kiosk use will help us understand the varying effect of technostress on cognitive load. To this end, we recruited low-stress participants and high-stress participants and investigated how hybrid technostress had different effects on cognitive load and performance in each group of participants.

Eye tracking analysis

Eye tracking data collected while performing tasks can reliably and non-invasively measure the cognitive load currently being experienced by the subject (Wang, Yang, Liu, Cao, & Ma, 2014). Representative eye tracking features for evaluating cognitive load include involuntary data (e.g., pupil dilation, number of blinks) and voluntary data (e.g., fixation, saccade, and scanpath length). Among involuntary data types, pupil dilation, which indicates the size of the pupil diameter while performing a task, is a measure of mental effort (Chen, Epps, Ruiz, & Chen, 2011). Large pupil dilation indicates that the subject is experiencing a higher cognitive load due to higher task difficulty (Siegle, Ichikawa, & Steinhauer, 2008). However, pupil dilation has a disadvantage in that there is a large deviation due to changes in gaze angle and illuminance (Joseph & Murugesh, 2020). Another involuntary data type, number of blinks, is an involuntary movement of the eyelids covering the pupil and cornea that occurs before and after experiencing a high cognitive load (Perkhofer & Lehner, 2019). A large number of blinks indicates that the subject is experiencing cognitive overload and stress (Jyotsna & Amudha, 2018). Unlike the pupil dilation, the number of blinks is not affected by external factors, such as gaze angle and illuminance (Perkhofer & Lehner, 2019). For this reason, in this study, we used the number of blinks as an involuntary data type of eye tracking when evaluating the cognitive load induced by hybrid technostress.

For voluntary eye movement data, fixations and saccades are the most representative measures used to evaluate cognitive load during information processing (Chen, Epps, Ruiz, & Chen, 2011). A fixation is the act of stopping a scene scan and focusing on a specific area of interest for 200 ms or longer (e.g., looking at a specific menu item in a kiosk for more than 200 ms). Fixation count and duration are commonly used to evaluate cognitive load (Wang, Yang, Liu, Cao, & Ma, 2014) and working memory (Zagermann, Pfeil, & Reiterer, 2016). High fixation counts and durations indicate how much attention the user had to work with (Chen, S., Epps, J., Ruiz, N., & Chen, F., 2011) and how much cognitive load increased as a result (Lin & Lin, 2014). A saccade is the act of moving quickly between two fixations in 30-80 ms (e.g., looking at two menu items in a kiosk alternately at high speed). Saccade velocity and length are commonly used to measure cognitive load associated with the difficulty of a task (Barrios et al., 2004). High saccade velocity and long saccade length indicate high task difficulty and consequently increased cognitive load (Zagermann, Pfeil, & Reiterer, 2016).

Recently, instead of measuring fixations and saccades individually, studies focus on the scanpath length to allow for a comprehensive evaluation of consecutive fixations and saccades outcomes (Zagermann, Pfeil, & Reiterer, 2016). The scanpath length is calculated as the sum of the lengths of eye movements that occurred while performing the task (Goldberg & Kotval, 1999). A longer scanpath length indicates that the task is difficult, which increases the cognitive load and results in less efficient information searching (Joseph & Murugesh, 2020). Therefore, in this study, we used scanpath length as voluntary data of eye tracking when evaluating the cognitive load induced by hybrid technostress.

Additionally, scanpath visualization techniques, such as a heatmap and a gaze plot, help to qualitatively analyze information processing patterns among subjects (Goldberg & Helfman, 2010). A heatmap that colorizes the degree of accumulation of fixation provides insights about the subject's visual attention patterns displayed while performing a task (Wang et al., 2014). A heatmap gives a gist of the areas that people paid the most visual attention to and the areas they paid no attention to when performing a task (Cai, Sharma, Chatelain, & Noble, 2018). In the case of a gaze plot, it provides a visual representation of the continuous sequence of fixations and saccades that occurred as a task was performed (Joseph & Murugesh, 2020). The gaze plot represents the subject's fixation as a circle, the fixation duration as a circle area, and the saccade between fixations as a straight line. In this study, we qualitatively analyzed the scanpath by visualizing them as both a heatmap and a gaze plot to assess whether participants efficiently and effectively process information in a stressful situation (e.g., whether the participant is processing non-task-related information or not).

Overall, the purpose of the present study was to evaluate how the effects of hybrid technostress on cognitive load differ between low-stress and high-stress people in a stressful situation. Based on eye tracking analysis, we hypothesized that high-stress people, given their higher cognitive load in a stressful situation, should generate a larger number of blinks, a longer scanpath length, a more distracted heatmap, and a more complex gaze plot compared to low-stress people.

Materials and methods

Participants

We recruited 25 participants from the university community. All of these participants were university students. They consisted of students with nine different majors: computer science (n = 8), applied artificial intelligence (n = 7), electrical and information engineering (n = 4), architectural engineering (n = 1), food engineering (n = 1), dance (n = 1), bibliology (n = 1), nursing (n = 1), and fashion (n = 1). To sort the participants into either the low-stress or high-stress participant groups, we assessed their sensitivity to hybrid technostress through qualitative interviews before undertaking the virtual kiosk test (see Appendix A, for details). We used the qualitative interview method because: 1) it is useful for understanding the progress of technostress (Tarafdar, Cooper, & Stich, 2019), and 2) it is effective for understanding technostress as experienced by participants in a holistic context (Pflügner, Baumann, & Maier, 2021). As a result of the qualitative interviews, the participants were divided into two groups consisting of 12 low-stress participants (6 males and 6 females; average age of 23.167 ± 2.478 years) and 13 high-stress participants (7 males and 6 females; average age of 22.769 ± 1.967 years). We hypothesized that high-stress participants (i.e., people more vulnerable to hybrid technostress associated with ICT use), in contrast to low-stress participants, would experience higher cognitive load under a stressful situation associated with kiosk use (e.g., while the user orders a hamburger with a kiosk at a fast food restaurant, the people waiting behind the user complain that the order is slow). Written informed consent was obtained from each participant after the experimental procedure was explained. This study was approved by the Institutional Review Board according to the Declaration of Helsinki (HYUH-2021-08-020-004).

Virtual kiosk test

The virtual kiosk test is an immersive VR-based test that measures cognitive load (through eye tracking analysis) and performance when ordering burgers, sides, and drinks using the kiosk in a virtual environment. The experimental space for the virtual kiosk test consists of dimensions of 1.3 m x 1.3 m x 2 m (see Fig. 1 ). During the experiment, participants wear a head-mounted display (HTC VIVE Pro Eye) for an immersive VR experience and hold a hand controller in their dominant hand for selecting and ordering menus from a kiosk in the virtual environment. Two base stations tracked the movement of the head-mounted display and hand controller. A high-performance desktop (Intel i7-10700, NVIDIA GeForce RTX 3070, 32GB RAM) was used to deliver the VR experience.

Fig. 1

Virtual kiosk test setup. (A) The experimental space and hardware for the virtual kiosk test. (B) Virtual kiosk test screen that participants see through a head-mounted display. A participant can use her/his virtual hand to select and order menus from a kiosk. The test starts when the participant clicks the ‘Start’ button with her/his virtual hand.

The virtual kiosk test had six sequential action steps, as shown in Fig. 2 : (Step 1) select either to eat at the restaurant or take out, (Step 2) select the burger out of eight menu items, (Step 3) select the side out of eight menu items, (Step 4) select the drink out of eight menu items, (Step 5) select either to pay with cash or credit card, and (Step 6) enter the four-digit credit card password and then complete the order. Prior to completing the order, participants could go back if they want to change their menu selections. Before the experiment, all participants were given the following six goals: “The place to eat is a restaurant (Goal 1). Please order a shrimp burger (Goal 2), a cheese stick (Goal 3), and Coca-Cola (Goal 4) using the kiosk. Please use a credit card as the payment method (Goal 5). The card payment password is 6289 (Goal 6).” Note that the human working memory can remember approximately three to five meaningful goals simultaneously (Cowan, 2010). Because the six goals in the virtual kiosk test exceed the human working memory capacity, it was expected that the test could assess the cognitive load experienced by the participants. If the participants fail to remember their goals during the virtual kiosk test, this could indicate the overloading of working memory due to hybrid technostress. Even if the participants perfectly remember their goals, the eye tracking analysis can quantify their cognitive loads.

Fig. 2

Six sequential action steps in the virtual kiosk test between ‘Start’ and ‘End’ screens. In Step 1, the participants decide either to eat at the restaurant or to take out. In Step 2, participants select a burger. In Step 3, participants select a side. In Step 4, participants select a drink. In Step 5, participants select either to pay with cash or a credit card. In Step 6, participants enter the four-digit credit card password and then complete the order.

During the virtual kiosk test, the participants’ raw eye tracking data (i.e., eye blink data and gaze coordinates) were collected every 100 ms. The collected raw eye tracking data were used to calculate two eye tracking features: the number of blinks (i.e., total number of blinks while conducting the test) and the scanpath length (i.e., the sum of the Euclidean distances between each gaze coordinate while performing the task). Based on the virtual kiosk test results, two performance features were calculated: the time to completion (i.e., the time taken to complete the entire test) and the number of errors (i.e., total number of incorrect dining location selections, incorrect burger/side/drink menu selections, incorrect payment method selections, incorrect credit card password entries). In a nutshell, the virtual kiosk test derived the following four features for assessing the participants’ cognitive load: (1) the number of blinks, (2) the scanpath length, (3) the time to completion, and (4) the number of errors.

To investigate the effect of hybrid technostress on cognitive load, a virtual kiosk test was performed in a stressful situation designed based on both inclusion and overload issues (see Figs. 2 and 3 ). In this stressful situation, two virtual avatars stand behind the participant. When the participant starts selecting the burger out of eight menu items (i.e., Step 2 in Fig. 2), the virtual avatars A and B start complaining together that the participant is taking too much time to order using the kiosk (e.g., when virtual avatar A asks “Why does the person in front of me take so long to order?”, virtual avatar B replies “Yeah, the person in front doesn't seem to be able to use the kiosk as well as we do.”). These complaints were repeated in various versions until the participant entered her/his credit card password and completed the order (i.e., Step 6 in Fig. 2). This was intended to make the participants feel psychological and temporal pressure and consequently to experience hybrid technostress.

Fig. 3

A stressful situation in the virtual kiosk test. While the participant uses a hand controller to interact with the virtual kiosk, virtual avatars A and B behind the participant verbally complain together that the participant is taking too much time to order using the kiosk.

To confirm that the virtual kiosk test generates a sufficient amount of cognitive load, we conducted a pre-test with five participants using the NASA-TLX five-point Likert scale (Mauro, Ardissono, & Lucenteforte, 2020). Because the virtual kiosk test did not require physical effort, we only assessed mental demand, temporal demand, performance, effort, and frustration. All participants practiced for at least ten minutes to become familiar with the test. The pre-test results showed that while practice reduced the cognitive load associated with performance (3.4 ± 0.5) and effort (3.8 ± 0.4), participants still experienced high levels of temporal demand (4.4 ± 0.8), mental demand (4.0 ± 0.0), and frustration (4.0 ± 0.6) due to complaints from virtual avatars.

Experimental procedures

All participants practiced for at least 10 min to become familiar with interacting with the virtual kiosk using a hand controller while wearing a head-mounted display in a virtual environment. Additional practice time was provided if the participant wanted it. No participants required additional practice time. After the practice, participants performed the virtual kiosk test in the aforementioned stressful situation (25.728 ± 11.508 s). To eliminate racial bias due to the virtual avatar's appearance, the participants were instructed not to look back during the virtual kiosk test. This instruction did not feel awkward to the participants in the light of social norms, as people usually do not look back when others behind them are talking among themselves. After completing the experiment, all participants reported that they felt comfortable with this instruction. Participants were able to stop the experiment at any time if they felt cybersickness while conducting the virtual kiosk test. None of the participants stopped the experiment due to cybersickness. It should be noted that the entire experiment was conducted in Korean.

Statistical analyses

To evaluate how the eye tracking features and performance features differed between low-stress and high-stress participants in the virtual kiosk test, we applied several statistical methods using Python. First, using a chi-square test, independent samples t-tests, and a Pearson correlation analysis, we compared the demographic characteristics between the low-stress and high-stress participants. Second, using independent samples t-tests, we examined how the eye tracking features (i.e., number of blinks, scanpath length) and performance features (i.e., time to completion, number of errors) in the virtual kiosk test differed between low-stress and high-stress participants. For all t-tests, their effect sizes were reported by Cohen's d. Third, using a Pearson correlation analysis, we investigated the relationship between eye tracking features and performance features in the virtual kiosk test. Fourth, using heatmaps and gaze plots, we qualitatively analyzed eye tracking data to compare information processing patterns between low-stress and high-stress participants. Finally, we utilized a support-vector machine (SVM) to evaluate how well we could differentiate between low-stress and high-stress participants using two eye tracking features (i.e., number of blinks, scanpath length) and a performance feature (i.e., time to completion) collected from the virtual kiosk test.

Results

Basic demographic characteristics

A chi-square test revealed no significant effect of gender on each participant group, χ2(1) = 0.037, p = 0.848. Differences in demographic characteristics between low-stress and high-stress participants were further analyzed using t-tests (see Table 2 ). It was found that there were no statistical differences in age (p = 0.841) or education level (p = 0.288) between the low-stress and high-stress participants. Note that the education level was calculated as the sum of years of schooling since starting elementary school. A Pearson correlation analysis showed no relationship between basic demographic characteristics and the virtual kiosk test results (i.e., eye tracking features and performance features).

Table 2

Basic demographic characteristics of low-stress and high-stress participants.

	Low-stress participants	High-stress participants	P value
Number of participants (male)	12 (6)	13 (7)	0.848
Age	22.667 ± 2.060	23.231 ± 2.522	0.841
Education level	12.333 ± 1.155	12.0 ± 0.0	0.288

Values for age and education level are expressed as the mean±SD.

Eye tracking features and performance features in the virtual kiosk test

Differences in eye tracking features and performance features between low-stress and high-stress participants in the virtual kiosk test were analyzed using t-tests (see Table 3 ). For eye tracking features, in a stressful situation, high-stress participants showed a larger number of blinks (effect size d = 0.903) and a longer scanpath length (effect size d = 0.904) while conducting the virtual kiosk test than the low-stress participants. For performance features, in a stressful situation, high-stress participants took significantly longer time to complete the virtual kiosk test than the low-stress participants (effect size d = 0.434). High-stress participants made more errors in the virtual kiosk test than the low-stress participants, but this was not statistically significant.

Table 3

Eye tracking features (number of blinks, scanpath length) and performance features (time to completion, the number of errors) from the virtual kiosk test.

	Low-stress participants	High-stress participants	P value	Effect size
Eye tracking features
Number of blinks	9.667 ± 4.661	16.846 ± 10.227	0.043	0.903
Scanpath length (pixels)	10.55 ± 3.745	14.799 ± 5.491	0.042	0.904
Performance features
Time to completion (seconds)	20.59 ± 4.952	30.469 ± 13.609	0.032	0.434
The number of errors	0.083 ± 0.276	0.462 ± 1.082	0.271	-

Values are expressed as mean±SD. Effect sizes are reported by Cohen's d.

Correlation between eye tracking features and performance features in the virtual kiosk test

A Pearson correlation analysis was performed to examine the relationship between eye tracking features (i.e., number of blinks, scanpath length) and performance features (i.e., time to completion, number of errors) in the virtual kiosk test results. All eye tracking features were correlated with performance features. The number of blinks was correlated with the time to completion (r = 0.444, p = 0.026) and the number of errors (r = 0.419, p = 0.037). The scanpath length was correlated with the time to completion (r = 0.872, p < 0.001) and the number of errors (r = 0.637, p = 0.001). These findings suggest that cognitive load (measured through eye tracking features) and performance features influence each other.

Heatmaps and gaze plots in the virtual kiosk test

Heatmaps and gaze plots show different information processing patterns between low-stress and high-stress participants in a stressful situation. As shown in Fig. 4 , in the heatmap, it was found that low-stress participants only focused their attention on goal-related information (i.e., only focused on target burger, side, drink, and password numbers), while high-stress participants displayed split-attention, such as being distracted by information that was not related to goals. For example, in Steps 2, 3, and 4, high-stress participants were distracted by extraneous menu items. Also, in Step 6, high-stress participants paid attention to other buttons besides the password number.

Fig. 4

Heatmaps of low-stress participants (top) and high-stress participants (bottom) based on eye tracking data in the virtual kiosk test. The colored areas indicate the degree of cumulative fixation time. Red areas indicate the participant's high interest. Blue areas indicate the participant's low interest.

As shown in Fig. 5 , in the gaze plot, low-stress participants showed effective and efficient gaze movements (i.e., using the kiosk with minimal gaze movements while only looking at information relevant to the goal); whereas, high-stress participants displayed complex and inefficient gaze movements (i.e., using a kiosk with unnecessary repetitive eye movements while distracted by information that is not relevant to the goal). For instance, in Step 3, high-stress participants wander through the wrong menu items until they find the right one. In Step 6, high-stress participants showed a distracting gaze pattern, such as repeatedly looking at unnecessary buttons other than the password number.

Fig. 5

Gaze plots of low-stress participants (top) and high-stress participants (bottom) based on eye tracking data in the virtual kiosk test. The red and purple circles represent the first and last fixation points, respectively. The blue circles with numbers indicate the fixation points and their order. The size of the circle is proportional to the fixation duration. The line connecting the circles represents the saccade.

Discriminative performance of the virtual kiosk test for identifying high-stress people

The results of t-tests showed that two eye tracking features (i.e., number of blinks, scanpath length) and a performance feature (i.e., time to completion) were able to differentiate between low-stress and high-stress participants. The number of errors was not considered as an input feature because it did not differentiate between low-stress and high-stress participants (p = 0.288). In this study, a support-vector machine (SVM) was used because it performed significantly better than the other three classification algorithms, in this case logistic regression, decision tree, and k-nearest neighbors (see Appendix B for details). SVM with a leave-one-out cross-validation procedure was applied to investigate whether eye tracking data and performance data measured through the virtual kiosk test could be used to discriminate high-stress participants. Note that 70% of the data were used for training and the rest of the data were used for validation. As shown in Table 4 , the high-stress participants were best discriminated from the low-stress participants when both eye tracking features (i.e., number of blinks, scanpath length) and a performance feature (i.e., time to completion) were used (89.0% accuracy, 100.0% sensitivity, 83.3% specificity, 75.0% precision, and 85.7% F1 score). The use of only eye tracking features or only a performance feature was able to classify a fraction of the high-stress participants. In effect, the inclusion of both eye tracking features and a performance feature from the virtual kiosk test provides additional discriminative power for the early identification of high-stress participants.

Table 4

Support-vector machine for classifying the low-stress and high-stress participants.

Input features	Accuracy (%)	Sensitivity (%)	Specificity (%)	Precision (%)	F1 score (%)
Eye tracking features (number of blinks, scanpath length)	78.0	100.0	33.3	60.0	75.0
Performance feature (time to completion)	78.0	100.0	33.3	60.0	75.0
Performance feature (time to completion) + Eye tracking features (number of blinks, scanpath length)	89.0	100.0	83.3	75.0	85.7

Discussion

The main objective of this study was to investigate the different effects of hybrid technostress—induced by both inclusion and overload issues—on cognitive load and performance among low-stress and high-stress people when using non-face-to-face ICT in daily life (inter alia, ordering a hamburger using a kiosk). We developed a virtual kiosk test to evaluate participants’ cognitive load (through eye tracking analysis) and performance when using the kiosk in a stressful situation. Consistent with our hypothesis, in a stressful kiosk use scenario, high-stress participants showed higher cognitive load than low-stress participants (i.e., generated a larger number of blinks, a longer scanpath length, a more distracted heatmap, and a more complex gaze plot). Also, high-stress participants performed worse in kiosk use than low-stress participants (i.e., took significantly longer to order and made more errors). Correlation analysis results confirmed this negative relationship between cognitive load and performance (i.e., the higher the cognitive load, the worse the performance, and vice versa). Additionally, machine learning with SVM recognized high-stress participants in a stressful situation by using both eye tracking features and a performance feature. Overall, our results demonstrated that technostress had a negative effect on cognitive load and performance when using non-face-to-face ICT, and that these effects differed between low-stress and high-stress participants. We are not aware of other studies evaluating technostress using eye tracking data, performance data, and machine learning in a non-face-to-face ICT context like this one.

Theoretical contributions

The theoretical contribution provided here is that the varying effects of hybrid technostress on low-stress and high-stress participants can be explained by the Yerkes-Dodson law (Seo, Fels, Kang, Jung, & Ryu, 2021; Yerkes & Dodson, 1908). As shown in Fig. 6 , the current study found that low-stress participants seemed to perceive an acceptable level of cognitive load in a stressful situation (i.e., a shorter scanpath length), resulting in improved performance (i.e., took much less time to complete the order); whereas, high-stress participants seemed to perceive cognitive overload when facing technostress (i.e., a longer scanpath length), resulting in decreased performance (i.e., took much longer to complete the order). This is in line with the Yerkes-Dodson law, which states that psychological arousal (i.e., perceived stress) increases performance to some extent, but decreases performance if arousal is too high. It might be said that an appropriate level of technostress can be a positive motivator to use and learn non-face-to-face ICT, but excessive technostress can actually hinder and worsen adaptation to new technologies (e.g., a digital divide; Lai & Widmar, 2021; Mehra, Sikes, & Singh, 2020). This may be the first evidence (to our knowledge) to explain the relationship between technostress and performance in the context of non-face-to-face ICT use in daily life using the Yerkes-Dodson law.

Fig. 6

The Yerkes–Dodson graph on the time to completion (performance) and scanpath length (cognitive load).

Analysis of heatmaps and gaze plots of eye tracking data from the virtual kiosk test reveals different information processing patterns between low-stress and high-stress participants (see Figs. 4 and 5). Low-stress participants showed focused attention, looking only at information relevant to the goal and ignoring irrelevant information (i.e., generated fixations only for target menu items and correct password numbers, and generated as few saccades as possible between these fixations only). This focused attention may have been possible because low-stress participants feel less extraneous cognitive load in a stressful situation and thus preserve their working memory capacity. In contrast, high-stress participants exhibited split-attention in which they were easily distracted by information unrelated to their goals (i.e., generated fixations for extraneous menu items and wrong password numbers, and generated unnecessary saccades repetitively between extraneous fixations). This split-attention may have occurred because hybrid technostress acted as an extraneous cognitive load on high-stress participants and depleted their working memory capacity (e.g., Phillips-Wren & Adya, 2020, Quaedflieg & Schwabe, 2018). In conclusion, working memory capacity depleted by hybrid technostress may have caused high-stress participants to be unable to focus on goal-related information and to be distracted by irrelevant information.

Taken together, our study confirmed the negative impact of hybrid technostress on cognitive load in high-stress participants. Hybrid technostress causes cognitive overload, making high-stress participants unable to focus on goal-related information and rather distracted by extraneous stimuli, which in turn leads to many errors and difficulties in using the latest non-face-to-face ICT. In order to introduce non-face-to-face ICT services without alienating the vulnerable in the COVID-19 situation, it is essential to identify and prevent points where high-stress people are negatively affected by hybrid technostress (Seifert, Cotten, & Xie, 2021). Determining how to achieve this is the key to non-face-to-face ICT design with less technostress.

Practical contributions

This study provides three practical contributions that may help alleviate hybrid technostress in non-face-to-face ICT usage scenarios. First, we found that the sudden display of a large number of menu items in a kiosk triggers high-stress participants to lose focus and become distracted (see Step 3 in Figs. 4 and 5). High-stress participants experience cognitive overload due to too many menu items and consequently fail to adapt to kiosk use. An ‘adaptive interface’, which adapts its layouts and elements to the needs of the user or context (Miraz, Ali, & Excell, 2021), can be a solution to this finding. By implementing an adaptive interface, users do not need to struggle to adapt to new ICT, but rather, the technology can adapt its layouts and elements to the contexts of the user (Seo, Dodson et al., 2021). For example, in a kiosk usage scenario, if it is determined that the user is experiencing cognitive overload due to a large number of menu items, the adaptive interface can alleviate the user's cognitive load by changing the menu structure to a simpler form or changing the layout to be concise (Ha & Kim, 2021; Hong & Choe, 2019). This adaptive interface can serve as safeguards to help users experience an appropriate level of cognitive load while interacting with non-face-to-face ICT.

Second, we found that participants spoke out the menu item they were looking for when they were visually confused by the complex menu layout. In the virtual kiosk test, when participants couldn't find their target out of eight menu items in the kiosk, they repeatedly mumbled their goals, such as “Where's the shrimp burger?” Participants explained their mumbling as a strategy to better remember goals when faced with cognitive overload. This anecdotal finding is in line with previous studies that found that vocalizing the goal helps participants reduce their cognitive load by externalizing thoughts and re-engaging sensory encoding and storage buffers (Baddeley, 2007; Diaz, Winsler, Atencio, & Harbers, 1992). These participants’ mumblings can be further leveraged to help participants find the target menu easily. For instance, by highlighting a menu button that a participant is mumbling, we can help the participant quickly find that button without further distractions. Indeed, conversation is the natural mode for information exchange in daily life, and voice interactions for search input and output can make information search convenient (Trippas et al., 2020). This ‘multimodal interaction’, which supports interaction using both visual and voice modalities (Wilson et al., 2019), can help reduce the user's cognitive load in a hybrid technostress situation.

Third, we uncovered the potential of ‘VR training’ to mitigate the negative effects of hybrid technostress on cognitive overload by utilizing a virtual kiosk test. After the experiment, participants reported that greater use of VR may help them use the kiosk well under other stressful situations. In a similar vein, recent studies showed that VR training helps users become accustomed to and better tolerate stressful stimuli by exposing them to stressful situations, such as post-traumatic stress disorder situations (Kothgassner et al., 2019), phobias (Maples-Keller, Yasinski, Manjin, & Rothbaum, 2017) and stressful work environments (Naylor, Ridout, & Campbell, 2020). Above all, VR training offers the advantage of the recipient being safely and repeatedly exposed to stressful events that are difficult to experience in daily life (Andersen, Konge, & Sørensen, 2018; Chae, Lee, Jung, & Ahn, 2018). In our case, we can help users overcome hybrid technostress by repeatedly exposing them to the hybrid technostress-related issues in their daily life. Recent research shows that training can relieve the stress associated with using technology (Yoon & Ha, 2021). Such training can change high-stress people, who experience cognitive overload in a stressful situation, into low-stress people, whose performance is rather improved.

Limitations and future studies

This study has several limitations. Although we compared the impact of hybrid technostress on cognitive load between low-stress and high-stress participants, further studies are needed to determine what differences occur in subjects with different characteristics. Specifically, we need to examine the effects of hybrid technostress on ICT vulnerable populations, such as the elderly, the disabled, and the homeless (Seifert, Cotten, & Xie, 2021). Note that we were unable to conduct experiments on vulnerable populations for safety reasons in the COVID-19 situation. Amid the growing digital divide, understanding the impact of hybrid technostress on the vulnerable populations and finding solutions to them will be important for inclusive design. In addition, future studies examining other antecedents of hybrid technostress, such as individual differences, context of daily life, and technology-related perceptions, will help to understand the multifaceted mechanism of technostress.

In the current study, we designed a stressful situation with the inclusion and overload issues as the core. Though both inclusion and overload issues are the most critical factors which induce hybrid technostress in the current COVID-19 context, other factors, such as privacy and complexity (see Table 1), are also important. Moreover, these factors might induce technostress in a different manner compared to the inclusion and overload issues. Future research should design scenarios based on more diverse factors that induce more technostress and evaluate how each factor has a different effect on cognitive load. This will help to comprehensively understand and evaluate various forms of technostress that can occur while using ICT in daily life.

In this study, we evaluated the effect of hybrid technostress on cognitive load using eye tracking data from a virtual kiosk test. If eye tracking data allowed us to understand the behavioral details of subjects’ perceived cognitive load, future studies should further measure physiological and neurological mechanisms to see the full picture of technostress. Physiological data, such as heart rate and galvanic skin response, can quantitatively measure the response of our sympathetic and parasympathetic nervous systems to stress (Riedl, Davis, & Hevner, 2014). Electroencephalogram (EEG), which measures voltage fluctuations resulting from ionic current within the neurons of the brain, can be used to evaluate stress and cognitive overload (Hou et al., 2015). In particular, thanks to the latest technological advances, it is possible to simultaneously measure physiological data and brain waves while performing tasks in virtual reality. Overall, in order to understand technostress from various aspects, it is necessary to study measurement experiments that include not only eye tracking-based behavioral data but also physiological data and brain science data.

There are caveats regarding the use of the virtual kiosk test. First, although no participants in our study complained of cybersickness during the test, poorly designed VR content could induce cybersickness, a source of additional technostress (Souchet, Philippe, Lourdeaux, & Leroy, 2022). The three major factors of cybersickness are hardware, content, and human factors (see Chang, Kim, & Yoo, 2020, for a review). Future research using VR should closely monitor participants’ cybersickness to prevent unintentional technostress. Additionally, potential racial bias due to virtual avatars should be fully considered. Note that the virtual avatar's appearance can significantly affect the cognition and behavior of participants (Peck, Good, & Seitz, 2021). During the virtual kiosk test, the participants were instructed not to look to prevent potential racial bias caused by the virtual avatar. If future research studies involve interactions with virtual avatars, potential racial biases should be fully taken into account when designing the VR content.

Conclusion

Overall, this study is the first to examine the impact of hybrid technostress—induced by both inclusion and overload issues—on cognitive load through eye tracking analysis in a virtual kiosk test. Our results found that in a stressful situation (related to the use of non-face-to-face ICT in daily life), high-stress participants experienced decreased performance due to cognitive overload, while low-stress participants improved their performance thanks to appropriate arousal. These varying effects of hybrid technostress on low-stress and high-stress participants can be explained by the Yerkes-Dodson law. Based on the findings here, we proposed three practical implications for alleviating hybrid technostress in non-face-to-face ICT usage scenarios: an adaptive interface, multimodal interaction, and VR training.

CRediT authorship contribution statement

Se Young Kim: Conceptualization, Methodology, Investigation, Data curation, Writing – review & editing, Visualization. Hahyeon Park: Methodology, Investigation, Data curation, Writing – review & editing. Hongbum Kim: Writing – review & editing. Joon Kim: Writing – review & editing. Kyoungwon Seo: Conceptualization, Methodology, Investigation, Writing – original draft, Writing – review & editing, Visualization, Supervision, Project administration, Funding acquisition.

Declaration of Competing Interest

The authors declare that they have no competing interests.

Table A1

Codebook for classifying participants into either the low-stress participant group or high-stress participant group.

Code	Description	Example response
Low-stress participant group
Better performance	Participants felt confidence when using ICT that they could complete their order more accurately and faster	(P22) “It is not hard to order faster if people are waiting behind me because I am accustomed to using kiosks.”(P25) “I think I can complete the kiosk order faster for people waiting behind me.”
Focused attention	Participants didn't feel any pressure so that they could focus on finding their menu items	(P12) “Even with other people around, I can focus on searching for my menu.”(P17) “I only think about the menu item that I want to order.”
Indifference	Participants felt indifference regarding other people around them	(P5) “Even if other people are around, it doesn't affect me.”(P27) “It doesn't matter if there are other people around when ordering.”
High-stress participant group
Worse performance	Participants felt a lack of confidence when using ICT. It made them worry about the delay in finding the menu item correctly	(P1) “I feel pressured to complete my order quickly because of the people behind me. Perhaps, I will make more mistakes.”(P16) “I feel worried about ordering the menu within a short time. So I am reluctant to change my previous choice.”
Distraction	Participants felt pressured by others, which interrupted them in their efforts to find the menu item they wanted when using ICT	(P2) “When someone is looking at my order, it makes it difficult for me to focus on finding the menu item.”(P3) “I feel difficult to concentrate because of other people. For this reason, I can easily get distracted by things that aren't important to me.”
Negative emotions	Participants felt stressed, reacting excessively and negatively to those around them	(P6) “I get very nervous and stressed when someone looks at me while using the kiosk.”(P21) “It irritates me if someone observes me using the kiosk.”

Table B1

Comparison of the discriminative performance of the classification algorithms.

Algorithms	Accuracy (%)	Sensitivity (%)	Specificity (%)	Precision (%)	F1 score (%)
SVM	89.0	100.0	83.3	75.0	85.7
LR	66.7	60.0	75.0	75.0	66.7
DT	55.6	50.0	60.0	50.0	50.0
KNN	55.6	50.0	60.0	50.0	50.0

SVM: support-vector machine, LR: logistic regression, DT: decision tree, and KNN: k-nearest neighbors.