Developing a model to evaluate and improve user experience with hand motions in virtual reality environments.

In video games, the evaluation of the user experience (UX) mainly refers to two main groups of aspects, those that refer to the player that is mainly oriented to make the player feel good while playing and those that refer to the video game that is oriented to make the video game easy to understand and play. The aspects considered that are related to the player are engagement, enjoyment, and flow; the aspects related to video game, usability, and dependability. Virtual reality environments today have changed the paradigm in various fields of application, such as health, education, entertainment, among others. Therefore, it is important to observe the effects of handedness with hand movements in virtual reality environments. This work proposes a model to evaluate and improve the user experience considering player and video game aspects, taking into account handedness with hand movements in virtual reality environments. Player and video game aspects can be added to evaluations of the effect of handedness, especially in virtual reality environments, in order to know the user's behavior in terms of skill, performance, and accuracy, among other features by using a particular hand to perform specific tasks. Next, a case study is presented with two groups of users using a virtual reality environment to perform several user tasks considering the dominant and non-dominant hand. By evaluating the user tasks it is possible to know the levels of engagement, enjoyment, motivation, and usability in a virtual reality environment. Finally, an analysis of results is presented in which several improvements of UX are presented.

Introduction

In the field of entertainment, the video game industry is the one that has presented the greatest growth, and is the most preferred by people. In 2019, the global video game market generated revenues of $152.1 billion, with a year-on-year increase of +9.6 percent, and by 2020 the estimated value was $196.0 billion with a Compound Annual Growth Rate (CAGR) of +9.0 % from 2018 to 2022. Today, video games have played an important role in various disciplines and are considered for multiple applications.

Several authors, such as Boyle et al. [1] present video games oriented to science, technology, engineering, and mathematics (STEM) [2-5], as well as video games that are oriented to companies with the purpose of knowledge acquisition. Among the most popular video games today, we can find health video games, which cover exercise, rehabilitation [6, 7], quality of life [8], and advanced life support training [9]. These health-oriented video games were considered to be entertaining and address affective, cognitive, and physiological states. Additionally, they are often considered a pleasant and rewarding activity [10, 11], improving interest in a method that keeps users at the limit of their performance.

Video games are considered a profoundly engaging activity because they focus primarily on elevating User Experience (UX), engagement, presence, immersion, flow, psychological absorption, and dissociation [12]. Nowadays video games have made the leap to Virtual Reality (VR) playing an important role in offering new forms of interaction within virtual reality environments making them even more attractive to users. Among the main advantages offered by virtual reality, we have the presence and telepresence [13], which refers to the feeling of being in an environment. Virtual reality is a technology that has been widely applied to different disciplines such as, entertainment [14], education [15], and health [16].

Authors such as Neri et. al. [17] have presented the positive effects of using virtual reality environments in older adults so that they can improve their mobility skills and balance measurements, in addition to obtaining positive effects regarding balance and fear of falling. Virvou et. al. [18] presented a study in which they measure the usability of virtual reality in an educational game context. The results of this evaluation concluded that the virtual reality environment was usable and enjoyable in students. Due to the wide application of virtual reality video games and their impact on the user experience, it is important to have strategies to consider other factors such as handedness to evaluate and improve the user experience in virtual reality environments.

This work is composed of nine sections: section two presents as background the main aspects that refer to the evaluation of user experience, in addition to handedness. Section three presents a review of works related to existing techniques in the literature to improve the user experience. The problem outline section presents the challenges of designing the proposed model. In section five, the elements that compose the proposed model are described to evaluate the user experience considering the handedness in virtual reality environments. In section six, the implementation of a case study under the proposed model is presented. The results of the case study are presented in section seven and their discussion is presented in section eight. Section eight presents the limitations of this work. Finally, section nine presents conclusions and future work.

Related work

This section presents a literature review of several studies that address the increase of user experience in video games with techniques such as gamification, matchmaking systems, adaptive physics, DDA [19-21], neural networks [22], ML [23], and RL [24]. Studies such as Amico et al. [25], focus on adjusting the game based on three DDA conditions: DDA-based performance, an affective-based DDA from a physiological signal based on valence and perspective excitation and a combination of performance and affective DDA.

The results of this study showed that games can be implemented in virtual reality to improve the overall gaming experience.

In these studies, the authors only focused on the data of the galvanic response of the skin and the scores between the three DDA conditions. In this study, player aspects were not evaluated, although users completed a brief survey and played a virtual reality video game. In general, most of these works address one or a few aspects of virtual reality environments. Table 1 presents a summary of the works focused on improving the user experience considering the aspects of the player and the aspects of the video game.

Table 1

Review of player-videogame interaction literature

Author	Engagement	Enjoyment	Input/output information	Contents	Multiplayer
Yannakakis Georgios et al. [22]	N/A	Flow	Auditory displays	Challenge gameplay difficulty: 3 levels	N/A
Gustavo Andrade et al. [24]	N/A	N/A	N/A	Gameplay: attack and defensive actions. Difficulty: 13 possible levels	N/A
Hunicke Robin et al. [19]	N/A	Flow mentioned	N/A	Challenge: actions off stage. Gameplay: health, upgrade ammo, and reduce strenght enemy	N/A
Ibáñez-Martínez Jesús, et al. [20]	Engagement	Flow	2 Hard controls: PC keyboard and gamepad	Gameplay: racquet length and speed, and ball speed. Difficulty: increase and decrease level	Competition
Yannakakis Georgios et al. [21]	N/A	Enjoyment: questions to participants	Auditory: different sounds Visual: different colors	Challenge: set of rules	N/A
Vicencio-Moreira et al. [26]	N/A	Enjoyment: statistical analysis	PC control	Challenge: player balancing Gameplay: game mechanics	Competition: player balancing
Bian Dayi [23]	Engagement: Random Forest Model from emotions signals	Emotion: physiological signals Enjoyment	N/A	Difficulty: increase and decrease level	N/A
Mirna Paula Silva et al. [27]	Engagement	Emotion flow	PC control	Challenge: AI Gameplay: attack and defense strategy Difficulty: 3 levels	Competition multiplayer
Simone Amico [25]	Immersion, presence and perceived realism: VR	Emotion: physiological signals Enjoyment	Hard control: VR Soft control: trackers	Gameplay: training phase Difficulty: increase and decrease level	N/A

Bian et al. [23] introduced a physiology-based task adaptation for driving skills training, using an affective model; this is a framework based on 3 different physiological data [28] to detect engagement based on an “engagement-sensitive” system (in other words, difficulty increased or decreased based on engagement and performance in 4 different states of difficulty).

However, because the difficulty changes were dependent on performance data, the user experience was not properly adapted, users could enjoy easy difficulty levels, while others might have a preference for difficult difficulty levels. Cowley et al. [29] developed a methodology based on the Person-Artefact-Task (PAT) model [30], this is a relational scheme based on task performance focused on software productivity, to understand how player profiles interact in games using machine learning techniques. However, this proposal does not consider changing the aspects of the video game in real-time, as well as telepresence and presence.

In the literature, several evaluation instruments help measure the aspects of the player and the video game. Such instruments are used by researchers and video game designers to make improvements in their video games. Although these instruments are robust, most are focused on the aspects of the player or the game.

To mention some examples [31, 32], focuse only on engagement [33], emotions and measures engagement and emotions [34, 35]. Some of these evaluations focus on a specific domain, such as violent video games [36] and sports games [37], among others.

As for the handling of the handedness, this has been observed in multiple applications: Barnett et al. [38] conducted a study on the influence of handedness on phantom limb syndrome. This is a phenomenon in which a person with an amputated limb manifests the feeling of that part of the body that no longer belongs to its anatomy from amputated and normal hands [39]. Another study by Grantcharov et. al. [40] presents the impact of handedness, gender and experience with computer video games in a virtual reality environment in which laparoscopy is performed; the study concludes that no difference was found between the control groups and the video game groups.

Pedowitz [41] presents an analysis of the performance of the dominant and non-dominant hand in serious video games in virtual reality. For this study, two groups were available, a group of expert arthroscopic surgeons (n=15) and another group of orthopedic surgery residents (n=10). Both groups tested the two conditions by exchanging instruments between hands.

In the experiment, no statistically significant differences were found between the dominant and non-dominant hands for expert surgeons. The studies presented do not distinguish between the groups analyzed according to handedness, as this can be caused by the interaction technique, the challenge of the tasks, the game experience, or the gender. Therefore, virtual reality promises an alternative to conventional approaches to assessing handedness.

Otherwise, the evaluation of usability in virtual reality environments, helps the designers of these environments and researchers to consider aspects that improve the user experience in an entertainment activity. There are related conceptualized experiences in Human-Computer Interaction (HCI) and several authors have studied these concepts from the gamer-video game domain, and have shown that players’ experiences are closely related in HCI contexts.

Caroux [42] presents an analysis of a total of 72 video game articles related to HCI. In this research it was shown that the player and game aspects were remarkably diverse in the literature; these two aspects are described in the following subsections (see Fig. 1).

Fig. 1

Aspects of player-videogame interactions, and their related concepts. Each aspect (colored frames) is presented in a specific subsection of the results section. Their related concepts (black frames) are detailed in each of these sub-sections (by Caroux et. al. [42], 2015)

Player aspects

According to Caroux [42] the aspects of the player are as presented in the following.

, primarily includes “immersion” and “presence” and is defined as a reflection of complete absorption in a challenging activity, with the occurrence of high concentration, interest and enjoyment without any distraction [43]

, is described as being involved in a video game-related task while possessing an awareness of the outside world [36]. In the literature, it is also referred to as a synonym for concentration [34, 44, 45]. Stanney [46] defines immersion as “a psychological state characterized by perceiving that one is involved, included, and interacting with an environment that provides a continuous stream of stimuli and experiences” [46, 47].

, is conceptualized as the psychological experience of being in a place or environment in a non-physical virtual world [31, 48]. High levels of presence in a virtual reality environment are considered to enhance vividness and intensity [49, 50]. On the other hand, perceived realism is widely studied and validated [51-53] and refers to the realism of feeling inside a virtual world.

, is based on activity-induced skills and challenges, where anxiety is evoked when the challenges are higher than the user’s skills, and boredom occurs when the challenges are lower than the user’s skills. Csikszentmihalyi describes flow as a process of optimal experience, where people under a certain activity, put their abilities to the limit, by concentrated concentration and high enjoyment [54-56]. Hamari et al. [43] describe participation in flow experiences as reflecting complete absorption in a challenging activity, with the occurrence of high concentration, interest, and enjoyment without any distraction. Schiefele et al. [57] report that concentration is related to meaningful learning, interest reflects elementary motivation and encourages users to continue the activity.

, have been widely studied by many disciplines in terms of valence and arousal [58], since affect recognition techniques could lead to various applications, such as, facial expression data sets [59-61], Facial Expression Recognition (FER) [62-64], physiological signals [65-67], sentiment analysis [68, 69], speech emotion recognition [70, 71], and health applications.

Videogame aspects

, are indispensable techniques for player-video game interactions. The output information refers to visual and auditory information, sounds, or music, and the quality of the visual and auditory screens improve the players’ experience positively. Input information refers to the ways in which a task is controlled. As part of HCI, the interaction involves humans using their sensors to control devices. There are two types of control forms, the hard control, such as buttons, switches, joystick, steering wheels, etc. and the soft control, which are the intangible controls within the interface, such as buttons within a software, Brain-Computer Interface (BCI) [72], and gestures among others [73].

, consist of the main scene containing objects with which the player interacts. Caroux [42] presents studies that have shown that visual and auditory information, contextual information, the representation of a virtual environment, sound or music, and the quality of the information displayed influence the performance and experience of the player.

, are important factors for user engagement and enjoyment; these factors in the literature are controlled by changing the difficulty of the video game, such as Dynamic Difficulty Adjustment (DDA), Machine Learning (ML), player-balancing, fuzzy systems, and Reinforcement Learning (RL). These studies are presented in detail in the following section. In addition, game narratives enhance motivation and immersion while playing. Park et al. [74] presented that the narrative context during the introduction or tutorial of the video game increased the player’s presence compared to a context where no narrative was presented.

, are divided into two types: local games, where a group plays locally on the same screen, or through a meeting of people with computers or video game consoles where a local area connection is established (LAN Party). On the other hand, in online games, the nature of the game partner (friend, stranger, or computer) changes where several players can play in the same virtual world through the internet.

Handedness in VR tasks

Handedness refers to the condition in which there is better performance or individual preference for the use of one party, called the dominant hand, over the other, the non-dominant hand, which refers to the less preferred and incapable [75]. As mentioned by Xiaolong [76], the handedness and its related effects are very useful to consider the design of physical work tools that we use today. This can be focused on virtual reality environments, since it implies that the user can have their hands occupied simultaneously, so the study of handedness in virtual reality tasks has a potential field of study. Several works present the effects observed in contexts of therapy, surgery, and games. In the following section, these studies are presented.

Problem outline

The previous section presented a wide variety of works that address the user experience. Only some of these works are focused on evaluating and improving the game experience [23, 25, 29] without considering the fact that all users have different preferences and therefore, the satisfaction of each one has to be achieved in a different way.

Thus, it is necessary to design a model to adapt virtual reality environments to the preferences of the user; the scales that allow the evaluation of the aspects of player and video game have to be analyzed, therefore the following research questions are presented:

Do hand gestures have a significant effect on aspects of the player?

Do hand gestures in virtual reality environments have a significant effect on usability?

Does handedness have a significant effect on virtual reality tasks?

Proposed model

There is a need for strategies that allow improving the user experience from a gaming perspective in a personal way and additionally consider aspects of handedness for virtual reality environments. In that respect, this work proposes an evaluation model that can support the improvement of user experience considering the factor handedness for virtual reality environments.

The evaluation of the improvement of the user experience is achieved through the user-centered model proposed in Fig. 2, analyzing user tasks (with or without the dominant hand) in several usability tests when users interact with virtual reality environments. The proposed model consists of three-stage described below.

Fig. 2

Proposed model for the evaluation and improvement of the user experience for virtual reality environments

In the first stage, a survey is carried out with the participants about their personal data (age, gender, right or left-handed among others) as well as the degree of their experience interacting with virtual reality. The participants were distributed either in a dominant hand group or in the non-dominant hand group.

In the second stage, for each user task, usability tests are carried out considering the handedness for virtual reality environments. The user experience questionnaire UEQ [77] scale is applied after each task.

In the final stage, data is extracted on the evaluation of user aspects, as well as the duration and effort made by each task analyzed in the previous stage. The stage ends, in the statistical analysis on the improvement of the user experience. The proposed model leads us to consider the following challenges related to the evaluation of the user experience in virtual reality environments:

Consider aspects of player-video game interaction in the game scales [33, 34, 36, 37];

To have a model for evaluating the user experience from a personal perspective [23, 28–30];

Consider an evaluation of handedness for virtual reality environments [38, 39, 41];

The rating scales are based on a large number of elements [31, 32, 35].

Implementation model

This section presents the implementation of the proposed model through a case study, where a set of tasks were developed by two groups: (a) the dominant group (participants who were required to use their dominant hand) and (b) the non-dominant group (participants who were required to use their non-dominant hand only). The following describes the implementation design in the context of the proposed model.

Participants

For this case study, we had the participation of 16 people belonging to our research institute (7 women, average age 25, SD = 3.9), 8 were selected to be considered as part of the dominant group and the other 8 for the group considered as non-dominant. It is worth mentioning that it was difficult to gather a larger number of participants because the COVID-19 [78] pandemic was on the rise and health measures were being implemented at the institute, making it difficult for people to come to the institute and it was decided to invite a limited group of people close to them.

In general terms, this number of participants was considered adequate, since it was identified that 85% of the usability problems are covered by testing between 5 to 8 users. To corroborate this, a literature review of works referring to user experience evaluations was carried out and it was found that the number of participants varies between 10 and 25 [79-83]. Participants were initially introduced to a pre-questionnaire before initiating interaction with the virtual reality environment to obtain participant information such as age, gender, virtual reality experience, and handedness. It was possible to corroborate through this pre-questionnaire that half of the participants had no experience with virtual reality; it was also noted that all participants showed interest in the use of virtual reality technology and none of them stated about having a physical disability.

Materials

Various hardware and software elements were used for the development of this experiment. To select the VR environment, a heuristic evaluation of usability for games [84] was selected, where the authors of the evaluation introduced a set of usability inspections in video games; from this, the environment was selected from the LeapMotion1 gallery (applications). These applications were considered appropriate because they are virtual reality environments that in their entirety the user must use their hands to interact. It is worth mentioning that these virtual reality environments are mostly demo versions, and were selected for their simplicity.

Table 2, presents a total of 24 virtual reality environments from the LeapMotion gallery that were examined; all available in the gallery were analyzed though some of them presented compatibility problems or did not work properly.

Table 2

Summary of game heuristics evaluation of LeapMotion Gallery

Video game	Consistency	Customizability	Predictability	Proper views	Skip content^a	Input mappings	Controls	Game status	Training and help	Visual repr.^b	Summary^c
Blocks	∙		∙	∙	∙	∙	∙		∙	∙	8
Particles	∙	∙	∙	∙		∙	∙			∙	7
Cat explorer	∙		∙	∙		∙	∙			∙	6
Mirrors	∙			∙		∙	∙			∙	5
Scaffolding	∙		∙	∙		∙	∙				5
VR visualizer	∙			∙		∙					3
Hovercast VR	∙	∙	∙			∙	∙			∙	6
Virtual Music 2	∙	∙	∙	∙		∙	∙				6
Rainbow Jelly AR	∙		∙	∙		∙	∙				5
UI Input module	∙	∙	∙	∙		∙	∙			∙	7
Pinch Draw	∙	∙	∙	∙		∙	∙		∙	∙	8
Attachments module	∙			∙		∙	∙				4

The ability in which the user is allowed to skip non-playable content so that it does not interfere with game-play. visual representations. summary of total usability evaluation of each video game. Aspects of video game represented

The virtual reality environment Blocks, is a demonstration version within the gallery LeapMotion. The game was selected because it compromises the usability according to the heuristic evaluation presented in Table 2. Blocks is a virtual reality environment in which the user can create cubes and prisms using hand gestures, also the user can grab, stack, or dump the blocks. Several gestures can be used to switch between shapes and turn on/off a gravity system available in the game.

The hardware used to run Blocks consisted of an HTC Vive virtual reality system and a desktop computer with a GeForce RTX 2060 graphics card. In addition, the integration of the Leapmotion controller sensor into the virtual reality system to perform hand and finger movement interactions (soft control); this technology was selected because we believe it improves the perceived realism of common hard controls (e.g., joysticks, keyboard, and virtual reality controls).

For the evaluation of the player aspects, the Game User Experience Satisfaction Scale (GUESS) was selected [85], which covers all aspects of gamer-video game interaction. GUESS consists of 55 items with 9 scales: “playability”, “narratives”, “play engrossment”, “enjoyment”, “creative freedom”, “audio aesthetics”, “personal gratification”, “social connectivity”, and “visual aesthetics”. Due to the simplicity of the virtual reality environment, we ignore the scales of “audio aesthetics” and “social connectivity” because Blocks does not support those features.

In addition, the aspects of the video game of the GUESS scale were not considered, since the objective of this scale was to focus on evaluating aspects of the player. Therefore, GUESS ended up with a total of 19 items from the “play engrossment”, “enjoyment”, and “personal gratification” scales. On the other hand, the usability evaluation was supported by the User Experience Questionnaire (UEQ) because the questionnaire takes about one minute to complete. UEQ contains 6 scales with 26 elements: “attractiveness”, general impression of the product; “perspicuity”, the comprehensibility and difficulty of the product; “efficiency”, i.e. effort in tasks; “dependability”, user control of the interaction; “stimulation”, motivation for the use of the product; “novelty”, product innovation.

Procedure

Participants were asked to voluntarily sign a consent form and were given a brief description of the study. Moreover, they were instructed and given a pre-screening questionnaire to capture information to generate a user profile. The participants entered the room where the virtual reality device is located, which makes up an area of 2.7m x 2.0m. In a first approach to the virtual reality environment, they were allowed to explore and interact with an avatar that acts as an assistant in an initial phase of training. Once the training phase was completed, they were asked to perform the following tasks:

Task 1: Pinch and release, create four cubes and with your hand stack them one by one until you create a tower. (See Fig. 3a);

Fig. 3

Participants tasks. a The participant creates 4 separate cubes and stacks them one into another with one hand. b The participant builds a turret of 4 cubes, turns off the gravity, spawns another cube and pushes it with the hand until the 4 cubes have been hit. c Participants build a turret of 4 cubes b spawning and throwing them 1 by 1, with both hands

Task 2: Levitation and hit, create a tower of four cubes, turn off gravity, create a tower of four cubes, turn off gravity, create a cube and with your hand push it towards the tower until it hits a tower cube, repeat the procedure until all four tower cubes have been hit. (See Fig. 3b);

Task 3: Finger pinch and release, create a tower of four cubes, generate a cube, and stack it, repeat the process until a tower of four cubes is formed (See Fig. 3c).

Design

According to the model presented in Fig. 2, participants entered a virtual reality environment, completed three tasks that were each evaluated, and the information obtained in each of these three tasks was used to perform statistical analysis. A design was developed between subjects with an independent variable (handedness). handedness consists of two levels: the dominant and non-dominant group. Subsequently, user profile variables such as gender, age, and virtual reality experience were considered to analyze a possible effect on the tasks. In the experiment, each participant was randomly assigned to begin the tasks to avoid an order effect.

Figure 4, illustrates the interaction of the experiment from different perspectives. After each task, the perceived usability load was measured using the UEQ, which usually took one minute to respond. Once the three tasks were finished, participants completed the GUESS scale and gave preference scores (e.g., which of the tasks was the least and least preferred, which of the tasks was the least and most preferred, as well as comments on the interaction).

Fig. 4

Video game interaction. a Perspective of the video game from the virtual environment (Blocks). b Participant’s perspective in the real world. c Experimental room

Results

This section presents the results from the information obtained from the experiment carried out in the previous section. An Analysis of Variance (ANOVA) was performed to evaluate the player, performance, and usability aspects of the tasks in the virtual reality environment. Since the purpose of the study does not to attempt to look for correlations between dependent variables, and the work is focused on a series of univariate hypotheses. According to Huberty el al. [86], it is appropriate to use a univariate analysis. The results of the GUESS scale (see Table 3), the data obtained in time and attempts of the tasks performed (see Table 4) , and the results of the UEQ scale are also presented.

Table 3

GUESS scale participant data

Sub-scale	Dominant										Non-dominant
	P2	P3	P5	P6	P7	P8	P9	P10	M	SD	P1	P4	P11	P12	P13	P14	P15	P16	M	SD
Game absorption
I feel detached from the outside world while playing	6	4	5	6	5	3	5	6	5.00	1.07	7	1	6	4	7	6	7	7	5.63	2.13
I don’t care about the events that occur in the real world during the game	7	5	7	5	2	4	2	4	4.50	1.93	7	1	4	5	1	7	5	4	4.25	2.31
I can’t tell I’m getting tired while playing the game	6	4	5	7	4	2	2	5	4.38	1.77	4	4	5	6	7	5	6	5	5.25	1.04
Sometimes I lost track of time while playing the game	5	1	7	5	4	4	1	2	3.63	2.13	7	1	4	7	1	7	6	7	5.00	2.67
I temporarily forget about my daily worries while playing the game	6	6	6	7	2	5	3	7	5.25	1.83	7	1	6	7	7	7	7	5	5.88	2.10
I tend to spend more time playing than I have planned for	7	3	5	6	5	6	5	6	5.38	1.19	7	2	6	7	1	7	7	4	5.13	2.47
I can block out most of the other distractions when playing games	7	4	7	6	4	4	5	7	5.50	1.41	7	1	4	3	4	7	7	5	4.75	2.19
Every time I stop playing the game, I can’t wait to play it again	4	5	7	4	2	1	2	4	3.63	1.92	7	3	4	7	1	7	4	4	4.63	2.20
Player enjoyment
I think the game is fun	7	7	7	7	5	7	7	7	6.75	0.71	7	6	7	7	7	7	7	6	6.75	0.46
I enjoyed playing the game	7	7	7	7	4	7	6	7	6.50	1.07	7	1	7	7	7	7	7	7	6.25	2.12
I felt bored while playing	1	1	3	1	4	2	2	2	2.00	1.07	1	7	2	2	1	1	1	1	2.00	2.07
I am likely to recommend this game to others	7	5	7	6	5	6	6	7	6.13	0.83	4	2	7	7	7	7	7	4	5.63	2.00
If the opportunity exists, I would like to play it again	7	5	7	7	4	6	4	7	5.88	1.36	7	1	7	7	7	7	7	6	6.13	2.10
Personal gratification
I am in suspense about whether I will succeed in the game	6	4	6	4	6	3	2	2	7.00	0.00	6	7	6	4	5	7	6	4	5.50	1.71
I feel successful when I overcome obstacles in the game	7	7	7	7	4	5	4	6	6.17	0.90	7	7	7	7	7	7	7	7	6.33	0.75
I want to do my best during the game	7	5	7	6	5	6	6	7	6.00	1.00	7	7	7	6	7	7	3	7	5.67	1.60
I am very focused on my own performance while playing the game	7	2	5	6	4	4	6	5	4.33	0.94	7	7	6	3	7	7	7	7	6.17	1.21
I feel that the game constantly motivates me to continue to the next levels	5	4	5	6	3	5	3	7	4.67	0.94	4	7	7	7	4	7	7	5	6.83	0.37
I think my skills gradually improve through the challenges in the game	7	5	7	7	4	5	5	6	4.33	1.49	6	7	5	7	7	6	6	6	6.00	1.41

The table shows the data divided into dominant and non-dominant groups. Descriptive data are shown for each participant

Table 4

Data captured from the case study to evaluate user experience

Group	Participant	Gender	Experience with VR	Time on task 1	Time on task 2	Time on task 3	Attempts at task 1	Attempts at task 2	Attempts at task 3
Dominant	2	M	Null	117.62	34.61	119.49	7	1	15
	3	F	little	60.41	167.20	185.32	2	14	49
	5	M	little	41.20	232.50	206.93	2	10	20
	6	F	Null	732.00	87.67	97.83	3	7	13
	7	F	Null	64.00	121.67	97.14	1	4	5
	8	M	Null	27.80	90.91	314..01	1	9	15
	9	M	Null	112.21	293.83	141.03	5	15	8
	10	M	little	40.26	53.65	40.67	3	2	13
Non-dominant	1	M	Null	70.00	68.20	181.59	3	1	13
	4	F	little	467.20	371.53	203.51	5	26	15
	11	M	little	50.83	202.34	71.04	2	12	9
	12	M	little	50.79	163.51	97.53	5	15	24
	13	M	Null	298.36	221.35	66.45	4	11	16
	14	F	Null	90.80	95.56	320.77	7	7	7
	15	F	little	276.38	78.59	35.31	3	5	5
	16	F	little	94.38	97.75	57.45	3	4	8

Time results are shown in seconds

Effects of hand gestures on the player aspect

The first research question (RQ1), presented in Sect. 3 refers to the fact that hand gestures have a significant effect on the player’s aspects. To answer this question, the evaluations of “player engrossment”, “player enjoyment” and “personal gratification” were analyzed. As presented in Fig. 5, the evaluation of “player engrossment” presented an average of 4.65 for the dominant group and 5.06 for the non-dominant group. Since there was no considerable variation in evaluation among participants, the difference was not statistically significant (, ns). Although not significant, both groups evaluated the task as immersive and engaging.

Fig. 5

Player aspects evaluation from GUESS scale between the dominant and non-dominant group

In the evaluation of “player enjoyment” no significant difference was presented (, ns), but high levels of pleasure were manifested in both groups. Finally, in the evaluation of “personal gratification” the means were 5.25 for the dominant group and 6.27 for the non-dominant group. The non-dominant group rated the tasks 17.7% more motivating than the dominant group, so there is a statistically significant difference (, p = .012). These results could be affected by the challenge of completing a task without the dominant hand, invoking a sense of achievement.

Effects of hand gestures on usability

The second research question (RQ2) posed in Sect. 3 refers to whether hand gestures for interaction in virtual reality environments have a significant effect on usability. To answer this question, the results of the UEQ scale presented in Fig. 6 were analyzed. Together, an ANOVA one-way between subjects was performed to compare the handedness gestures on usability.

Fig. 6

UEQ scores for each task. The score level ranges from − 3 as the most negative value, to  3 the most positive value

The results showed a non-significant effect of hand gestures on usability at the level. Except for “efficiency” and “dependability”, all scales showed high values, this was due to the fact that the participants manifested to require a greater effort to complete the tasks, it is worth mentioning that none of them had experience in performing hand movements in virtual reality environments. Therefore, the participants, in general, got a great overall impression of the tasks, understandable, exciting and valuable as shown in Fig. 6.

Effects of handedness on VR tasks

To address the issue raised in RQ3, i.e whether handedness has a significant effect on virtual reality tasks—time and attempts—an ANOVA was conducted between the two groups of participants.

As can be seen in Fig. 7, a significant effect of handedness was shown in the Task 1 attempts with (, p = .084). In the attempts of Task 2 (, ns) and Task 3 (, ns) no significant effect was shown. The recorded times for Task 1 (, ns), Task 2 (, ns) and Task 3 (, ns) can be seen in Fig. 8.

Fig. 7

Task attempt performance in the virtual reality environment between the dominant and non-dominant group

Fig. 8

Task time performance in the virtual reality environment between the dominant and non-dominant group

Additionally, it was explored if there is any effect on the participants considering aspects such as gender (male or female), the experience with virtual reality (none or little) and the handedness (left or right hand) in the tasks within the virtual reality environment.

The results obtained showed that there is no significant effect between these aspects, and the data obtained show that between left and right-handers the time of Task 3 is at (, p = .022) level, while the gender aspect in Task 1 (, p = .078) at p<.10 level.

On the gender aspect, a trend was found that males completed the tasks 42.9% faster over time (the gender averages were 5.47 for men and 8.46 for females), as well as in attempts with 47.9%, the averages were 26.44 for men and 43.14 for women.

In addition, there was no noticeable difference between users who had a virtual reality experience and novice users. The results are discussed in the following section.

Discussion

This section discusses the results obtained from the implementation of the proposed model in Sect. 3. Several ANOVA analyses were performed for the three proposed tasks. The results showed that handedness movements performing hand movements do not have an effect on the tasks in the virtual reality environment, but rather establish high levels of player and usability aspects through the GUESS scale. These high levels could be caused by the immersion, presence and realism perceived by the user within the virtual reality environment. Gestures used in virtual reality tasks were considered as “soft control” (no physical contact). In addition, the comparison between different interactions or control techniques can contrast the user experience, for example, “hard control” may be preferred by users while other users may prefer eye-tracking or multimodal techniques.

Although the participants described the tasks they performed within the virtual reality environment as attractive and innovative, they showed little insecurity or unpredictability about being immersed in virtual reality.

The participants stated that Task 2 with 68.75% (n = 11) was their favorite and the most attractive to perform. Following this, 31.25% of the participants expressed a preference for Task 3,—none of the participants selected Task 1 as the most preferred. Q10: “I liked Task 2 because I could do more things within the environment,” Q12: “I could use the levitation effect, that was a lot of fun,” Q14: “I preferred Task 3 because I had practice, and it was the easiest,” Q9 and Q1: “Task 1 was pretty simple”.

In terms of limitations, our study had a relatively small difference between each task, which we assume led to similar results in usability. It is acceptable that more complicated tasks or levels require more effort; consequently, different results could be reflected. Also, to develop more elaborate tasks, one can make use of virtual reality environments that evoke different contexts, such as sports games or environments that need precision and that can be considered a key human factor in evaluating aspects of the player and usability.

The effect of handedness on VR tasks addressed the problem where there was no evaluation of handedness in entertainment VR video games. Results from handedness on attempts and time did not show a significant effect at the p<.10 level, which is somewhat surprising; these results may be influenced by the reason that participants developed skills during the experiment and tasks were effortless and uncomplicated. A 0.10 significance level was used due to the limitations of the study: (i) sample size (n=16)—although a number of 5 to 8 users covers 85% of usability problems [87]; (ii) tasks interaction were slightly different—, due to the limitations of the game; and (iii) there is no authoritative reference to use a standard or conventional p level [88, 89]. Moreover, user profiles presented an effect where males completed the tasks faster and in fewer attempts, while no difference was observed in the VR experience, which could be as a result that none of the participants was a regular user for VR or hand tracking. In addition, there was no difference effect between left and right-handed players.

In terms of user profiles, it can be noted that the male participants completed the tasks faster and in fewer attempts, while no differences were observed in the virtual reality experience, which could be due to the fact that none of the participants were regular users of virtual reality or hand-tracking. Also, there was no difference between left-handed and right-handed players.

The experiment in this study consisted of an evaluation of handedness in virtual reality tasks with hand and finger movements as inputs. In addition, different tasks or levels result in different levels of user satisfaction, some users may prefer difficult level tasks, while others may prefer simple tasks. From this, there are multiple works to improve user experience in video games, and techniques such as DDA [19, 25, 27] and machine learning [23, 90] are the most applied. As algorithms improve the user experience, they tend to generalize the preferences of a group, ignoring the fact that each user has different preferences. Other approaches, such as reinforcement learning and multi-modal systems (e.g. Machine learning & physiological signals) ensure the user experience for each user in a personal way.

Limitations

The pandemic caused by the COVID-19 [78] limited physical contact and closeness between people, so the nature of this research was oriented to an experimental design in order to comply with the sanitary measures established by the government. The following are some of the limitations of the present work:

The limited number of participants involved in the experiment was due to health restrictions. It was not possible to have face-to-face meetings with the participants and to conduct research with an experimental design with various participant profiles, such as left-handed and right-handed players;

The implementation of other design strategies for the experiment, such as within-subjects and between-subjects [91]. It is acceptable that more complicated tasks or levels require more effort; consequently, different results could be reflected. In addition, to develop more elaborate tasks, virtual reality environments that evoke different contexts, such as sports games or environments that require accuracy and can be considered a key human factor to evaluate aspects of player and usability, can be used;

Tasks involving precise manipulation of objects using the dominant versus non-dominant hand were not included. The effect of laterality in VR tasks addressed the problem that there was no laterality assessment in entertainment VR video games. There was no difference effect between left-handed and right-handed players. In addition, the user profiles showed an effect of males completing tasks faster and in fewer attempts, but no difference was observed in the VR experience, which could be because none of the participants were regular users of VR or hand tracking.

Conclusion and future work

This work has evaluated the effects of user experience (player and video game aspects) in a virtual reality environment under a proposed model. The effects of handedness on player and video game aspects with user experience scales were analyzed. Based on 3 different tasks (n = 16), it was found that the model led to high levels of engagement, enjoyment, motivation, and usability in the proposed virtual reality environment.

Likewise, no significant differences were found between the dominant and non-dominant groups in terms of handedness in the case study. Similarly, it was found that the time and attempts were similar among the three tasks. This leads us to consider future virtual reality environments with a context that allows the incorporation of tasks with a higher level of complexity and analyze their effects. These evaluations provide design elements for new virtual reality environments oriented to different contexts such as education, medicine, training, among others. In these environments the tasks to be performed by the user are an important aspect and can improve their experience within the virtual environment.

As future work, we propose the incorporation of aspects that allow expanding the user’s experience in a personal way within virtual reality environments, including tasks with a degree of specialization, such as writing, drawing in contexts according to the user’s needs in which the user’s experience can also be measured and thus establish improvements in virtual environments. Moreover, we plan to include a greater variety of users with diverse profiles that allows a more thorough investigation of their preferences. In addition, we plan to extend the proposed model to incorporate reinforcement learning algorithms, consider aspects of the video game such as audio, content, and strategies so that the elements within the virtual reality environments can automatically adapt to users to improve their satisfaction and avoid negative results or emotions.