Enhancement of perceived body ownership in virtual reality-based teleoperation may backfire in the execution of high-risk tasks.

Guided by previous research on the role of embodiment in virtual environments, this study aimed to investigate the potential effects of using human-like (compared to robotic) virtual hands on work performances in the context of virtual reality (VR)-based teleoperation of high-risk machinery. A 2 × 2 mixed factorial design experiment (N = 74), with the virtual hand representation as a within-subjects factor (robotic vs. human-like virtual hands) and the risk of danger as a between-subjects factor (low vs. high), was conducted to examine the effects of virtual hand representations (i.e., human-likeness) on perceived body ownership (i.e., embodiment), risk perception, intention to work using the teleoperator, and work performance (i.e., the number of successful task completions). In addition, the moderating effects of the risk of danger on the relationship between perceived body ownership and risk perception were explored. Results showed that the enhancement of perceived body ownership in VR-based teleoperation, induced by the use of human-like hands, increased the risk perception and degraded workers' task performances in the execution of high-risk tasks. Further implications of the findings were discussed.

Introduction

With recent advances in automation technologies, manufacturing industries have begun to replace human workers with automated machinery (i.e., robots) for the execution of high-risk tasks. While the adoption of automated machinery may provide an opportunity for the reduction labor costs and prevention of human injuries in the process of operating machinery under high-risk conditions, it should be noted that there are many complex tasks that require human intervention (i.e., monitoring and controlling) for the reduction of operation errors. To solve these drawbacks, teleoperation systems were developed for the remote operation of machinery by human workers. In general, teleoperation refers to the human operation of a machine on its environment at a distance (Sheridan, 1992).

Teleoperation has received significant attention from researchers for the prevention of worker injuries and fatalities from the 1940s onward (Mollet, Chellali, & Brayda, 2009). In alignment with this, the industrial report from Teleoperation and Telerobotics (2020) forecasts that the global teleoperation and telerobotics market will continue to grow and reach a market size of $98.3 billion by 2027. Intriguingly, this report also highlights that more than 50% of the investments in new virtual reality (VR) technologies will be for the solution of problems in the teleoperation industry. This trend denotes that at the heart of the growing teleoperation market is the integration of teleoperators and VR technologies. Conceptually, VR refers to “a real or simulated environment in which a perceiver experiences telepresence” (Steuer, 1992, pp. 76–77). In this classical definition, telepresence is defined as a feeling of being there in a virtual environment (Biocca, 1997). It is noteworthy that the term telepresence was first coined by Minsky (1980) to explain the characteristic of a teleoperation system.

A plethora of research has indicated that VR technologies can be effective for teleoperation (e.g., Goto, Inoue, Tezuka, & Yoshikawa, 1995; Mollet et al., 2009; Mostefa, El Boudadi, Loukil, Mohamed, & Amine, 2015), given that the integration of teleoperator and VR technologies (i.e., VR-based teleoperators) allows for the safe and remote operation of machinery under high-risk conditions by human workers. For instance, VR-based teleoperators can be used by human operators at a distance for the safe searching and rescuing of people under disaster situations or for the safe processing of dangerous explosive materials. In line with this notion, previous research demonstrates that a human-involved VR-based teleoperator can be more effective than a fully automated system or direct telepresence system, based on accurate simulations of human worker experiences. In particular, human workers with a VR-based teleoperator were found to demonstrate significantly higher performances than a fully automated system or a direct telepresence system in the execution of the grasping and assembly task (Lipton, Fay, & Rus, 2017).

For the further improvement of worker task performances in teleoperation, it has been suggested that the capacity of the workers to perceive the body and hands of the machinery as their body and hands (i.e., the sense of embodiment; Blanke & Metzinger, 2009) can play a pivotal role in enhancing the operator control (Toet, Kuling, Krom, & van Erp, 2020). However, this may not be the case when workers using VR-based teleoperators carry out high-risk tasks while in control of a “human-like” virtual body designed for the enhancement of perceived embodiment. In a recent study, it was found that the enhancement of perceived embodiment through the rendering of realism into a virtual body may be ineffective when the high risk of danger is translated into the virtual environment (VE) (Argelaguet, Hoyet, Trico, & Lécuyer, 2016). This research implies that VR-based teleoperators developed for the accurate handling of high-risk tasks may fail and decrease the work performances when perceived embodiment is enhanced by the use of a human-like virtual body.

In this study, we aim to explore whether and how enhancing the sense of embodiment in VR-based teleoperation will improve worker task performances when the risk of danger is apparent in a task. With reference to the literature on perceived embodiment and related works on risk perception in VEs, this study specifically attempts to investigate a) the effects of rendering human-likeness into virtual hands (i.e., robotic vs. human-like virtual hands) on perceived embodiment (body ownership), and b) the effects of perceived embodiment on the intention of workers to use the VR-based teleoperator in the future and their work performances (i.e., the number of tasks successfully completed) due to an increase in risk perceptions upon the assignment of high-risk tasks. In addition, c) the moderating effects of the risk of physical danger (i.e., low vs. high) on the relationship between perceived embodiment and risk perception were examined to ascertain whether the risk of physical danger will amplify the risk perception upon the enhancement of perceived embodiment by the use of human-like virtual hands. The findings of this study are expected to serve as a basis for the design of improved VR-based teleoperators.

Background

Virtual reality-based teleoperators

The integration of teleoperators and VR technologies (i.e., VR-based teleoperators) has attracted significant attention in various fields such as the manufacturing (Lipton et al., 2017), medical (Alaraj et al., 2011), and military (Kot & Novák, 2018) industries. Such scholarly interests are based on the premise that rendering the sense of embodiment in teleoperating machinery through VR technologies can significantly improve the controllability of machinery at a remote distance (Zhai & Milgram, 1991). Indeed, workers at a remote distance may fail to precisely operate a teleoperation system if the medium designed to connect the human workers and machinery fails to render a natural perception of embodiment. In line with the idea, recent literature suggests that inducing a sense of embodiment in the teleoperation machinery using VR technologies may significantly improve worker task performances (e.g., Toet et al., 2020). The psychological construct underlying the accurate simulation of first-person worker experience is perceived embodiment (Kilteni, Groten, & Slater, 2012).

Perceived embodiment (body ownership) in the teleoperation of high-risk tasks

Perceived embodiment refers to the “subjective experience of using and ‘having’ a body’’ (p. 7) (Blanke & Metzinger, 2009). Kilteni et al. (2012) demonstrate that embodiment is a complex construct that is mainly comprised of three sub-dimensions: the sense of 1) self-location, 2) agency, and 3) body ownership. According to Argelaguet et al. (2016, p. 2), self-location occurs in “the space in which we perceive the self to be located,” whereas agency and body ownership respectively occur when one experiences “the feeling of being in control of the avatar” and “the feeling that the avatar is the source of experienced sensations” (p. 1). Kilteni et al. (2012, p. 378) propose that experiencing “a minimum of one of the three senses at a minimal intensity” can be interpreted as one having the sense of embodiment.

Among the three sub-dimensions of perceived embodiment, a recent study conducted by Argelaguet et al. (2016) implies that enhancing the feeling of machinery (virtual body) ownership may play a critical role in moderating the task performance under high-risk conditions. Of numerous potential factors that could enhance the sense of body ownership, previous studies have consistently highlighted that the realism of the visual appearance of an external object in control (i.e., the morphological similarities between an actual body component and the external object in control) can contribute significantly to perceived body ownership (Tsakiris et al., 2008, 2010). Similarly, Kilteni et al. (2012) suggested that the degree to which the virtual representation of a body component (e.g., hands or arms) appears human-like can significantly affect the sense of body ownership in VEs. In support of the notion, numerous studies have found that the rendering of realism (or human-likeness) into a virtual body can enhance perceived body ownership in VEs (Argelaguet et al., 2016; Haans, Ijsselsteijn, & de Kort, 2008; Lin & Jörg, 2016; Maselli & Slater, 2013).

Previously, much research has been conducted to understand whether and how embodiment in VEs could be utilized for positively changing embodied user perceptions and behaviors under various contexts such as pain management (Mancini, Longo, Kammers, & Haggard, 2011; Martini, Perez-Marcos, & Sanchez-Vives, 2014), phobia treatment (Rothbaum et al., 1995), physical exercising (Born, Abramowski, & Maic, 2019) and musician training (Bissonnette, Dubé, Provencher, & Sala, 2016). In general, the enhancement of the sense of embodiment was found to have positive effects on embodied user perceptions and behaviors. For example, the embodiment of a virtual body or location in VEs is found to significantly reduce pain and phobia among patients (Bissonnette et al., 2016; Martini et al., 2014; Rothbaum et al., 1995), while increasing the performance of players in VR exergames (Born, Abramowski, & Masuch, 2019). However, it merits attention that the role of perceived embodiment on task performance in the context of teleoperation has been barely examined by previous researchers (Toet et al., 2020).

Albeit limited, there have been a few studies which found that the enhancement of perceived embodiment in VR-based teleoperation can have positive effects on task performances (Almeida, Patrao, Menezes, & Dias, 2014; Almeida, Menezes, & Dias, 2017). Such studies consistently articulate that enhancing the sense of embodiment may improve the task performance by significantly improving the controllability of teleoperators. However, the results might unfold in a different way when the risk of danger is translated into the VR-based teleoperation. This speculation is particularly guided by the previous findings in which the sense of embodiment negatively influenced the psychological responses of users as well as their task performances.

For example, in a study conducted by Gamberini, Cottone, Spagnolli, Varotto, and Mantovani (2003), participants in a VE were found to perceive a virtual fire that was situated close to their avatars as physically threatening, although they were aware that the threat of a virtual fire does not have physical consequences. Intriguingly, Argelaguet et al. (2016) suggest that such negative perceptions might also extend to users’ task performances in VEs. Specifically, Argelaguet et al. (2016) found that the enhancement of perceived embodiment using human-like virtual hands in a VE can decrease the accuracy of task performances by increasing the risk perception when dangerous obstacles are placed close to the human-like virtual hands.

Overall, these findings imply that the enhancement of perceived embodiment may have a negative influence on the psychological responses of workers as well as on their task performances when a high-risk task is simulated in a VR-based teleoperation task. However, despite the findings, there is lack of empirical evidence in support of the possible negative consequences of rendering the sense of embodiment in such a condition. Therefore, this study attempts to explicate the role of embodiment on the task performance of workers, particularly focusing on its effects in the context of VR-based teleoperation under high-risk conditions. Such an investigation may provide important insights into the future design of VR-based teleoperators, given that teleoperators are fundamentally developed to take a role in preventing human injuries in the process of operating machinery under high-risk conditions.

Research model and hypotheses

To investigate the potential role of perceived embodiment on risk perception, task performances and technology acceptance under high-risk conditions, we developed a research model in which the sense of embodiment, enhanced by the rendering of realism into a virtual body (i.e., virtual hands), is predicted to increase risk perception and subsequently reduce work performances and intention to work using a VR-based teleoperator in the future. In addition, this research model predicts that the relationship between the sense of embodiment and risk perception will be moderated by the risk of danger in a task. Fig. 1 shows the holistic view of the hypotheses postulated in this research.

Fig. 1

Research model.

Virtual body representation and perceived body ownership

As mentioned earlier, numerous studies have demonstrated that the rendering of realism into a virtual body can increase the sense of body ownership in VEs (Argelaguet et al., 2016; Haans et al., 2008; Lin & Jörg, 2016; Maselli & Slater, 2013). In past studies, human-like virtual hands are frequently compared with robotic virtual hands to investigate the effects of virtual body representation on the sense of embodiment (e.g., D'Alonzo et al., 2019; Lugrin, Latt, & Latoschik, 2015). In the context of teleoperation, such a comparison should be more interesting, considering that teleoperation often encompasses the concept of telerobotics (Sheridan, 1989). Therefore, the current study employs the experimental manipulation in which the hands of a telerobot located at a remote workplace (i.e., robotic virtual hands) are presented to workers. The comparison between the effects of using robotic virtual hands and human-like virtual hands for teleoperation becomes even more interesting, because the use of human-like virtual hands may potentially elicit the illusion that the telerobot's hands operated at a distance are part of workers' body to a greater extent, which is often deemed an important prerequisite for facilitating task performances in teleoperation (Toet et al., 2020). Taken together, the current research investigates whether the use of human-like virtual hands for teleoperation will engender a greater sense of body ownership (i.e., embodiment) than the use of robotic virtual hands in a virtual factory environment:

The use of human-like virtual hands for the VR-based teleoperation of machinery in a virtual factory environment will engender a greater perception of body ownership when compared with the use of robotic virtual hands.

Perceived body ownership and risk perception

The enhancement of perceived embodiment can increase the perception of physical risks (i.e., the perception of “the potential threat to an individual's safety, physical health, and wellbeing”; Lu, Hsu, & Hsu, 2005, p. 109) in a task when the risk of physical danger is translated into the VR-based teleoperation task. Potentially, perceived body ownership, induced by the human-like virtual body representation in a VE, may result in a greater risk perception in the execution of high-risk tasks using a VR-based teleoperator, by creating the illusion that virtual threats will lead to significant physical consequences (i.e., physical damage). This argument is in accordance with the previous notion that a scenario is likely to be perceived as high-risk when the occurrence of a harmful event (i.e., vulnerability) is anticipated to lead to severe physical consequences (i.e., severity) (Bauer, 1960). Furthermore, there are some studies which provide direct empirical evidence for our study to predict that the enhancement of perceived embodiment will increase risk perception when the risk of physical danger is translated into a VE (Argelaguet et al., 2016; Gamberinie et al., 2003). The findings of such studies imply that enhancing the sense of embodiment through the rendering of human-likeness into virtual hands can make workers feel unsafe in handling high-risk tasks in VR-based teleoperation insofar as it induces the illusion that they will be harmed by the tasks regardless of the virtuality of their experiences. Based on the argument, we propose the second hypothesis:

A greater perception of body ownership, induced by using human-like virtual hands, will increase the risk perception in the teleoperation of high-risk machinery.

Moderating effects of the risk of physical danger

The risk of physical danger may moderate the relationship between perceived embodiment and risk perception, such that the association between perceived embodiment and risk perception becomes stronger when the risk of physical danger is higher in a VR-based teleoperation task. This speculation is particularly based on the findings of a neuropsychology study, wherein the risk level of physical danger is manipulated by simulating a knife attack on the virtual right hands of participants or on the tables situated close to their hands in a VE (González-Franco et al., 2014). In this study, the increase in the realism of the human-like virtual hand was found to amplify the risk perception of the subjects with respect to the physical damage of their hands (i.e., mu-rhythm event related desynchronization [ERD] in the motor cortex was observed, in addition to readiness potential [C3–C4] negativity). Such findings indicate that the enhancement of perceived embodiment by the use of human-like virtual hands in the VR-based teleoperation of high-risk tasks may potentially amplify the risk perception in accordance with an increase in the risk of physical danger. Accordingly, the following hypothesis is proposed:

The risk of danger will moderate the relationship between perceived body ownership and risk perception, such that the relationship is enhanced when the risk of danger is higher.

Risk perception and intention to work using the VR-based teleoperator

The increase in risk perception, induced by perceived embodiment, may subsequently reduce the likelihood of using a VR-based teleoperator in the future. The investigation of the behavioral intention to use (i.e., acceptance) a technology in the future is important, as behavioral intention plays a critical role in the prediction of actual future behavior (Ajzen, 1991). In the domain of human-robot interaction, positive attitudes towards a robot are often found to result in an increased intention to work with the robot due to a decrease in the fear of failure (i.e., You & Robert, 2018). In a similar vein, You, Kim, Lee, Kamat, and Robert (2018) found that the perceived safety of a task can significantly increase the intention to work with a robot in the future. Moreover, the perceived risk is considered as a basis of behavioral intention (i.e., intention to accept a technology) in the extended version of the technology acceptance model developed by Lu et al. (2005). Worthy of note, this research found that the perceived risk can negatively affect the intention to use an online application under security threats. Such findings consistently suggest that the increase in risk perception will negatively affect the intention to use the VR-based teleoperator in the future by inducing a negative attitude. Taken together, the following hypothesis is presented:

The increase in risk perception, induced by a greater perception of body ownership, will negatively influence workers' intention to use the VR-based teleoperator in the future.

Risk perception and work performance

Despite a volume of research on embodiment and teleoperation, the link between the sense of embodiment and work (i.e., task) performances has been poorly examined in the context of teleoperation (Toet et al., 2020). Accordingly, the current study explores the potential relationship between risk perception and work performance to ascertain whether workers will demonstrate improved performances in the teleoperation of machinery under high-risk conditions upon the enhancement of perceived embodiment using human-like virtual hands in a virtual factory environment. Potentially, the increase in risk perception may limit the work performance, as fear is the dominant affective response under high-risk conditions (Sharf & Binder, 1983). Williams (2012) demonstrates that fear may evolutionarily facilitate the defensive mechanism, thus causing humans to avoid high-risk scenarios. When a high risk of danger is perceived during a task, the innate defensive mechanism activated by fear may lead to hesitation or the non-execution of tasks. Consequently, the activation of the defensive mechanism may limit the capacity of workers with respect to the execution of high-risk tasks.

Empirically, previous studies have shown that task performance may decrease by the activation of a protective mechanism when risk perception is increased (Rice & Thomas, 2000). For example, Rice and Thomas (2000) found that subjects engaged in protective behaviors when asked to pour hot water (high-risk condition), as opposed to when asked to pour cold water (low-risk condition). In particular, participants under the high-risk condition were found to behave slowly and to more significantly adjust their behaviors, which subsequently decreased the task performances. A similar study was thereafter conducted on the elderly, and the results were consistent with the previous findings (Thomas & Rice, 2002).

The study conducted by Argelaguet et al. (2016) provides more direct evidence to predict that the increase in risk perception will decrease the performance of workers in cases wherein they operate high-risk machinery in a VE. This research implies that the enhancement of perceived embodiment by the use of human-like virtual hands may increase risk perception and decrease the accuracy of pick-and-place task performances under high-risk conditions (i.e., upon exposure of participants to fire), when compared with low-risk conditions (i.e., upon exposure of participants to plain bricks or barbed wire). In sum, the following hypothesis is posited:

An increase in the risk perception, induced by a greater perception of body ownership, will negatively influence the work performance of users in the teleoperation of high-risk machinery.

Methods

Participants and experimental design

A 2 × 2 mixed factorial design laboratory experiment, with the virtual hand representation as a within-subjects factor (Robotic vs. Human-like) and the risk of danger (low vs. high) as a between-subjects factor, was conducted to test the hypotheses presented in the research model (see Fig. 1). A total of 74 university students at a private university in South Korea were recruited using the convenient sampling method. The ages of the participants recruited for the study ranged from 19 to 28 (M = 21.34, SD = 1.64). In order to control for any possible sex effects, the sex of the participants was approximately balanced among the between-subjects conditions (Low danger risk condition: n male = 15, n female = 21; high risk of danger condition: n male = 15, n female = 23). The results from the chi-square test revealed that the sex was statistically well balanced among the between-subjects factors: χ (1) = 0.04, n.s.. Regarding participants’ previous experiences with VR, 10.8% of participants (i.e., eight participants) indicated that they had no experience with VR. 83.2% of participants (i.e., 61 participants) had low to moderate level of experience, while only 6.8% of them (i.e., five participants) indicated that they were familiar with VR. In addition, the order of the virtual hand representation was randomly assigned and balanced across the participants to control for possible order effects.

Stimulus and experiment

The Unity Engine software (Unity Technologies, 2020) was used to develop a virtual factory environment that allows for the teleoperation of a metal press machine in a VE. In this VE, there was a start button on the left side of the participants for the start of the task. Upon pressing the start button, the conveyor belt system delivered a raw material. The participants were then assigned a task that involved the grasping of the raw material and its placement under the metal press machine. After the machine pressed the raw material, participants were instructed to grasp the processed material and place it on the conveyor belt situated on the right side of the work desk for the completion of a task. The task was designed based on the pick-and-place task assigned to participants in the study of Argelaguet et al. (2016). During the assigned task, participants were interrupted by the sound of an alarm if their hands were trapped in the machine, which indicated the failure of the task, and were required to restart the task.

For the experiment, the HTC VIVE Pro head-mount display was employed. The natural three-dimensional movement of the virtual hand was rendered using the Leap Motion software development kit (SDK), as it allows for the seamless tracking of the forearm and fingers without the use of wearable devices (i.e., gloves) (Argelaguet et al., 2016). For the rendering of robotic and human-like virtual hands, we purchased the Leap Motion core asset. In the virtual factory environment, the risk of danger was manipulated by differentiating the pressing durations of the machine start button as follows: 3 s for the low-risk condition and 1 s for the high-risk condition. This manipulation was based on the results of a preliminary test conducted on five participants. The screenshot of the stimuli (i.e., virtual factory environment) developed for the experiment is presented in Fig. 2 . The images of the virtual hands used in the experiment are presented in Fig. 3 .

Fig. 2

The Virtual Factory Environment developed for the Experiment (Upon pressing the start button (A), the conveyor belt system delivered the raw material (B). The participants were then asked to grasp the raw material and place it under the metal press machine (C). After the machine pressed the raw material, participants were instructed to grasp the processed material and place it on the conveyor belt situated on the right side of the work desk (D).).

Fig. 3

Robotic and Human-like Virtual Hands used in the Experiment (Retrieved from: https://developer-archive.leapmotion.com/documentation/v2/unity/unity/Unity_Hand_Assets.html).

Procedures

Upon entering the laboratory space assigned for the experiments, the participants were instructed to complete a consent form and a pre-survey questionnaire. The pre-survey questionnaire asked the participants to report their demographic information (i.e., age and sex). After the completion of the pre-survey, participants were provided with background information on the experimental task. Participants were informed that the objective of the experiment was to test the design of a VR-based teleoperating system developed by researchers, based on the completion of a teleoperation task using a metal press machine in a virtual factory environment. Also, participants were instructed to put themselves into the shoes of factory workers who will operate machinery using a VR-based teleoperator at a distance. A tutorial session was then conducted for 2 min to familiarize the participants with the control settings of the VR system and rationale of the task.

In the main experiment, all the participants were assigned 2 min to complete the experimental task, as detailed in the Stimulus and Experimental Task section. A maximum of 20 trials were conducted per session. As previously mentioned, the order of the task with respect to the different virtual hand representations (i.e., robotic and human-like virtual hand) was randomly assigned among the participants. After the completion of each session, participants were asked to complete a post-survey questionnaire. The duration of the experiment was approximately 20 min per participant, and students were provided with extra course credits as a reward for their participation.

Measures

The perceived body ownership (embodiment) was measured by four seven-point Likert scale items extracted from the study of Caspar, Cleeremans, and Haggard (2015). The items such as “I felt as if I was looking at my own hand” and “I felt as if the virtual hand was part of my body” were used to measure the degree to which participants perceived body ownership with respect to different representations of virtual hands. The internal consistency of the measure was acceptable (robotic hand: α = .86; human-like hand: α = 0.89).

The risk perception using three seven-point Likert scale items adapted from the previous study conducted by You and Robert (2018). The three items were “I felt as though I encountered personally hazardous scenarios while using the virtual hands for the given tasks,” “I felt as though the work experience using the virtual hands was physically dangerous,” and “I felt as though I was directly exposed to physical harm in executing the virtual factory work tasks using the virtual hands.” The internal consistency of the measure was acceptable (robotic hand: α = 0.86; human-like hand: α = 0.89).

The intention to work using the teleoperator was measured via three seven-point Likert scale items extracted from the previous study (You & Robert, 2018). The items used were “Assuming I had another project similar to this one, I am willing to use the teleoperating system with the virtual hands,” “This teleoperating system with the virtual hands and I will most probably make a good team,” and “I can see myself using the teleoperating system with the virtual hands in the future.” The internal consistency of the measure was acceptable (robotic hand: α = 0.93; human-like hand: α = 0.93).

The work performance was operationalized as the number of tasks successfully completed. The number of tasks successfully completed was counted by inputting the code into the Unity software during the experimental task. The program recorded all the successful task completion cases during the experiment. The number of tasks successfully completed theoretically ranged from 0 to 20, as a maximum of 20 trials were conducted per session. The number of tasks successfully completed ranged from 2 to 13 under the robotic hand condition (M = 8.5, SD = 2.93), and 0–13 under the human-like hand condition (M = 7.66, SD = 2.77).

Data analysis

The hypotheses were tested using a two-step process. In the first step, a repeated measures analysis of variance (ANOVA) was conducted to determine the influence of virtual hand representations (robotic hand vs human-like hand) on perceived body ownership (H1). In the second step, partial least squares structural equation modeling (PLS-SEM) was conducted to validate the relationships between perceived body ownership, the risk of danger (i.e., moderating effects on risk perception), risk perception, intention to work using the teleoperator, and work performance (H2–H5). The PLS-SEM is superior to other latent modeling approaches, as it requires a relatively smaller sample size and allows for the evaluation of complex causal path models (Chin, 1998). As an analytical software, WarpPLS 6.0 (Kock, 2017) was used. Given that a repeated measure design is employed for the current research, the difference scores (i.e., human-like hand score – robotic hand score) were computed to validate the significance of paths in the research model. This analytical technique was proposed for the analysis of the path model with a two-condition within-subjects design (Montoya, 2019). In the structural model, sex ([0] = female; [1] = male) was included as a control variable for the control of demographic differences among individuals.

Manipulation check

A manipulation check was conducted to ensure the suitability of the virtual hand representation (i.e., human-likeness) and the risk of danger (i.e., pressing durations of 1 s and 3 s) manipulation. To ensure the proper manipulation of the human-likeness of virtual hands, a single seven-point Likert scale item adapted from the study conducted by Kim and Sundar (2016) was used: “The virtual hands I saw and used in the virtual environment were highly similar to my real hands.” To test the risk of danger, a seven-point-Likert scale item was obtained from a study conducted by Hayes, Perander, Smecko, and Trask (1998): “I felt as though I could be easily injured during the task.” Results from a mixed-design ANOVA revealed that the virtual representation and risk of danger were properly manipulated. Human-like virtual hands (M = 4.73, SE = 0.20) were perceived as more similar to the hands of the participants than the robotic virtual hands (M = 3.63, SE = 0.20), F(1, 72) = 27.48, p < .001, η p 2 = 0.28; and a 1 s pressing duration (M = 4.13, SE = 0.27) was perceived as a higher risk of danger than a 3 s pressing duration (M = 3.36, SE = 0.27), F(1, 72) = 4.08, p < .05, η p 2 = 0.05.

Results

Measurement validity

PLS-SEM was employed for the testing of the validity of the measurement model. In the research model, the risk of danger was the only binary variable ([0] = low risk of danger; [1] = high risk of danger). The remainder of the variables measured for the testing of the research model were reflective indicators. Following the statistical criteria suggested for the validation of a suitable reliability for reflective indicators, the item loadings of the reflective indices were tested to determine whether they were greater than 0.50 with a significance level of p < .001 (Kock, 2017). The results indicated that the measurement model was highly reliable (see Table 1 ). In addition, the internal consistency reliability of all the constructs were acceptable, as they were all above 0.70.

Table 1

Item loadings for reflective indicators.

PBO	Item Loading	RP	Item Loading	IWT	Item Loading	WP	Item Loading
PBO1	.75∗∗∗	RP1	.50∗∗∗	IWT1	.83∗∗∗	WP1	1.00∗∗∗
PBO2	.78∗∗∗	RP2	.64∗∗∗	IWT2	.84∗∗∗
PBO3	.58∗∗∗	RP3	.65∗∗∗	IWT3	.83∗∗∗
PBO4	.83∗∗∗

Note. PBO = Perceived body ownership, RP = Risk perception, IWT= Intention to work using the VR-based teleoperator, WP = Work performance, RoD = Risk of danger.

∗∗∗p < .001.

Hypotheses testing

H1 posited that the use of human-like virtual hands will engender a greater perception of body ownership when compared with the use of robotic virtual hands for the operation of a VR-based teleoperator. Consistent with our prediction, the results from a repeated measures ANOVA revealed that the human-like virtual hands (M = 4.95, SE = 0.16) engendered a greater perception of body ownership than the robotic virtual hands (M = 4.33, SE = 0.16), F(1, 73) = 19.61, p < .001, η p 2 = 0.21. Therefore, H1 was supported.

Given that we used the difference scores method to validate the association among perceived body ownership, risk perception, intention to work using the VR-based teleoperator in the future, and work performances, the statistical significance of the difference in the measured variables were also tested. In alignment with the significant difference found in the perceived body ownership through a repeated measures ANOVA, the difference in other measured variables were also found to be significant: risk perception, F(1, 73) = 15.39, p < .001, η 2 = 0.17; intention to work using the VR-based teleoperator in the future, F(1, 73) = 6.06, p < .05, η 2 = 0.08; and work performances, F(1, 73) = 4.28, p < .05, η 2 = 0.06, respectively.

Thereafter, PLS-SEM was conducted to examine the significance of the causal paths posited in the structural model (H2-H5). H2 hypothesized that a greater perception of body ownership will lead to a greater risk perception in the execution of high-risk tasks using a VR-based teleoperator. The PLS-SEM results indicated that perceived body ownership significantly increased the risk perception during the operation of a VR-based teleoperator (β = .40, p < .05, R 2 = 0.04). The results were in support ofH2.

H3 predicted that the risk of danger will moderate the relationship between perceived body ownership and risk perception, such that the increase in the risk of danger will amplify the effects of perceived body ownership on risk perception. Although the risk of danger was found to significantly moderate the relationship between perceived body ownership and risk perception (β = −.21, p < .05), the relationship between perceived body ownership and risk perception was rather found to be significantly enhanced in the low risk of danger condition (see Fig. 4 ). Therefore, H3 was not supported.

Fig. 4

Moderation Effects of Risk of Danger on the Influence of Perceived body ownership on Risk Perception.

Last, H4 and H5 respectively posited that risk perception will negatively influence the intention to work using the teleoperator in the future and work performances. Consistent with our prediction, the results from the PLS-SEM have shown that risk perception is negatively associated with the intention to work using the VR-based teleoperator in the future (β = −0.50, p < .01, R  = 0.26) and work performance (β = −0.39, p < .01, R  = 0.15). Therefore, H4 and H5 were supported. Fig. 5 shows the holistic view of the PLS-SEM results.

Fig. 5

PLS-SEM results. Note. Solid lines indicate significant paths and dashed lines indicate non-significant paths. ∗p < .05, ∗∗p < .01.

Discussion

In the current study, we examined whether and how the use of human-like virtual hands for the VR-based teleoperation of machinery will negatively impact worker performances under high-risk conditions. We specifically posited that although the use of a human-like virtual hand for the teleoperation will enhance the perceived embodiment (i.e., body ownership), this may have a negative influence on the intention of workers to use the VR-based teleoperator in the future, in addition to their task performances, by the increase in risk perception under high-risk conditions. This conjecture was mainly based on the findings obtained from a study conducted by Argelaguet et al. (2016).

Overall, the results were found to support our predictions. In particular, the results from the mixed factorial design experiment revealed that the rendering of realism into a virtual body for VR-based teleoperation can paradoxically decrease the worker task performances by increasing their risk perceptions under high-risk conditions. In addition, the increase in risk perception, influenced by the perceived body ownership, was found to reduce the intention of workers to use the VR-based teleoperator in the future. These findings suggest that the enhancement of perceived embodiment in VR-based teleoperation may backfire when the risk of danger is demonstrated in a task.

Discussion of results

As predicted, the use of a human-like virtual hand was found to significantly increase the perceived body ownership of the user, when compared with that of a robotic virtual hand for VR-based teleoperation. The findings were in accordance with the proposition postulated by Kilteni et al. (2012), wherein the extent to which the representation of a virtual body approximates the visual appearance of an actual body is predicted to enhance perceived body ownership. Although several studies (e.g., Lugrin et al., 2015; Poliakoff, Beach, Best, Howard, & Gowen, 2013) found that even using realistic virtual hands without avatar faces in VR can decrease perceived body ownership upon the activation of the uncanny valley effects (Mori, 1970), the results of our research suggest that the recent advances in simulation technologies might have at least overcome the limitations of uncanny valley effects in the context of realism in virtual hands. The fact that the results of our study are in good agreement with relatively more recent studies (Argelaguet et al., 2016; Lin & Jörg, 2016) seems to add much to this speculation.

The results further revealed that perceived body ownership, induced by the use of human-like virtual hands in a virtual factory environment, can increase the risk perception of workers in the teleoperation of high-risk metal press machinery. In alignment with our speculation, the enhancement of perceived body ownership, induced by the rendering of human-likeness into virtual hands, led workers to feel unsafe during the teleoperation of high-risk metal press machinery by simulating physical danger in the VE. This is in accordance with the previous findings, in which participants in a VE experienced virtual, yet realistic, threats as more physically threatening upon a greater perception of body ownership (Argelaguet et al., 2016; González-Franco et al., 2014).

Although the results from the moderation analysis revealed that the risk of physical danger significantly moderated the relationship between perceived body ownership and risk perception, the moderation was found to be inverse. More specifically, the risk of danger was found to rather enhance the relationship between perceived body ownership and risk perception in the low-risk condition. However, the relationship between perceived body ownership and risk perception was weakened in the high-risk condition, which was inconsistent with our prediction. One possible explanation for this unexpected result may pertain to the idea that the availability bias (Sunstein & Zeckhauser, 2011), which is found to intervene in the process of judging the risk level of threats under high-risk conditions, influenced the risk perception of workers. According to Sunstein and Zeckhauser (2011), the risk level of threats can be overestimated as the availability bias intervenes in the cognitive process. This research suggests that the availability bias might have led workers under high-risk conditions to overestimate the risk level and made them focus on the threatening tasks rather than their virtual hands during the experiment (i.e., tunnel vision) as fear reduces cognitive flexibility (Dillard & Wilson, 1993). As a result, the relationship between perceived body ownership and risk perception might have been degraded in the high risk of danger condition. Although further investigation is required to ascertain our speculation, it is noteworthy that the participants in our research reported the highest risk perception when human-like virtual hands were used in the high risk of danger condition.

Finally, the increase in risk perception, influenced by perceived body ownership, was found to negatively impact the intention of workers to use the VR-based teleoperator in the future and worker performances in the execution of high-risk tasks. These results ascertained that risk perception can play a pivotal role in negatively affecting the intention to use a technology (Lu et al., 2005; You & Robert, 2018); and risk perception can significantly limit the performance of workers, as fear intervenes in the execution of high-risk tasks (Sharf & Binder, 1983). Such results seem to extend the previous finding in which the enhancement of perceived embodiment by the use of human-like virtual hands increased the risk perception and decreased the accuracy of the pick-and-place task performances (Argelaguet et al., 2016).

However, while these results suggest some important implications by finding that the enhancement of perceived embodiment could backfire depending on the degree to which workers perceive a teleoperation task as dangerous, the applicability of our findings to other contexts should be carefully interpreted. This is because there may be situational contexts in which a greater perception of risk in teleoperation is necessary for preventing the negative physical consequences of workers’ behaviors that could be directed towards the safety of a remote workplace environment. For example, a greater perception of risk might be necessary in the context of a surgical teleoperation since the inadvertent surgical errors resulted from less risk perception might incur severe consequences with respect to the safety of patients (e.g., death). Similarly, a greater perception of risk might be also required in the context of processing explosive materials through teleoperators, because inadvertent errors might incur severe consequences in the workplace environment (e.g., explosion). Nonetheless, it should also be taken into account that an excessive perception of risks might have a negative impact on the speed of task performance, while the speed of task performance is important for improving the effectiveness (i.e., fast surgical operation for emergency cases) and efficiency (i.e., industrial productivity) of works in such examples as well. Given the idea, our findings seem to be applicable to the teleoperation context in which the mitigation of risk perception is required for the fast completion of high-risk tasks.

Implications of results

Theoretically, our research seems to advance the current scholarship with respect to the role of realism in new media technologies by providing a new perspective. While rendering realism into media technologies has long been premised to engender positive effects on user perceptions, recent studies seem to contradict the premise by finding that realism could backfire depending on situational contexts (e.g., Shin, Song, & Chock, 2019). Our findings are also in line with the studies and, therefore, advances the recent scholarship by providing a new perspective on the role of realism in a VE. Simultaneously, the current study also provides a new perspective on the potential role of embodiment in VEs.

Another theoretical implication of our study pertains to the fact that we extended the investigation of the role of embodiment to the realm of actual behaviors and performances beyond the measurement of user perceptions in a VE. Although a considerable amount of research has been conducted to examine the perceptual effects of certain design factors on the sense of embodiment in a VE (Haans et al., 2008; Lin & Jörg, 2016; Maselli & Slater, 2013), it merits notice that only few studies (e.g., Argelaguet et al., 2016) have explored whether and how the induced sense of embodiment will subsequently extend to actual behaviors or performances. In light of this fact, our research contributes to advancing the current scholarship not only by supporting the relationship between virtual body representation (i.e., realism) and the sense of embodiment in a VE, but also by adding the empirical evidence that the sense of embodiment, induced by realism in virtual body representation, can extend to actual user behaviors and performances. Moreover, this study extends the implications of the earlier findings to a real-life context (i.e., VR-based teleoperation), and thereby contributes to the partial understanding of the direct link between perceived embodiment and task performances in the context of VR-based teleoperation, which has yet to be empirically addressed by researchers (Toet et al., 2020). Our work is expected to work as a guideline for future research on embodied teleoperation.

Our research also seems to provide practical implications to the current teleoperation industry by suggesting that rendering realism in VR-based teleoperators may not always be essential to induce positive user perceptions and behaviors. Although rendering realism into immersive media technologies has often been considered an expensive but advantageous work that could engender positive consequences, our study showed that reducing realism in a VR-based teleoperator could be more effective for improving work performances in the execution of high-risk tasks. In this sense, our findings, in which rendering a reduced level of realism into a VR-based teleoperator induced a greater perceived safety of work and a better work performance, seem to directly provide insights into the design of VR-based teleoperators in the current industry.

In addition, our finding with respect to the mitigated risk perception induced by the use of robotic virtual hands has the potential to provide hints important for developing effective healthcare and training programs in other domains (e.g., phobia treatment, military training). In our study, we found that the reduced sense of embodiment, rendered through replacing a realistic body image with a robotic image (i.e., robotic virtual hands), can mitigate the risk perception of embodied workers. This finding implies that reducing the sense of embodiment using less realistic or distorted representation of a virtual body could be effective for moderating embodied users’ negative affective responses (e.g., fear, anxiety) towards negative stimuli simulated in a VE. In line with this idea, rendering a distorted virtual body image in VEs (i.e., reducing or enlarging the size of a virtual body area) was found to reduce perceived pain of patients in the domain of healthcare (e.g., Mancini et al., 2011; Moseley, 2008).

Potentially, this strategy using the VR simulation technology could be conjugated as means for gradually reducing the level of fear or anxiety among patients with phobia or military soldiers that need to familiarize themselves with fear- or anxiety-evoking stimuli. Without reducing the sense of embodiment in such cases, the availability bias (i.e., people's tendency to overestimate the risk level in high-risk situations) might activate the defensive mechanism and subsequently lead them to withdraw from familiarizing themselves with the negative stimuli. In sum, this finding is expected to provide important hints for designing effective VR healthcare and training programs.

Limitations and future directions

There are several limitations to this study. First, while the present study deliberatively employed a mixed factorial design to minimize and control for the individual differences among participants, a repeated measures design is known to be susceptible to carryover effects (Montoya, 2019). Although the fact that we randomized the order of the within-subjects factor (i.e., virtual hand representation) seems to mitigate the issue of carryover effects, it would be interesting to see if employing a between-subjects design will replicate our findings.

Second, the results of our study with respect to the intention to work with the VR-based teleoperator should not be overgeneralized in light of the fact that student samples were used for the experiment. While participants in our experiment were provided with a scenario in which they were asked to put themselves into the shoes of factory workers who operate machinery using a VR-based teleoperator at a distance, this manipulation would not have been sufficient to generalize our findings to the population of real workers in the current manufacturing industry. Given that the context of this study relates to teleoperation in the manufacturing industry, future studies should be conducted on actual workers in the manufacturing industry to ascertain the generalizability of our findings.

In addition, future studies should consider using different types of high-risk tasks (e.g., more complex and highly risky tasks) in VEs for the validation of whether the findings of this research will remain the same when workers are asked to repeatedly operate the machinery for a longer period of time. While little is known about the longitudinal impact of embodiment (Toet et al., 2020), there is some potential that the relationship between the measured variables might be weakened as workers familiarize themselves with high-risk tasks over time. In this study, participants were assigned a simple task (i.e., pick-and-place task) through which they were able to easily familiarize themselves with such a high-risk, yet simple, task. Therefore, the results of our study based on such a simple task are not expected to change over time. However, it should be taken into consideration that the results might unfold in a different way when more complex or difficult tasks are assigned to workers. Future studies should test if the relationship between perceived embodiment and risk perception will change over time to add much to the generalizability of our findings.

Finally, while this study found that the increase in risk perception can negatively influence the task performance of workers when the work performance is operationalized as the number of tasks successfully completed within a limited time (i.e., an objective indicator of the effectiveness of rendering perceived embodiment in a VR-based teleoperation system), further research should be carried out to provide a clear understanding of the role of risk perception on the various aspects of work performance, namely the accuracy of task performance. Worthy of note, when the risk perception starts to make workers cautious in handling high-risk tasks, the slower task performance resulted from the risk perception (i.e., less number of tasks successfully completed) might be interpreted as less negative considering that the risk perception may potentially increase the accuracy of task performance. In such cases, the accuracy of task performance could partially compensate for the negative outcomes of slower task performance. In light of this fact, our findings in terms of the relationship between risk perception and work performance should not be overgeneralized without taking into account the role of risk perception on the accuracy of task performance.

To further explicate the relationship between embodiment, risk perception, and task performance in the context of teleoperation, future studies should investigate whether there will be a trade-off relationship between the speed and accuracy of task performance within our study context. Other object indicators such as the number of task failure and task completion or response time, in addition to the number of tasks successfully completed within a limited time, could be measured to provide more detailed insights into the understanding of the role of risk perception on the balance between the speed and accuracy of task performance, which is important for improving the work productivity. Nonetheless, the findings of Argelaguet et al. (2016), in which the enhancement of perceived embodiment in a VE decreased the accuracy of task performances under high-risk conditions, at least provide some evidence for future studies to predict that the increase in risk perceptions, influenced by the sense of embodiment in VR-based teleoperation, may also decrease the accuracy of task performances as well as the speed of task performances.

Conclusions

In this research, we found that rendering realism in VR-based teleoperators may backfire when the risk of danger is apparent in a task. The findings of this study suggest that researchers should more carefully predict the role of realism in VEs to prevent the degradation of the effectiveness of VR-based technologies. Worthy of note, the recent outbreak of COVID-19 has significantly increased the need for teleoperation. Although this research may not contribute considerably to the solution of the current global crisis, we hope that the findings of this study could serve as a basis for the design of future work environments, thus resulting in increased industrial productivity.

Credit author statement

Mincheol Shin: Conceptualization, Methodology, Formal Analysis, Writing – Original Draft, Writing – Review & Editing, Sanguk Lee: Investigation, Writing – Original Draft, Stephen Wonchul Song: Investigation, Writing – Original Draft, Donghun Chung: Conceptualization, Writing – Original Draft, Writing - Review & Editing, Supervision, Project Administration, Funding Acquisition.