A Human-Robot Interaction Perspective on Assistive and Rehabilitation Robotics.

Assistive and rehabilitation devices are a promising and challenging field of recent robotics research. Motivated by societal needs such as aging populations, such devices can support motor functionality and subject training. The design, control, sensing, and assessment of the devices become more sophisticated due to a human in the loop. This paper gives a human-robot interaction perspective on current issues and opportunities in the field. On the topic of control and machine learning, approaches that support but do not distract subjects are reviewed. Options to provide sensory user feedback that are currently missing from robotic devices are outlined. Parallels between device acceptance and affective computing are made. Furthermore, requirements for functional assessment protocols that relate to real-world tasks are discussed. In all topic areas, the design of human-oriented frameworks and methods is dominated by challenges related to the close interaction between the human and robotic device. This paper discusses the aforementioned aspects in order to open up new perspectives for future robotic solutions.

Introduction

Recently, an increasing research interest in assistive and rehabilitation robotics is observed as a result of their greater capabilities in supporting and augmenting users. This stems from societal needs such as applications in healthcare, for example, mobility aids for people who are aging or with motor impairments (Dollar and Herr, 2008; Windrich et al., 2016), or in industry, such as the augmentation of workers carrying heavy loads (Dollar and Herr, 2008; Yan et al., 2015). Even though device designs have the functionality to perform desired tasks, many robotic devices demonstrate limited effectiveness not only due to technical limitations (Dollar and Herr, 2008; Yan et al., 2015; Windrich et al., 2016) but also due to insufficient knowledge about the human (Yan et al., 2015). Thus, assistive and rehabilitation robotics research and applications appear to require human-oriented approaches, since critically the devices interface with humans (Yan et al., 2015; Christ and Beckerle, 2016). Therefore, considering both technical and human aspects is crucial, and techniques from neural and human sciences should be considered beyond engineering techniques (Yan et al., 2015; Christ and Beckerle, 2016). Taking into account safety, functionality, effectiveness, and acceptance requires the collaboration of disciplines like mechatronics, computer science, biomechanics, neuroscience, and psychology. A focus on aspects of human–robot interaction (HRI) and interface technologies is needed, not the least in order to promote a pathway to systematic frameworks that consider, in both development and operation, the challenges of interfacing with the human.

In this perspective paper, the organizers and speakers of the 2nd Workshop on Human-Oriented Approaches for Assistive and Rehabilitation Robotics (HUMORARR 2016) reflect and extend their discussions to point out directions for future research. The workshop comprised talks and discussion from experts working on assistive/rehabilitation robotics and/or human–robot interaction and was held in conjunction with the IEEE International Symposium on Robot and Human Interactive Communication (IEEE RO-MAN 2016) in New York, USA, on August 26, 2016.

This perspective covers four topics: control design, machine learning, sensory feedback, and affective computing, as well as reliability and assessment. Scientific issues determined in the workshop are formulated, and their relevance and potential are discussed.

Control Design (Matei Ciocarlie, Brenna D. Argall, Fulvio Mastrogiovanni, Claudio Castellini, Heni Ben Amor, and Philipp Beckerle)

One goal for assistive robotic devices is to not only “fit like a glove” but also to be just as easy to use as such a simple garment: the user simply dons the device, which starts doing its job without demanding additional cognitive load from the human. However, many existing assistive machines are burdensome to operate, or use input modalities that are easily accessible to non-impaired people but difficult for their target population.

Reducing (or maybe even eliminating) training time can speed up adoption and allow valuable therapist time to be used more effectively. One way to achieve this is to employ learning methods that incrementally adapt to the subject, situation, and environment (Castellini et al., 2015). Another possibility is to introduce shared autonomy to traditionally human-operated assistive machines (Argall, 2014, 2015; Jain et al., 2015) since semiautonomous operation can off-load cognitive and physical load from subjects to the machine and thereby increase user independence. Furthermore, the literature suggests the use of force and tactile information (Dahiya et al., 2010) for contact modeling and recognition as well as for motion analysis and planning, since these robotic devices will be in contact with the user (Cannata et al., 2010; Menguc et al., 2014). Therefore, perceptual and cognitive factors such as the classification of large-scale contacts (Muscari et al., 2013) or non-verbal communication signals should be taken into account.

A key challenge in semiautonomous solutions is how to appropriately share control between the robotic device and the human: the autonomous behavior should be predictable, providing assistance when appropriate and yet never taking control when undesired by the user (with the exception being cases of safety) (Argall, 2014; Erdogan and Argall, 2017; Gopinath et al., 2017). Furthermore, it is challenging how to adapt this control sharing over time, as the human’s abilities and preferences change.

The input provided by the human user presents significant challenges in itself: the human users and the robotic autonomy often do not provide control signals in the same space (Broad and Argall, 2016), and decoding user intent is a formidable task, especially if the output space is high dimensional. However, new advances in sensing (touch detectors, electromyographic sensors strain gages, etc.) and processing tools (dimensionality reduction, regression and classification, pattern recognition, etc.) are bringing us closer to that goal (Castellini et al., 2014).

An example of rapidly evolving technologies that could help in the design of appropriate human–robot interaction are tactile sensing and robot skins, which could acquire contact information from large-scale surfaces (Muscari et al., 2013; Denei et al., 2015; Youssefi et al., 2015a,b). Such robot skins need to be conformable, cheap, and easy to manufacture (Anghinolfi et al., 2013; Bianchi et al., 2016a; Le et al., 2016). Furthermore, tactile-based sensing and control cannot be seen as isolated features of novel rehabilitation devices. On the contrary, the way contact information is processed and the associated robot motion strategies must be informed by top-down rehabilitation or assistive policies. For example, given a contact regime, robot motions could be compliant in certain conditions, whereas they may be such to exert forces to implement a given rehabilitation tasks on other situations. Sensor skins meeting these requirements will enable robotic devices to physically interact with humans according to high-level policies and goals. Beyond obvious benefits that robot skin can provide to rehabilitation devices, it is noteworthy that they can also serve as an open benchmark to investigate cognition processes associated with non-verbal communication. These are related to the understanding of affective gestures such as constrained motions and are of the utmost importance in carrying out exercise routines in the most useful way.

Machine Learning (Claudio Castellini, Satoshi Endo, Sandra Hirche, Heni Ben Amor, and Philipp Beckerle)

Whenever confronted with a new tool, human beings learn to use it and adapt to it. Through neural plasticity and behavioral adaptation, exercise quantity and regularity can influence movement capabilities and alter the requirements for assistance or rehabilitation from a robotic device. Thus, robotic devices might guide users to carry out a desired exercise routine that is continually adapted to account for the individual user’s needs, in order to stimulate training and maximize its effect.

Recent literature in assistive robotics and prosthetics tries to explore and quantify this phenomenon. For instance, myoelectric signals are observed to distinctly change over days and weeks when used by subjects to control a hand prosthesis (Powell and Thakor, 2013). Novel muscle strategies are elicited by engaging people in a video game controlled via surface electromyography (Ison and Artemiadis, 2015). When such strategies are retained after weeks and months, this adaptation can be highly beneficial for control strategies derived via machine learning techniques, since signals become more separated and repeatable (Bunderson and Kuiken, 2012).

As human users and control algorithms adapt, exploiting potential co-adaptation to achieve mutual interaction is a promising research challenge. Besides observing and quantifying the phenomenon, considering it in algorithm design could promote a faster and more complete restoration of lost functions. One way of exploiting co-adaptation is to add to the training data synthetic patterns of activation based upon prior knowledge; even if such patterns do not match the current subject’s activations, the subject itself will adapt to them (Nowak and Castellini, 2016).

Mutual interaction is also a promising advancement for robot-assisted exercises: through physiotherapy training or progression of motor deficits, co-adapting interaction strategies that predict user behavior based on data-driven models are needed. Deep learning, for example, has recently gained much interest in robotics applications due to its expressive power and capability to predict overt human behavior (Hartford et al., 2016). Yet, other stochastic modeling techniques such as Gaussian processes can flexibly be designed to additionally address underlying sensory-motor functions important for neurorehabilitation such as neuromechanics of the human motion (Medina et al., 2016).

Sensory Feedback and Affective Computing (Matteo Bianchi, Claudio Castellini, and Philipp Beckerle)

For the effectiveness of robotics-enabled aides, sensory feedback can be as important as actuation, and especially haptic feedback due to the importance of touch in everyday life. In prosthetics, for instance, the natural action-perception loop is interrupted and the resulting lack of sensory information from the external environment is probably one of the main causes of device abandonment (Biddiss and Chau, 2007). However, trying to reproduce the richness of haptic information through artificial human–robot interfaces is a daunting task. Although studies exist on uninjured subjects which confirm that detailed multipoint feedback may help (Patel et al., 2016), the simultaneous delivery of too much information in general does not increase task execution performance. Especially whenever visual feedback is present, non-visual feedback is usually disregarded, and in extreme cases the simultaneous delivery of multiple types of information could even degrade performance (Kim and Colgate, 2012). Generally, appropriate feedback is the basis for successful multisensory integration that can facilitate embodiment of the device by the user (Giummarra et al., 2008; Christ et al., 2012a).

A possible strategy to tackle this issue could be the theoretical framework of sensory synergies (Bicchi et al., 2011). From the sensing point of view, we can identify an analogous mapping between low-level sensory variables (e.g., mechanoreceptors) to stabilize high-level human percepts, similar to motor synergies (Latash, 2008; Santello et al., 2016). Thereby, one might exploit that humans integrate data from multiple sensors to produce a coherent perceptual representation. Synergy approaches can be considered reductionist, since it aims at moving from the biological patterns underpinning human perception toward mathematical models. This aims at explaining how our brain produces low-dimensional perceptual representations from the abundance of sensors distributed in our body (Hayward, 2011). Therefore, computational approaches such as the tactile flow model that considers the flow of strain energy density can be applied (Bicchi et al., 2008). Understanding and mathematically modeling these sensory mechanisms could help to characterize mappings between cutaneous stimulation, motor acts, and proprioception (Moscatelli et al., 2016). Such a representation scheme could help to identify the most useful information for users for task accomplishment and to develop simple, effective, and intuitive interfaces for human–robot interaction (Bianchi and Serio, 2015). This is particularly important in assistive and rehabilitation robotics, where engineers need to know which information is most important for task accomplishment and how this relates to sensing and feedback.

Another important but scarcely considered aspect is the emotional response that assistive and rehabilitation robots can elicit in users, especially if feedback is provided. In the study by Bianchi et al. (2016c), a general framework to evaluate the emotional counterpart of haptic stimuli is developed. This framework is based on the circumplex model of affect (CMA) (Posner et al., 2005) and shows that both discriminative and affective haptic systems are able to elicit emotional responses that correlate with stimulus parameters.

Emotional quantification through CMA can also be associated with physiological measures related to the autonomous nervous system (ANS), which is directly connected to evoked human emotions (Bianchi et al., 2016b). Possible measures are electroencephalographic activity, heart rate variability, respiration dynamics, or electrodermal response (Schmidt et al., 2013; Valenza et al., 2016). The acquired data might feed new design approaches for robotic devices (Christ et al., 2012b; Beckerle, 2014) and, likely, increase user acceptance. Furthermore, considering ANS data in device control could enable estimating subtler but important aspects such as stress, fatigue, comfort, and motivation. Using such estimates, natural transfers between human and artificial control in shared control applications could be reached (Bianchi et al., 2016b; Gopinath et al., 2017). To calibrate such techniques and verify the effectiveness of the proposed methods, human-in-the-loop techniques are promising (Beckerle et al., 2012; Caspar et al., 2015) in regard to the embodiment of devices.

Reliability and Assessment (Gionata Salvietti, Domenico Prattichizzo, Simone Rossi, Claudio Castellini, and Philipp Beckerle)

Standards to evaluate the assistive and rehabilitative facets of novel robotic devices are still lacking. This is due mainly to the complexity of assessing user improvement that is attributable to the robot (Lo et al., 2010). Despite the availability of methods for assessing user improvements during rehabilitation (Gresham et al., 1997), standard protocols that go beyond usability questionnaires are missing for assistive devices: the gold standard for functional assessment would be a quantitative comparison of the gain reached in certain activities with and without the device. Such an assessment is, however, a complex task due to the sheer variety of assistive devices, as reviewed in the studies by Dollar and Herr (2008), Yan et al. (2015), and Windrich et al. (2016).

Toward the definition of standard assessment procedures, future approaches could adapt classical rehabilitation tests to examine assistive devices in the early stages of, or after, intense treatment. As an example, the Frenchay Arm Test (Heller et al., 1987) is used to quantify the number of successful actions with assistance (Salvietti et al., 2016). Alternatively, task or action sequences to be performed with the device could be specified. For instance, a set of bimanual tasks is used to investigate device performance in activities of daily living (Hussain et al., 2016). Regarding the lower limbs, aspects such as biomechanical functionality or metabolic cost seem to be promising (Au et al., 2009).

Although machine learning seems to be an effective method for control (Castellini et al., 2014), its reliability is still arguable (Jiang et al., 2012): it can work perfectly in the laboratory, but then largely fail in real-world settings. Regarding algorithmic reliability, incremental learning (Castellini, 2016) and shared autonomy might lead to a new generation of devices that interact with the human subject. Hence, future functional assessment protocols will need to take incrementality and interactivity into account (Castellini et al., 2015).

Finally, improving wearability would help to increase the use of devices in unstructured environments, and thereby support rehabilitation by extensive use in users’ homes. Passive and intrinsically compliant devices could foster wearability by combining safety, simplicity, and robustness with an excellent power-to-weight ratio (Wehner et al., 2013; Cappello et al., 2016). Furthermore, the device kinematics might differ from human ones; for example, this could relate to a robotic extra-finger that cooperates with the paretic arm of chronic stroke users to achieve a stable grasp and safe interaction in everyday tasks (Hussain et al., 2016; Salvietti et al., 2016).

Conclusion

Human–robot interaction plays an essential role in assistive and rehabilitation robotics. We have presented our perspective on the potential of, and also challenges within, this domain, from the standpoints of control, machine learning, sensory feedback, and affective computing, as well as reliability and assessment methods.

Regarding control, a key challenge is to design shared control or machine learning techniques that are predictable for users and do not override their demands. A grand topic in machine learning itself is mutual adaptation that allows users to explore and train the robotic aides themselves. Sensory feedback is a possibility to close human–machine control loops and could rely on models of basic sensory dimensions. Evaluation metrics going beyond questionnaires are scarce and functional assessment protocols that consider real-world task complexity and training progress are required, especially for learning devices. To tackle human–robot interaction systematically in design, human-oriented methods and human-in-the-loop experiments are promising topics for future research.

Author Contributions

PB coordinated the writing process, the development of the paper structure, and integration of individual contributions. GS and RU supported this regarding the whole paper. All the authors contributed to the discussion in the perspective subsections, their names are mentioned in brackets after the subsection titles.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.