An analysis of design recommendations for socially assistive robot helpers for effective human-robot interactions in senior care.

As the global population ages, there is an increase in demand for assistive technologies that can alleviate the stresses on healthcare systems. The growing field of socially assistive robotics (SARs) offers unique solutions that are interactive, engaging, and adaptable to different users' needs. Crucial to having positive human-robot interaction (HRI) experiences in senior care settings is the overall design of the robot, considering the unique challenges and opportunities that come with novice users. This paper presents a novel study that explores the effect of SAR design on HRI in senior care through a results-oriented analysis of the literature. We provide key design recommendations to ensure inclusion for a diverse set of users. Open challenges of considering user preferences during design, creating adaptive behaviors, and developing intelligent autonomy are discussed in detail. SAR features of appearance and interaction mode along with SAR frameworks for perception and intelligence are explored to evaluate individual developments using metrics such as trust, acceptance, and intent to use. Drawing from a diverse set of features, SAR frameworks, and HRI studies, the discussion highlights robot characteristics of greatest influence in promoting wellbeing and aging-in-place of older adults and generates design recommendations that are important for future development.

Introduction

By 2050 the population of adults 60 years of age and older is expected to double to 2.1 billion, and those 80 and older to triple. As life expectancy increases, there is a greater prevalence of health-related issues and a greater number of seniors are needing to transition to living in long-term care (LTC) homes, which provide 24-hour onsite professional care to support their physical and cognitive needs. Consequentially, the demand for an already dwindling healthcare system is expected to grow substantially with an estimated caregiver shortage of more than 100,000 workers in the US alone by 2030. This lack of staffing combined with a new environment can lead to older adults feeling isolated from social circles, often worsening existing conditions such as dementia. There exists an urgent need for innovation in the care of older adults to improve quality of life and overall wellbeing, and to help address the strain on our labor force, and the various needs of a diverse aging population. Solutions must be multifaceted, adaptive, and sustainable, and need to be supported by government policies and programs that also consider socioeconomic factors that affect health to meet both urgent and future senior care needs.

Assistive technologies offer opportunities to improve care with respect to assurance, compensation, and assessment. In addition to challenges associated with cognitive and physical decline of older adults, assistive solutions must be accessible for senior users who are often novices in newer forms of technology. Socially assistive robots (SARs) are a unique type of robotic technology that use social communication modes to engage with people. SARs have the potential to aid caregivers by providing safety assurance through monitoring, assisting older adults in the completion of activities of daily living (ADLs) such as eating, and assessing changes in physical and cognitive abilities over time all in a single multi-facet technology; that may decrease interest in standalone solutions such as reminder systems or fall monitoring pendants, and alleviate caregiver burden. The unique ability of SARs to adapt their behaviors to older adults can help support their individual needs and preferences as they age. This adaptability combined with the potential efficiency of SARs to help advocate for appropriate caregivers-to-older-adults ratios in LTC homes aligns with policies aimed at meeting the health and social needs of older adults created by a shortage of caregivers in order to provide personalized quality care.

Crucial to having positive human-robot interaction (HRI) experiences in senior care settings is the overall design of the robot, considering the unique challenges and opportunities that come with novice users in specific task and environment contexts. Findings in senior care have shown a diverse set of SAR features responsible for varying HRI outcomes, presenting a challenge in determining their relative importance to different users, activities, and scenarios. While these individual studies have advanced the design of SARs, there still lacks an overall comprehensive investigation and detailed evaluation that accumulates these findings from a robot design perspective to analyze trends and provide recommendations targeting successful SAR development.

This paper presents a novel and holistic analysis of current literature through a results-oriented framework that considers crucial aspects such as trust, adherence, intent to use, and acceptance in identifying the key design features and frameworks for HRI and their influence on senior care. Multiple robots, objectives, and social approaches are examined to gather a diverse set of interactions and user experiences. The goal of this paper is to generate design recommendations applicable to the diverse uses of SARs. Considering the unique needs of seniors as they age, especially their perceptions and expectations of robots, results in distinct design characteristics that are important to future deployment and long-term use.

Methodology

To determine design features that are critical for successful long-term deployment of SARs with older adults, we conducted a mixed systematic-integrative review of existing literature to address the following research questions: 1) what appearance features and interaction modes have been used by SARs? 2) what SAR frameworks have been developed for user awareness and behavior adaptation? and 3) what is the impact of these SAR design features on HRI?

The first stage of review was completed using a meta-search engine including scientific databases such as IEEE Explore, PubMed, and Scopus. Keywords included: older adults, socially assistive robot, elderly, robot behavior, and long-term care, and led to over 100 scholarly articles. The second stage used the following inclusion criteria: 1) robots that directly provide social and cognitive assistance through direct interaction with the robot itself, eliminating SARs with telepresence as their main function, 2) HRI studies that incorporate the aforementioned design features and assess users’ experiences and perceptions. Results were used to conduct a comparative analysis in a results-oriented framework to investigate the effect of SAR design on HRI with older adults in a variety of different contexts.

Open challenges

There are several open challenges to designing SARs as effective long-term assistants for older adults. Herein, we identify these challenges, whereas the design discussions that follow address the limitations of existing SARs and provides valuable suggestions for future research.

SAR physical features and interaction modes

Individual preferences need to be considered in the design of SARs, while maintaining feasibility for mass production and deployment. SARs can range in such appearance characteristics as human-likeness, size, expressiveness, and material composition. Interaction modes include social interactions via verbal and non-verbal communication, gestures, displays, and physical touch. A mixed-methods approach in used a combination of questionnaires, interviews, and focus groups to determine the older adult expectations towards a hypothetical SAR for everyday assistance. Questions on preferred height, exterior finish, and favorite overall appearance (from a list) showed no one option was most preferred. Additionally, older adults expressed their desire to understand the functionality of the SAR before forming an opinion on its appearance.

Accommodating preferences is important in HRI as personalization increases engagement and enjoyment, having a positive impact on overall use by older adults. Existing SAR designs for older adults have shown significant differences in appearance and interaction modes even when robots perform the same task [9,17,18] suggesting diverse expectations have challenged SAR developers to optimize robot design. In accommodating such preferences there is a risk in underrepresenting the diversity of users including of racial, cultural, gender, and age minorities, which can create inherent biases, i.e. similar to some medical voice dictation systems being more accurate for men than women. The complexity of this challenge is increased by the changes in behavioral and attitudinal response when comparing those that directly engage in HRI with physically embodied robots to those who are asked their opinions on images or videos of SARs. Researchers must determine which features are important based on user abilities and interaction context, while ensuring SAR accessibility to a broad and diverse userbase of older adults.

The Need for SAR Adaptable Behaviors

SAR behaviors can encompass varying strategies from emotional to persuasive, while also considering social norms to engage with older adults. An open challenge exists to adapt robot behavioral strategies to achieve the expectations of older adults and gain user trust and adherence. Focus groups of older adults in were presented with an imaginary scenario that put in conflict adherence to SAR recommendations to promote independence and older adult autonomy in disobeying the SAR suggestions. The study highlighted the expectations for SARs to have adaptive behaviors that consider user emotions and engagement as well as long-term user patterns in schedules and moods. Designing behaviors to consider social norms presents challenges in determining which cultures to consider. Caution must be taken to avoid ageist views of selected norms. Beyond potential demographic biases, ethical concerns in developing SAR behaviors for older adults include: 1) privacy over recording user data to influence behavioral adaptation, 2) transparency of SAR intent, and 3) user autonomy in situations where a SAR attempts persuasion.

Development of adaptive behaviors requires training of learning methods to : 1) detect and classify user state, 2) determine an appropriate SAR behavior, and 3) learn from user responses. Advancements in AI including machine and deep learning methods can improve the robustness of SAR behavior adaptation frameworks to changes in older adult behaviors overtime, which may occur due to cognitive decline. Recently deployed SARs with adaptive behavior frameworks are yet to offer a holistic solution that both synthesizes a wide range of available data on the user state and applies this data to modify behavioral strategies accordingly. The combination of inter-group and intra-user variability requires SAR behavior adaption that considers user preferences and cognitive changes while maintaining reliable task performance.

Intelligent autonomy

Current deployments of SARs in senior care vary in their control architectures from teleoperation scenarios, where a human operator (visible or non-visible to the users) must be present, to full autonomy, where a robot is capable of HRI without expert human intervention. For long-term use, autonomy is the only sustainable option and to be achieved SAR architectures need to directly incorporate user(s), robot, and task environment information. For older adults, cognitive decline can decrease their ability to express thoughts using typical sentence structures or facial expressions, limiting the use of standard natural language processing (NLP) and facial expression detection methods for interpreting user state. To account for the inexperience of older adults with robotics, SARs must provide alternate means of maintaining core functionality in the presence of hardware failures such as leveraging multimodal interaction modes using sensor fusion techniques.

Although historically robots in manufacturing only needed to be proficient in a single repeated task in a structured environment, autonomy for SARs in senior care is further complicated by the multiple tasks older adults expect them to reliably perform [34,35] in diverse environments from kitchens to bedrooms in private homes to common dining and recreational rooms in LTC. SARs applications need to handle high environment variability and learn to adapt to their users’ abilities and needs, while dealing with sensing uncertainty or unpredictable human behavior.

Design features

The main design features to consider when developing SARs for older adults are: 1) overall robot appearance, and 2) interaction modes. We discuss each feature within the context of promoting effective social HRI and improving health and wellbeing outcomes while aging.

SAR appearance

In general, older adults have specific, yet varying, preferences for the appearance of SARs which aid in increasing trust, perceived competence, and acceptance of these robots. These attributes can be classified as human-likeness, expressiveness, size, and material composition.

Human-likeness

The appearance of a SAR may be classified as: 1) human-like, 2) character-like, 3) machine-like, or 4) animal-like, depending on the body and face features. Human-like robots have similar human facial features including eyes, eyebrows, a nose, and a mouth, and body features including a torso and two arms; Character-like robots have rounded heads and bodies, with minimal features, such as a face with only eyes. Machine-like consists of varying heads and body shapes ranging from square to rectangular with components including parts and linkages exposed; and Animal-like robots have shapes resembling those of the animals they mimic with many possessing fur.

An example of a human-like SAR from the waist-up is Brian 2.1 which has a torso with a waist and two arms to promote familiarity and a silicone face with two eyes, eyebrows, a mouth, and a nose that can deform to display facial expressions. Brian has been used to assist older adults in LTC with cognitive interventions including memory games and meal eating. Milo R25 is a human-like SAR similar in appearance to a small child having an elastic frubber (foam + rubber) face with two eyes, eyebrows, a mouth, and a nose. Milo R25 has been used to provide conversation therapy to older adults living with Alzheimer’s disease. Alice, an older version of Milo R25 was deployed in aging-in-place to support older adults with depression.

Character-like SARs with a combination of a head and arms include 1) Pepper, Casper, ARI, Stevie, Bandit, NAO, and Mini which all have a rounded face with eyes and a mouth and a torso with two arms, and 2) Hobbit a one-armed robot with a head consisting of only eyes. Character-like SARs with a head but without arms include Pearl and iCat (mouth and eyes), and Kompai and Max (eyes but no mouth). Some applications of character-like SARs are ADL assistance such as Casper for meal assistance ; cognitive stimulating games with Stevie ; monitoring for falls and providing calendar reminders using Max ; and exercise facilitation with NAO,[46,55] and Bandit.

Tangy is an example of a machine-like robot due to its square face and torso, and its visibly exposed cables. Tangy has been used to facilitate group-based cognitive interventions like Bingo and Trivia. Baxter, used for exercise, is also machine-like with its large frame, square head and exposed cables. Companion robots such as the popular seal-like PARO and cat-like JoyForAll Cat are animal-like as they resemble real life animals in shape and texture and are used for older adult pet-therapy to address loneliness and depression.

Expressiveness

Focusing on non-verbal visual expressiveness through embodiment, SARs may be classified using any combination of: 1) gaze direction, 2) facial expressions, 3) gestures, and 4) head and whole-body poses. Hobbit uses head pan and tilt rotations to adjust its gaze direction. Max displays both gaze direction and facial expressions through its LCD eyes by changing eye direction, color, and shape. iCat actuates its head, mouth, eyes, and eyebrows for gaze direction and facial expressions. Pearl is able to blink its eyes. PARO and JoyForAll Cat can blink and use head and whole-body movements to show emotions. Mini and Stevie have gaze direction, facial expressions using animated eyes, and head and whole-body movements. Other robots including Brian 2.1, Milo R25, Pepper, Casper, ARI, Bandit, NAO, Tangy, and Baxter use all four types of visual expression for a variety of tasks such as exercise and games to promote user engagement.

Size

SARs can be classified by three different height ranges: 1) small-size (<100 cm), 2) mid-size (100–125 cm), and 3) large-size (125–170 cm). Small-size SARs include desktop robots NAO, Mini, Milo R25, and iCat in addition to companion robots PARO and JoyForAll Cat, the latter of which are similar in size to the animals they resemble. Pepper, Casper, Pearl, Kompai, Max, Bandit (when on a mobile base), and Hobbit are all mid-sized SARs and have been deployed in a wide variety of interactions. Large-size SARs are ARI, Brian 2.1, Tangy, Baxter, and Stevie which are near the average female height of 165 cm, are all deployed in LTC for cognitive interventions, ADL assistance, and games.[44,56]

Material Composition

The materials used to develop the robot’s outer-shell/casing include: 1) hard plastic, 2) metal, or 3) soft materials. All types of shells are used to prevent robot damage from external factors. SARs with hard plastic shells are Pepper, Casper, ARI, Stevie, Pearl, Kompai, Max, Hobbit, Bandit, Baxter, iCat, and NAO. Tangy has an aluminum structure which suits its machine-like appearance. Soft materials include silicone for Brian 2.1 and a custom formed frubber for Milo R25 to emulate artificial skin. Fabrics including artificial fur on Mini’s torso and on the outer layers of PARO and JoyForAll Cat customize appearance and texture, and promote physical touch.

Interaction Modes

Interaction modes describe the interfaces SARs use to communicate with older adults including speech, sounds, visual displays, gestures, and physical touch.

Speech

Speech is important for SARs interacting with older adults as it provides them a familiar and intuitive form of bidirectional communication. SARs may be classified based on their capability to: 1) speak, 2) detect spoken keywords, and 3) detect word associations (sentences). Some SARs can only speak such as iCat, Tangy, and Bandit. Other SARs that speak also recognize certain keywords to initiate, pause, or end tasks such as Pearl, Kompai,[51,68] Max, Mini, and Hobbit. SARs capable of both speech synthesis and recognition include Pepper and NAO with their built in NAOqi Natural Language Processing (NLP), Brian 2.1 using Julius, , and Casper using IBM’s Watson. , ARI and Stevie have built in speech modules yet to be implemented in HRI studies with older adults. To-date, acoustic models and training data for NLP specific to older adults is limited and standard available NLP software is less accurate due to differences in voice acoustics, cognitive function, and sentence structures for this user group.

Sounds

SARs use sounds proactively or reactively to express robot states such as sleep, wakefulness, or excitement to increase engagement. PARO and JoyForAll Cat make sounds such as cooing or meowing at various volumes and tones for pet-therapy. Mini uses sounds such as laughter, whistling, and yawning. Due to cognitive decline, non-verbal vocalizations like “hmm-mm” or “ugh” are more frequently used by older adults to express themselves, however these sounds have yet to be used as input for HRI.

Gestures

Human gesture types include : 1) illustrators that add emotional expression and emphasis to speech (i.e., body language), 2) manipulators used subconsciously that involve interaction between body parts or other objects like fidgeting, and 3) emblems used deliberately to represent words like head-nods or head-shakes. SARs for older adults focus mainly on displaying and detecting illustrator and emblem gestures. SARs that use illustrator gestures include Brian 2.1, Pepper, Casper, ARI, Bandit, NAO, Mini, Stevie, and Tangy to indicate focus of attention when speaking[56,57] or complement the emotion in speech. Brian 2.1 can determine user engagement by detecting illustrative gestures based on Canadian cultural norms. Pepper uses emblem gestures such as bowing and waving to display cultural competency specific to either Japanese or British backgrounds. Hobbit uses emblem gestures for different commands such as swiping for menu navigation. Bandit, Baxter, and NAO[46,55] use emblems during exercise tasks to communicate proper exercise form and detect user compliance.

Displays

Visual displays are used to provide task specific instructions, show pictures or videos, or for teleconferencing with other people. Displays may be output only or interactive touchscreens. Tangy uses its torso display to show Bingo numbers and Trivia questions to augment its speech.[56,57] SARs with touchscreens include Casper to provide meal assistance instructions and offer recipe choices, Stevie for voice/video calling, and Pearl to add upcoming appointments to its calendar. Furthermore, Pepper, Kompai, Max, Hobbit, Mini, and ARI all display cognitive games on their touchscreens, which is especially valuable for older adults where comprehension speeds will vary. Display height is typically targeted to accommodate older adults in a seated position, some displays may be tilted to improve accessibility when standing.[42,51,52]

Physical touch

In general, SARs do not touch a user however some can detect physical touch. Physical touch detection may be categorized as: 1) affective, for showing appreciation, 2) instrumental, to achieve a specific task, 3) controlling, to get attention, and 4) ritualistic, for greetings or departures such as handshakes. SARs that detect affective touch such as petting or stroking include the pet-like robots JoyForAll Cat and PARO. Baxter detects instrumental touch during interactive exercise games. Mini responds to controlling touch (e.g. a tap on head) for initiation of tasks. Pepper detects ritualistic touch from sensors, i.e., on the top of its head, as a means of putting the robot in/out of sleep mode. Culture was not explicitly considered in developing physical touch for SARs, however, it could help to promote generalizability to older adults with different cultural backgrounds.

Table 1 presents a summary of the design features and applications of the aforementioned SARs, with respect to the categories for type of appearance and interaction mode. In general, studies on the effectiveness and efficacy of SARs for older adults have shown positive outcomes most notably in cognitive training, ADL assistance, and as multifaceted solutions to prolong aging-in-place.[9,18,85] These applications are the most common for the SARs in Table 1. The use of SARs for social and psychological therapy requires more rigorous testing as current studies are limited to short-term interventions and their results are influenced by external factors such as changes in the daily lives of older adult.

Table 1.

Existing socially assistive roboticss for older adults.

SAR	Applications	Appearance Categories	Interaction Modes	Works
Brian 2.1 Courtesy of ASBLab, UofT	Engages in cognitive and memory games, meal-eating assistance	Type: Human-likeFace: Gaze direction, facial expressions through deformable faceBody: Gestures, and head and upper -body movementsSize: Height: 135 cmOuter ShellMaterial: Silicone face, aluminum body	Input: Sentence recognition, affect detection through body poses and gestures, wearable and object-based task specific sensors, illustrator gesturesOutput: Speech synthesis, facial expressions illustrator gestures	[17,38] 39
Milo 25 Courtesy of Robotkind	Conversation therapy for older adults with Alzheimer’s disease	Type: Human-like,Face: Gaze direction, facial expressions through deformable faceBody: Gestures, and head and upper -body movements Size: Height: 50 cmOuter ShellMaterial: Polymer face, hard plastic body	Input: Sentence recognitionOutput: Speech synthesis, illustrator gestures	[40,41]
Pepper Courtesy of RobotLAB	Engages in conversations, facilitates games and exercise	Type: Character-likeFace: Different eye colors for showing emotionBody: Gestures, head and whole-body posesSize: Height: 120 cmOuter ShellMaterial: Hard plastic (injection molded)	Input: Sentence recognition, touchscreen, ritualistic touch on head to sleepOutput: Speech synthesis, illustrator and emblem gestures, touchscreen	[37,42] [61,76] 77
Casper Courtesy of ASBLab, UofT	Assists with meal preparation	Type: Character-likeFace: Gaze direction, facial expressions through LEDsBody: Gestures, head and whole-body posesSize: Height: 125 cmOuter ShellMaterial: Hard plastic (3D Printed)	Input: Sentence recognition, touchscreenOutput: Speech synthesis, illustrator gestures, touchscreen	[29,36] 78
ARI ©PAL Robotics 2021, all rights reserved	Provides reminders for scheduled activities, cognitive games, fall detection, audio/video calling,	Type: Character-likeFace: Gaze direction, different head colors for showing emotion Body: Gestures, head and whole-body posesSize: Height: 165 cmOuter ShellMaterial: Hard plastic	Input: Sentence recognition, touchscreen Output: Speech synthesis, illustrator gestures, touchscreen	[43,62]
Stevie Courtesy of Trinity College Dublin	Engages in conversations, facilitates group games	Type: Character-likeFace: Gaze direction, facial expressions through LCD displayBody: head and whole-body posesSize: Height: 140 cmOuter ShellMaterial: Hard plastic	Input: Sentence recognition, touchscreenOutput: Speech synthesis, illustrator gestures, touchscreen	[44,53]
Bandit Courtesy of USC Interaction Lab	Physical exercise coach, cognitive games	Type: Character-likeFace: Gaze direction, facial expressions through actuated eyebrows and mouthBody: Gestures, head and whole-body posesSize: Height: 110 cmOuter ShellMaterial: Hard plastic	Input: Exercise emblem gesturesOutput: Speech synthesis, exercise emblem gestures	[45,79]
NAO Courtesy of RobotLAB	Smart home interface, exercise coach	Type: Character-likeFace: Gaze direction, different eye colorsBody: Gestures, head and whole-body posesSize: Height: 58 cmOuter ShellMaterial: Hard plastic	Input: Sentence recognition, exercise emblem gesturesOutput: Speech synthesis, illustrator gestures, exercise emblem gestures	[46,55] 80
Mini Courtesy of UC3M Robotics Lab	Cognitive games, interactive dance	Type: Character-likeFace: Gaze direction, facial expressions, and head and whole-body posesSize: Height: 50 cmOuter ShellMaterial: Hard plastic with fur clothing	Input: Sentence recognition, touchscreen, controlling touch for starting tasksOutput: Speech synthesis, illustrator gestures, touchscreen	[47,64]
iCat Courtesy of Christoph Bartneck	Engages in conversations, provides reminders and weather information	Type: Character-likeFace: Gaze direction, facial expressionsSize: Height: 38 cmOuter ShellMaterial: Hard plastic	Output: Speech synthesis	50
Hobbit Courtesy of Hobbit Project	Provides reminders, household object retrieval, cognitive games, fall detection, exercise	Type: Character-likeFace: Gaze directionSize: Height: 125 cmOuter ShellMaterial: Hard plastic	Input: Keyword recognition, command emblem gestures, touchscreenOutput: Speech synthesis, touchscreen	[48,81]
Pearl Courtesy of CMU Robotics	Provides reminders, mobile navigation guide	Type: Character-likeFace: BlinkSize: Height: 120 cmOuter ShellMaterial: Hard plastic	Input: Keyword recognition, touchscreenOutput: Speech synthesis, touchscreen	[7,49]
Kompai Courtesy of Kompai	Provide reminders, mobile navigation guide, audio/video calling, cognitive games	Type: Character-likeFace: StaticSize: Height: 125 cmOuter ShellMaterial: Hard plastic	Input: Keyword recognition, touchscreenOutput: Speech synthesis, touchscreen	[51,68]
Max Courtesy of Ilmenau University of Technology	Provide reminders, smart home interface, audio/video calling, cognitive games, fall detection	Type: Character-likeFace: Gaze direction, eye-color to show current task, eye shape to show stateSize: Height: 120 cmOuter ShellMaterial: Hard plastic	Input: Keyword recognition, touchscreenOutput: Speech synthesis, touchscreen	[52,54]
Tangy Courtesy of ASBLab, UofT	Group cognitively stimulating activities	Type: Machine-likeFace: Gaze direction, facial expressions through mouth actuationBody: Gestures, head and whole-body posesSize: Height:140 cmOuter ShellMaterial: Aluminum	Input: Keyword recognition, task progress through RGB-D cameraOutput: Speech synthesis, illustrator gestures, display	[22,56] [57,82]
Baxter Courtesy of CORiS, OSU	Exergames	Type: Machine-likeFace: Gaze direction, facial expressions on LCD displayBody: Gestures, arm posesSize: Height: 178 cmOuter ShellMaterial: Hard plastic	Input: Exercise emblem gestures, instrumental touchOutput: Exercise emblem gestures	58
PARO Courtesy of PARO Robots	Pet therapy	Type: Animal-likeFace: BlinkingBody: Head and whole-body movements including head shaking, tail and flipper movementsSize: Height: 16 cmOuter ShellMaterial: Hard plastic with fur cover	Input: Affective touchOutput: Expressive sounds and movements	[59,65] [66,75] [83,84]
JoyForAll CatCourtesy of JoyForAll	Pet therapy	Type: Animal-likeFace: BlinkingBody: Head and whole-body movements including head nodding and paw raisingSize: Height: 26 cmOuter ShellMaterial: Hard plastic with fur cover	Input: Affective touchOutput: Expressive sounds and movements	[60,84]

Senior care studies on sar features

Several HRI studies have been conducted with older adults to measure and compare appearance features and interaction modes of SARs using key concepts such as trust, likeability, and intent to use. In general, these studies either have low participation numbers (e.g., between 5–10 users)[54,68,78] or are limited to a single interaction[50,58,65] This is primarily due to limitations in working with vulnerable populations such as older adults with dementia who may face cognitive fatigue when engaging in such research studies. However, critical user trends within these studies can still be identified with respect to such measures as trust, intent to use, and enjoyment, and can be used to inform other similar studies. It is important to note that this field of HRI is still in its infancy, while also considering the challenges of working with vulnerable populations and the novelty of the SARs being tested.

Human-likeness

In, the influence of robot embodiment in assisting with a tea-making ADL on the overall perceptions and experience of HRI for older adults with mild cognitive impairments (MCI) was investigated. Three different platforms were used consisting of a character-like robot (Casper), machine-like robot (Ed), and a tablet placed on a table. Questionnaire results showed that Casper was the most preferred and engaging robot due to its dynamic features.

In, three robot characteristics were individually manipulated: robot face (none, machine-like, character-like), voice (none, digitized, human), and interaction mode (none, display tablet, touchscreen) to determine their influence on the affect of older adults during medication delivery. A character-like face and a touchscreen had the most influence on self-reported positive affect with the latter also increasing engagement as measured with heart rate. Using a human voice also increased positive affective response, however, with a lower effect size.

Expressiveness

In, the iCat character-like robot was used to explore the combined effect of facial expressions (smiling and nodding/none) and gaze (looking at user/not looking at user) on SAR acceptance by older adults during an information providing task (weather, reminders, etc.). Participants who interacted with the expressive iCat showed more conversational expressions, however, this did not increase SAR acceptance as measured by post-interaction surveys.

Size

In, the 50 cm tall SAR Mini was placed in a LTC home for 2 months where residents could freely interact with the SAR to engage in exercises using its touchscreen. Questionnaires for older adults, caregivers, and relatives showed high scores for usefulness, ease of use, and satisfaction with the SAR being perceived as friendly, smart, and safe. However, older adults did not believe Mini could increase their autonomy.

Material composition

In, PARO was used with independent living older adults to explore potential emotional support benefits. Older adults participated in a guided introduction to PARO and its capabilities with opportunities to hold and interact with the SAR using touch. Post-interaction interviews showed that older adults most liked PARO’s fur, color, and cute appearance while they least liked PARO’s limited functionality including inability to understand speech.

Verbal Communication

In, Kompai was deployed in the homes of older adults living alone. Users could ask the SAR to perform several tasks using either verbal communication or a touchscreen. Users below 80 years of age had no clear preference of communication mode, however, there was a significant preference for adults older than 80 to use speech.

Sounds

In, older adults engaged with the JoyForAll Cat in their own homes during two months to investigate if the robot could decrease loneliness. Older adults reported a decrease in loneliness, and interviews showed they appreciated the presence of the SAR. However, while the SAR could make sounds, many older adults noted the lack of interaction and responsiveness.

Gestures

SARs that can show and understand illustrative gestures have been rated highly by older adults for intent to use and enjoyment. Expressing emblem gestures using Pepper to display cultural competency can increase the emotional wellbeing of older adults as shown in HRI studies with this robot using culturally appropriate greeting gestures such as bows and waves with users from both Japan and Britain.

Displays

The addition of touchscreens increases positive emotional response and engagement and supports verbal information from SARs through the use of text and visuals in tasks like Trivia or cognitive games. Furthermore, a separate study with Hobbit showed speech and a touchscreen were significantly preferred to understanding emblem gestures for giving the SAR commands.

Physical Touch

In, the machine-like Baxter robot was used to determine the effect of physical touch (hitting/none) on older adult enjoyment of exercise games. Participants completed 8 games with varying amounts of physical touch in the form of hitting pads mounted to force sensing actuators. All activities that involved hitting rated high for enjoyment.

In, the SAR Max was deployed in residential care apartments to allow older adults to use features such as medical reminders, audio/video calling, and emergency detection. Some participants reacted emotionally to its behaviors, speaking to it as a social entity and frequently touching the SAR during interaction. Physical touch has been shown to also increase moods in pet-therapy sessions with PARO.

SAR awareness and behavior frameworks

To design adaptive behaviors and intelligent autonomy for SARs, robot architectures have been developed to achieve robot awareness (task and user classification) and personalization (adaptive behaviors) based on the varying needs of older adults.

Task and User State Classification

Task classification is used to identify and monitor the steps needed in completing a particular task. User state classification considers user affect and engagement throughout the interaction as input for robot behaviors. Both classification forms may use data from onboard robot sensors including RGB-D cameras[56,88] or user and object sensors.[39,89]

Task State Classification

In, task classification was performed by the character-like HomeMate SAR using RGB-D and laser scan data with a dynamic Bayesian network (DBN) to determine observed task states between the ADLs of meal preparation, cooking, eating, and taking medication. The SAR extracted features from the skeleton model of older adults and the relevant objects, i.e., dishes or a fridge for the DBN to classify the most likely task observed.

In, Tangy autonomously facilitated Bingo games with LTC residents. The game progression for a player was monitored when the older adults requested personalized assistance using an infrared reflective system detected by the Robot’s IR sensor. Once the SAR approached the older adult, a 2D camera was used to detect the Bingo card features using its unique identifier picture and determine the location of number markers to classify the Bingo card as: 1) marked correctly, 2) incorrectly marked and/or missing markers, and/or 3) a winning card.

User State Classification

In, the Pepper robot classified user valence and arousal into an affect detection model using multilayer perception neural networks during a robot emotion elicitation activity. The affect of older adults from a LTC home was measured using an EEG sensor during robot emotional dancing which was defined as upper body movements to express either positive valence and high arousal or negative valence and low arousal based on movement speed and dynamics.

In, NAO used RGB cameras to identify facial features for emotion classification between seven different expressions using a Random Forest Classifier. Using a combination of distance, polygonal area, and elliptical area features resulted in good accuracy with older adults even when their faces were partially occluded by fingers or glasses.

Task and User State Classification

In, Brian 2.1 classified the progression of the meal eating task with older adults using a smart tray (with embedded force sensors) and utensil tracking system (using Wiimotes onboard the robot) to provide appropriate prompts, social encouragement and reinforcement. Body language and face orientation were also tracked and classified using a 3D Kinect sensor to determine if the older adult was distracted or accessible during meal eating to reengage them if needed using different robot emotions.

Adapting SAR Behaviors

Behaviors of autonomous SARs were initially designed with finite state machines (FSMs) to provide predefined responses for sets of identified inputs from users and their environments. Recently, adaptive behavioral control has been used in SARs via: 1) robot self-learning through direct interactions with users, 2) learning from demonstrators (e.g., caregivers or experts), or 3) a combination of the two. In designing SAR behaviors for effective HRI with older adults, emotional or persuasive strategies can be implemented to adapt to user and task specific preferences.

Task Behavior Learning Methods

Reinforcement learning (RL) methods have been used for SAR self-learning using rewards such as level of engagement. However, since all behaviors must be attempted for a SAR to learn those that elicit high rewards, there is a risk that older adults may need to repeat a negative response to poorly received behaviors several times, causing confusion or frustration. Learning from demonstration (LfD) has been used for SARs to learn new skills by directly observing them from caregivers, such as Tangy learning to autonomously facilitate Bingo sessions from caregivers in LTC homes. LfD benefits from fewer user interactions to determine behavioral strategies, however, it can rely on a significant number of demonstrations. A unique hybrid approach using both LfD and RL can also be used to provide robot learning of task-specific behaviors through LfD and then personalization through online learning using RL, as was used by Casper in assisting older adults to make tea.

Emotional Behaviors

SARs use emotional models to communicate their intent and internal states to older adults during assistance to improve older adult understanding of the robots. FSMs use transition rules to relate inputs from SAR sensors to robot emotion state changes to improve older adult task performance.[38,55] Alternatively, in, an nth order Markov Chain based emotion model was developed for the Salt robot to determine when to display the four emotions of happy, interested, sad and worried, in response to user engagement in an activity, user affect and the robot’s own emotional history.

Persuasive Behaviors

Persuasion in HRI seeks to change users’ attitudes or behaviors. Persuasion strategies used by SARs when interacting with older adults can be categorized as: 1) motivation strategies, and 2) compliance gaining persuasive strategies. Persuasion strategies are frequently used by assistive technologies to achieve compliance and engagement in ADLs by older adults and present opportunities for similar strategies to be used by SARs.

Senior care studies on SAR behaviors

Studies have been conducted with older adults to investigate various robot behavior learning methods and the use of SAR emotion and/or persuasion on HRI experience. User and state classification have been mainly used as inputs for SAR behavior adaptation.

Task Behavior Learning Methods

In, Pepper used RL to determine discussion topics and robot gestures based on user engagement, measured by older adult verbal responses, in order to improve user emotional states. The SAR’s dialogue was personalized overtime as users engaged in conversations and playing games with the robot. Conversation personalization was focused on British and Japanese cultural topics. After two weeks of interactions, emotional wellbeing improved compared to a baseline group.

In, Tangy used LfD to learn from caregivers how to autonomously facilitate Bingo sessions using behaviors including calling out Bingo numbers and checking Bingo cards. Teachers could further customize the robot’s learned behavior by modifying the SAR actions using a graphical user interface. An HRI study with LTC residents showed that older adults believed the SAR was easy to use and found Tangy’s behaviors helpful and enjoyable.

In, Casper used LfD to learn assistive behaviors from allied-healthcare students from nursing, occupation and physical therapy, and speech-language pathology, to assist older adults in the tea making activity using verbal and non-verbal-based prompts with varying levels of speech directness (assertive/suggestive) and movement activity (high/medium/low). Casper’s behavior was further personalized using on-line RL based on completion of activity steps. User studies with Casper and residents in a retirement home showed they perceived Casper as socially intelligent and had high levels of engagement and positive affect.

Emotional Behaviors

In, Brian 2.1 played a matching card memory game with older adults in LTC setting. The robot displayed emotional behaviors (happy, neutral, sad) using an FSM-based behavior model that autonomously determined voice and facial expressions based on player accessibility (high to low). Questionnaire results showed that emotional expression was the most liked feature of Brian 2.1, which also received high scores for enjoyment and acceptance.

In, the Salt robot autonomously facilitated exercise sessions for older adults living in LTC. The robot guided the participants through multiple repetitions of upper-body exercises, using its nth order Markov model to determine its emotional response (happy, interested, sad, worried). The majority of users maintained a positive valence throughout the sessions with Salt and believed their physical health was improved. They were also motivated to continue performing daily exercises with the robot after the 2-month study was completed. Both emotional behavior adaptation studies were based in Canada.[38,93]

Persuasive Behaviors

In, Bandit was used to facilitate a musical cognitive game with older adults to explore whether it could improve cognitive attention through adaptive motivational behavior. Bandit changed its assistance level, using an FSM based on user reaction time and the percentage of game questions answered incorrectly, between 1) no hints, 2) directing when to press a button, and 3) saying which button to press. Analysis of older adult engagement during the activity confirmed the SAR was able to maintain user attention and improve task performance.

In, Tangy used a Thompson Sampling based approach during Bingo game facilitation to learn a personalized persuasive strategy for encouraging a specific older adult to comply with requests for playing. The persuasive strategies learned included neutral, praise, suggestion, and scarcity. A user study with Tangy and residents of a LTC home was conducted to explore engagement based on visual focus and compliance during group gameplay. Tangy’s personalized assistance was found to increase engagement with all users having very high compliance with SAR requests.

Discussions

Does the appearance of the robot matter? In comparison studies that focused on assistive tasks, SARs with more human-like appearance received higher ratings for engagement, perceived intelligence, and intent to use. Studies using simplistic character-like SARs suggest that robot capabilities, namely the tasks performed and interaction modes enabled by its appearance such as gestures, were the main design aspects that older adults were concerned with over appearance. This focus on capabilities over appearance has also been shown in focus groups on assistive robots with older adults.

Typically, preference studies have focused on showing pictures and videos of SARs[15,95] instead of physical robot interactions, further limiting the real in-person experiences of older adults. Additionally, the considered works do not specify any cultural differences in appearance preferences and there are also no quantifiable differences between studies due to large intra-study variation. To fully understand appearance preferences requires long-term studies deploying and comparing SARs with similar capabilities but varying appearance types to isolate appearance effects while considering demographic and culturally diverse users. It is suggested that in developing SARs for older adults, functionality and familiarity are very important to this user group and should be the main priorities during feature design.

How many interaction modes is too much? When available, verbal communication was found to be the most used and liked interaction mode with older adults for social interactions and task commands. The presence of sounds for pet therapy, gestures for exercise, touchscreens for cognitive games, and physical touch for exergames were also found to positively influence HRI. Current works have not directly considered cultural differences in interaction mode preferences, however, some have customized modes such as greeting gestures based on culture which has shown to further improve HRI. Cultural customization can be applied to both verbal and non-verbal interaction modes such as spoken expressions or using symbolic representations on interfaces. SARs with multiple interaction modes provide older adults accessibility and flexibility in using the modes that best suit their physical limitations and personal preferences which improves HRI. An open challenge is to determine when the cost of adding additional interaction modes outweighs the benefit to the older adult users who may not use these modes or find them annoying.

SAR developers should focus on identifying and improving existing highly valued interaction modes with this population, such as verbal communication, which numerous studies claim as being well-liked[39,68] but also dysfunctional due to existing audio and speech issues. There are also opportunities to explore less commonly used interaction modes in new contexts or develop new interaction modes to meet the specific needs of older adults. Some studies have found older adults have physically touched SARs even when they lack such capabilities, suggesting the potential in exploring physical touch beyond exercise environments or pet therapy to include tasks like ADL assistance. Older adults are more likely to use non-verbal utterances due to cognitive decline, presenting opportunities to improve accessibility by understanding the potential intent of these sounds in HRI.

How should SARs behave? SAR behavior that can adapt to user preferences and affective states has shown increased task performance and compliance among older adults. As developers seek to improve the social abilities of SARs, behaviors will need to focus on providing a personalized approach while considering issues with respect to privacy, transparency, and user autonomy. Further work is required to develop ethical frameworks specific to SARs with older adults to understand these concerns from a design perspective similar to what has been done for telepresence robots.

An example requiring ethical consideration is the demand for a transparent approach to be taken to avoid developing deceptive or manipulative behaviors that will decrease long-term trust and efficacy of SARs, even if they gain short-term user compliance. For emotional models, the relationship between cultural background and emotional expression requires SAR behaviors to be sensitive to different cultural norms of intended users, and aim to create culture-neutral expressions when possible. It is also critical to understand that on an individual level how adults age, and what their needs and wants are with respect to SARs, as these can vary from one individual to another. SAR behavioral models must account for such diversity in user abilities and aspirations.

Empathy is an underdeveloped promising strategy for SARs to use with older adults which may be defined as “The act of perceiving, understanding, experiencing, and responding to the emotional state and ideas of another person”. Empathy presents unique challenges as it requires integration of classification, adaptation, and emotional frameworks. For older adults, empathy has the potential to improve social stimulation and connection which is critical for applications that seek to decrease loneliness and depression. Empathetic strategies specific to older adult mental health have already been an area of study in healthcare, and future SAR developments may use the outcomes from this research to design empathetic frameworks.

Conclusion

As the strain on existing health and social care systems increases, older adults are facing challenges associated with loneliness, mental health, and physical and cognitive decline. There is a significant opportunity for SARs to provide intelligent and autonomous care to older adults and support their caregivers in facing these growing challenges. Given the urgent caregiver shortage, in the short-term SAR development should focus on tasks shown to promote positive HRI, intent to use, and acceptance, which can be reliably performed under the supervision of LTC staff such as facilitating group activities like Bingo and providing ADL assistance to help alleviate the burden on existing caregivers. In the long-term, as intelligent autonomy is improved and ethical concerns of privacy, transparency, and autonomy are addressed, SARs, can be deployed around-the-clock in LTC homes as autonomous tools to prolong independence with a number of everyday activities by providing assistance, monitoring and reminders based on personalized preferences. In exploring SAR appearances, interaction modes, and behaviors, we have highlighted the importance of the integration of these factors for the successful design and deployment of these robots for long-term use with the aim of improving quality of life for this vulnerable population.