How Service Robots Can Improve Workplace Experience: Camaraderie, Customization, and Humans-in-the-Loop.

This paper presents the results of a three-week in-the-wild deployment of a wizarded service robot in a shared campus workplace. The study introduces robot-centric ethnography, a concept in which a wizarded robot acts as a mediated anthropologist, used in this case, to further our understandings of how service robots impact and integrate into everyday workplace experiences. Our research site included participants familiar with robots, recruited from 90+ students and faculty working in a shared lab space. Our wizarding team visited these participants each workday they were there for three weeks, navigating open office and lab spaces to remind participants to be aware of their mental, physical, and nutritional health needs. Using a semi-structured format, the wizards adapted the standard interaction flow to the situation. This interaction sequence was guided via pre-populated buttons on our health coach interface, with human wizards triggering the timing and adding extra responses as felt natural. Our ethnography-informed approach used the social knowledge of both participants and wizards, blending the robot into the cultural environment in which it was operating. Our data supports the positive impact of fluent service robot experience on participant mood and overall workplace experience. This suggests that effectively designed service robots can benefit workplace environments above and beyond their intended functions.

Introduction

The coronavirus pandemic and the ensuing physical distancing guidelines seem to have rendered human-in-the-loop control as the most prevalent paradigm of robots operating in human settings. Though current developments focus on rising applications like delivery robots and corresponding physical manoeuvring practicalities [29, 42, 56], wizarded systems seem to be trending towards more social applications like clinical settings [5, 51, 53]. One can infer that continuing innovations will only make further inroads, with robots entering our common everyday environments, such as workplaces, where social capabilities will be even more critical [9, 11, 30, 36, 68]. How can researchers identify the cultural and social situation of such systems before they materialize in the real world, gaining formative design insights for this rapidly developing future?

This paper introduces a novel methodological concept of robot-centric ethnography, which involves applying anthropology to human-robot interaction and using the improvisational method of wizarding to develop technology solutions, with human-in-the-loop controllers (wizards) remotely teleoperating the robot. This draws inspiration from both software engineering and interaction design, where ethnographic methods are often used as tools in user-centered participatory design [60]. It also draws from previous uses of human-in-the-loop remote control, a.k.a., wizarding, to allow the robot to seem autonomous in order to better understand people’s reactions and attitudes toward particular social designs [9, 11, 30, 36, 68]. The ethnographic approach of this paper allows consideration of the cultural situation of the robot, helping the researchers explore how the robot can integrate into and impact its social environment. Robot-centric ethnography also helps surface qualitative data about how the robot was considered by its environment. In particular, we seek to understand RQ1: How can robot-centric ethnography help us understand the social situation of future workplace service robots? and RQ2: How can human-in-the-loop control (wizarding) inspire the design of future interactive robot systems?

To instantiate the research, our 3-week deployment period utilized the concept of a robot health coach as a backdrop for evaluating service robots in shared workplaces (Fig. 1). Consumer health technologies like Fitbits and Apple Watches, generally focus on monitoring one’s own behaviors, providing notifications to cue current behaviors, and/or leveraging the social dynamics of groups (e.g., sharing your activity level with friends). Furthermore, research has shown that integration of social robots into such health activities leads people to find their experience more enjoyable and socially attractive [24, 39]. In a world with job pressures that have been increasing for years [61], this platform sought to remind workers to check in on their physical, nutritional, and mental health. Using the robot-centric ethnographic method, wizards improvisationally control the robot to lead the participant through before-and-after mental health questions, offering physical exercise and a healthy snack. The wizarded control allows them to flexibly react to participants’ responses. Both wizards and participants offer insight to our ethnography-inspired analysis, hence their feedback is included throughout the structured experiments and techniques outlined in our methods section.

Fig. 1

ResolutionBot interacting with a study participant. The robot health coach operated in a shared workspace, offering snacks, exercises, and a break from the busy days of the people working there

The results show that service robots have the potential to positively impact workplace camaraderie. For example, participants that exercised more enjoyed their colleagues more too. Additionally, the ability to customize robot behaviors to particular contexts was tractable to do live, hence offering insights on how to design both autonomous and human-in-the-loop systems, such as when participants were grateful that the robot asked them to do exercises that they liked. Finally, the human-in-the-loop control that this study employed allowed the robot to make jokes/conversation about the workplace environment to sometimes serve as icebreakers before starting an interaction.

The related work section will continue with background insights of wizarding techniques, user expectations, and robot health-related applications (Sect. 2). This is followed by clarifying the robot-centric ethnography (Sect. 3) that this study employed and introducing our hardware design/study design (Sects. 4 and 5). The test results of health activity participation and survey response analysis will be discussed in Sects. 6, 7 and 8, will provide holistic interpretations of the results. Lastly, future implications will be highlighted in Sect. 9.

Related Work

To understand how service robots might be able to improve workplace experience, we investigate prior work revolving around developing everyday service robots, data collection mechanisms, prior HRI study results, and other existent health-supporting technologies.

Prior Uses of Wizarding in HRI

Wizard-of-Oz (WoZ) is a method in which a person operates a robot remotely, to make it seem as if the robot itself is the agent interacting with the people around it Riek [63]. WoZ is often used in technology prototyping and entertainment. For example, previous uses of the WoZ method in social robotics often simulate higher levels of technical capability than might actually exist, in order to help technology designers make recommendations about what should actually be programmed, built, or used [36, 68]. A helpful technique for early design stages of robot operations [32], wizarding can also be used to test which features of robots work well in particular situations [4], e.g., utilizing methods from controlled user studies [7]. Wizarding can also be used in entertainment applications, such as the Disney trashcan robot, Push [66], which appeared to be playfully interacting with park visitors, but was in fact operated by a person disguised as a visitor in the crowd. This and other technology-mediated exhibits [1] utilize humans ‘behind the curtain’ to improvise playful interactions with park visitors. The present paper leverages the flexibility and fluidity of the WoZ research method to explore desired features for service robot interaction and to imbue the robot with a compelling machine personality.

Human-in-the-loop control, which shares many attributes with Wizard-of-Oz, has become even more prevalent and useful in a post-pandemic world, from tele-medicine robots [38, 51, 74] to robots acting as social agents [5, 53]. Due to the desire for physical distancing, there has been a shift to remote rather than collocated operators, with companies like Starship Technologies pioneering the robot food delivery industry and completing more than 2 million deliveries as of October 2021 [72]. Wizard of Oz has been a helpful way to explore prospective scenarios for delivery that may come up as technology progresses into the future [29, 42, 56, 73], revealing formative results that can be used toward insight-based design of final interfaces and system capabilities.

People’s Expectations of Service Robots

Many previously deployed robots have been discontinued due to not having met people’s expectations [19, 34, 43, 54, 62], so the stakes of getting robot sociability right are high in service applications. An example of such discontinuation is Fabio, a hotel robot that was unable to respond to customers posing it simple random questions [34], leading to the robot exhibiting unexpected social cues [54]. Similarly, the Jibo robot has been replaced by cheaper and ‘smarter‘ smart speakers that are available in contemporary markets [57]. One possible reason for such failed products might be the lack of an effective framework that allows exploring, prototyping and understanding peoples’ expectations of social robots and their benefits, hence leading to ineffective market research and highlighting the need for an ethnographic approach to social robot deployment in the field. Inspired by its applications in HCI, auto-ethnography, a technique in which a researcher acts as a participant, has been proposed by researchers as a method to facilitate roboticists’ exploration into scenarios where robots regularly operate [15]. Prior work in social robotics has also made use of participatory design, a framework that involves the end user in the design of a product (here, the robot), to obtain realistic human expectations to aid robot design [8, 50]. Prior work suggests that everyday service robots that behave like people are often considered easier interaction partners [10]. This is consistent with early recommendations in social robotics [18], which suggest that establishing appropriate social expectations, having readable social cues, being able to read human cues, and limiting communications to what the robot can handle are four fundamental capabilities for systems that utilize social metaphors.

Prior Uses of Tech Applications to Support Health

This application was motivated by work in psychology suggesting that health goals are easier to accomplish with other people [20], and that doing so in group settings can increase camaraderie [59]. Technology-based health applications have leveraged these prior findings in human-human psychology research, resulting in both robotic and non-robotic health-supporting implementations [37, 41, 52]. In particular, fitness technologies have gained widespread acceptance among consumers, demonstrating people’s interest in technology-supported health applications [2]. Because of social robots’ ability to cue social responses in people, social robots may one day gain similar acceptance. Early studies show that social robots can encourage exercise in children [28] and elders [6]. This effect varies based on various aspects of the interaction. For example, people will do more physical therapy exercises with a robot that matches the user’s introversion/extroversion style [71]. Building on the short term boosts provided by electronic notifications [13], and leveraging the embodiment of the robot to keep people with the program for longer (as opposed to Iqbal and Horvitz [33]; Zach et al. [75]), robot health coaches may also help people meet their own fitness [58] and nutrition goals [40], and even provide a fun break for people at work.

Robot-Centric Ethnography

The conceptual innovation of this work is the idea of using “Wizard of Oz” remote teleoperation to place the ethnographic researcher as an integral actor in the field, which takes inspiration from and extends the concept of auto-ethnography, in which the researcher acts as a participant in social data collection and understanding [14]. A second key idea is that of participatory ethnography, which provides a conceptual framework to give research participants a chance to integrate their own life situations (e.g. their own health and moods) and to express them during the technology development and theory building process [17, 48]. Thus, the integration of applying the anthropology method to human robot interaction and utilizing the improvisational method as a technical solution for deploy and program the robot is called robot-centric ethnography. Instructing wizards to use ethnography-inspired approaches allows them to leverage their own social understanding in modifying the interaction on the fly, providing contextually-appropriate behaviors and accommodating unexpected situations and reactions in a way that is socially normative. This aspect of our robot behavioral design exploration leverages the ways in which anthropologists use reflexive understandings to inform a dynamic data collection process, which can later shape sociological theory. Identifying similar future opportunities for robots to integrate in everyday human spaces has some parallels to design research, in which researchers integrate users into the robot design process before the robot’s actual behavior design is complete [8, 15, 50].

Prior work when designing robots has often been confined to programming behaviors and then deploying robots with these behaviors. On the contrary, our deploy and then program effort allows robot developers to integrate humans early in the design process, and enables the collection of a large quantity of data, both behavioral and sensor-generated, that we can use to inform the building of behavioral models which will ultimately be utilized in our system. The innovation here is the operationalization of ethnographic methods via improvisation, relying on both preset interaction goals (see interaction flow in Fig. 5), and the dynamic response of the wizards to the interaction needs not covered by that anticipated flow. This improvisational method is inspired by improvisational theater [35] to design robots in a way that can handle diverse human responses in order to build interaction models for future robots [26, 32, 69].

Fig. 5

ResolutionBot’s Interaction Flow. A complete outline of the target interaction flow, which was divided into three segments: mood reporting, exercising, and snack taking. This interaction flow provided a consistent structure across all of ResolutionBot’s interactions with participants over the study. If participants deviated from this target interaction flow, the incorporation of the WoZ technique allows the wizards to handle any interruptions and then flexibly guide the interaction back to the target flow

Incorporating the concept of improvisation into traditional Wizard of Oz control techniques, our lab has many prior publications integrating improvisation into robot behavioral design and development processes [3, 22, 44–46, 64]. Utilizing the prior knowledge of low degree-of-freedom robots gestures, such as ChairBot’s simple translations (robot chair) [3, 46], we now design robots that can be deployed in a real non-controlled world. Using this improvisational control method, we have access to holistic ethnographic data that can help us develop technological solutions for robots operating in non-controlled human spaces. From prior experience using improvisational methods in our lab, human improvisers can be flexible and creative when describing a human-robot interaction [22, 64]. In this research, we amalgamate our prior experience with this improvisational approach and the operationalization of ethnographic methods to a shared human workspace.

Ethnographic methods for robot deployments extend our lab’s prior multidisciplinary research experience in applying anthropology to human-robot interaction and developing technological solutions for robots operating in human spaces. Our lab has collaborated with an anthropologist (the third author) for three years, during which she has contributed to four publications exploring the relationship between anthropology and human-robot interaction [14, 16, 22, 23]. Applying ethnographic methods to robotics can help researchers understand how robots operate in human environments and can optimize the robot design process [16]. In the third author’s fourth publication [14] with the lab, she holistically introduces ethnographic reflexivity suggesting that during human-robot experiments, the researcher should “physically” and “socially” assimilate into the scenario to be able to collect primary feedback from the participants. For example, the researcher can approach the participants and have conversations with them about their initial thoughts. Teaming with the anthropology expert helps us provide an ethnographic point of view when interpreting social influences and when analyzing the social behavioral data.

The Robot Health Coach

Fig. 4

A concept of wizards sitting side by side, with one using a PS4 controller to control the robot’s motion and the other using a laptop to control the robot’s speech. The router served as a communication bridge between the wizards’ laptop and ResolutionBot

We augment a TurtleBot2 platform to serve the bi-directional communication needs of our robot health coach application, also developing a wizarding interface and maintaining a target interaction flow.

Hardware Description

As many service robots (e.g., Knightscope [47]) have relatively simple forms, we target a simple physical design for our robot health coach (Fig. 2), based on the Turtlebot2 platform. This design also provides an opportunity to explore potentials for minimal robot social communication, and reduce manufacturing costs relative to higher degree-of-freedom systems. We utilize an open-source Turtlebot2, consisting of a Roomba like YUJIN Kobuki base and an Asus 1215N laptop (Fig. 2). The robot has 4 degrees of freedom since it is able to rotate and translate across the floor.

Fig. 2

ResolutionBot’s hardware components. To enable bi-directional audio communication, ResolutionBot’s hardware included a webcam with a microphone, speakers, three haptic feedback buttons (mood buttons) and a GoPro. An robot’s laptop, was the central hub for these connected peripherals and wirelessly communicated with the wizards’ laptop as shown in Fig. 4. This whole setup was situated on a roomba-like Turtlebot2 base for locomotion

Turtlebots are a common research platform [21], which meant that we could utilize previous code examples in setting up the robot’s ROS integration, also making it relatively easy to add features relevant to its health coach application. Abilities that we added to improve robot-participant communication included mood input buttons that participants could use to report their current mental state, a speaker that allowed the robot to “speak”, and a webcam with an integrated microphone and vertically rotating servo that acted as the robot’s “head”. The mood reporting system consisted of a custom-made module mounted with three large buttons (red, white, and green) that detected button presses and communicated with the Turtlebot2 laptop via an Arduino Uno. The webcam’s integrated microphone was also used to capture participant audio allowing the wizards to react to the participant’s live responses.

Additional hardware requirements prompted installation of a GoPro on the top of the robot to provide quality video footage of the experiment, and a basket to hold snacks that was mounted on the back of the robot. The GoPro offered a live wide-angle view to help the wizards drive the robot and the snack basket allowed participants to pick their snacks during the interaction. The robot maintains an SQL database to track everything that it says, including both commands triggered by the blue buttons and commands typed into the green bar into graphical interface shown in Fig. 3. Fliers were carried by the robot in a 3D-printed structure on its top to provide additional information about the experiment and the team’s contact information, consistent with our university-approved IRB protocol.

Fig. 3

The robot’s customized graphical user interface. The blue buttons can be used to trigger high frequency phrases and are organized according to the target interaction flow (Fig. 5). The green dialog box allows the robot to speak any desired text, providing conversational flexibility to the interaction. The wizards can see this interface on the wizards laptop. The top right of the image has a map of the building and the bottom one-third of the image shows a history of what ResolutionBot has said. (Color figure online)

Wizarding System

This subsection presents the interfaces that the human wizards could use to drive the robot remotely. The robot is intended to be wizarded by two human operators, each of whom manages a particular control interface for the robot: the interaction lead operated the screen-based UI, Fig. 3, while the navigation lead operated a joystick.

One wizard manages the screen-based UI, which has a camera view from the robot perspective, integrated buttons for pre-scripted utterances and gestures, displays of mood button presses, and the potential to type new text as desired (Fig. 3). The integrated buttons were ordered as per our target interaction flow, allowing the drivers to efficiently wizard the robot. The robot commands and communications were all run from an Ubuntu-based laptop over a wireless network. Any new text string typed into the green dialog will be said out loud by the robot. To decrease latency, the robot caches the last 100 command audios in a database on the robot. When a stored command is repeated, the robot directly plays the audio file, bypassing the text-to-speech conversion.

The motion control is done by using the PS4 controller and the GoPro view by the second wizard. The wizard uses a PS4 controller to move and rotate ResolutionBot in the x-y plane and to rotate its webcam’s servo motor up and down. The GoPro with a super wide view mounted at the rear of the webcam, Fig. 2, wirelessly transmits the video feed to the wizard’s phone (not pictured) to allow the driver to avoid obstacles when moving from one location to another (Fig. 4).

Interaction Flow

Wizards consistently used the target interaction flow diagram (Fig. 5) as they operated ResolutionBot. Chronologically, activities in the interaction flow included mood reporting, doing exercises, and taking a snack, as well as typical social greetings and health-activity related conversation. This flowchart acted as a proxy to what would otherwise be specified by robot software, allowing for a consistent interaction structure, also reinforced by the user interface organization (Fig. 3).

To illustrate a full example of how these interfaces and interaction flow worked together in practice, we provide an example walkthrough of how a health coach interaction would go, including typical motion and speech behaviors. First, the robot drives to the participant, stopping in front of them or by their desk and facing them to indicate that the robot would like to interact with the participant. The robot greets and asks the participant, “Is now a good time?” If the participant responds in the affirmative, the interaction proceeds. If not, the robot leaves and moves on to interact with another participant. Usually, it would be the robot that would initiate the verbal interaction, but sometimes participants would start the verbal conversation before the robot could (possibly since the sound of the robot’s motion would alert them of the robot’s approach).

Proceeding with the interaction, the robot asks the participant to report how they are feeling using the 3 buttons on its front (Fig. 2). The green button corresponds to a good mood, the white button corresponds to neutral mood, and red button corresponds to bad mood. The green dialog box in Fig. 3 can be used by the wizards to type out the robot’s responses. The blue buttons on the left can be used for high-frequency phrases like “How are you feeling today?” and “Thank You”. To establish conversational bonding, the robot offers to tell the participant a random joke followed by a fact about exercise and how it is good for one’s health.

The robot then asks the participant whether they want to exercise. If they do, the robot suggests an exercise, e.g., squats, a short walk, push-ups or jumping jacks; options that were pre-populated into the robot control interface. As the participant exercises, the robot counts them off while moving its “head” (the USB webcam) up and down.

After the exercise, or in the case that the participant did not want to exercise, the robot tells the participant a nutrition-related fact and suggests they help themselves to a healthy snack. ResolutionBot turns 180 to present the food basket attached to its back and the participant is free to take any snack from among oranges, bananas, apples and granola bars.

Finally, ResolutionBot again asks the participant to report how they are feeling using the 3 buttons. After this, ResolutionBot says its goodbyes and moves along to the next participant. The interaction flow from Fig. 5 was intended to maintain consistency throughout every participant’s interactions with ResolutionBot, while the control interfaces also offered flexibility on how that consistent flow might adapt to particular participant needs or contexts.

Research Site, Deployment, and Data Collection

This section presents the research site at which this study was conducted, the robot’s three-week deployment with the 15 participants who signed up to receive its services, and our data collection and analysis procedures.

The selected research site has a large ‘open office’ layout, which is associated with increased communication and connection in workplaces [76]. Such open office plans have a “no-walls” approach to individual work-spaces, often selected by companies for the goal of enhancing collaboration and interaction among employees [27]. Located at Oregon State University, the Graf Hall building, in Fig. 6, is a two-floored collaborative lab workspace, consisting of 13 lab groups led by robotics faculty and 90 students with regular access to the labs and desk facilities. Investigating service robots in this space was intended to reveal potentials for shared service robots in similar architectural setups, not unlike the robots that have been deployed on multiple tech company campuses [31, 47].

Fig. 6

An overhead view of the research site: Graf Hall. Interactions with ResolutionBot would occur on both floors and the wizards would use some of the obstacles around the lab space to hide from the participants’ view

Another potential advantage of this site is the health needs of typical university researchers [25]. Previous research has shown that graduate students experience high levels of stress and anxiety [25] due to an amalgamation of classes, research, theses and degree requirements. In addition, prior work has demonstrated that robots can reduce people’s stress while performing cognitively demanding tasks [55]. By approaching the participants with a social and friendly robot that told jokes and offered exercise and snacks, we were curious to investigate whether the robot would be able to impact the worker experience in the research site, particularly as related to their mental, physical, or nutritional health.

A final consideration was the level of knowledge that the participants had about robots before and during the study. Participants had a range of robotics background which we hoped would allow participants to offer realistic feedback into the design and interaction design of a robot health coach. Regardless of participants’ conception of whether they thought the robot was autonomous or tele-operated, the goal of the study is to collect feedback on how to improve the social robot design to better immerse and improve the shared workplace experience. We also expected them to have realistic first expectations of robot capabilities. Although, they do not offer full information on how the general public might respond to workplace service robots, evaluations with this targeted group offer a first step to designing and developing a plausible robot health coach that could be developed for evaluation with generalized user applications.

Study Description

This study took a total of three weeks; 14 days and 28 trials (Fig. 7). There were often two trials in each day, one in the morning and one in the afternoon. Some trials were missed due to departmental or wizard time conflicts as shown by the “–” in Fig. 7. This resulted in 14 trial days with 2–13 total interactions with each participant (Figs. 8 and 9). After sending out emails and posting flyers around the research site, 15 participants (7 males and 8 females) signed up for the study by filling out the sign-up survey. At the start of each experiment day, the wizards would setup ResolutionBot to make sure that the audio, webcam, GoPro, WiFi connection and robot maneuverability were all functioning properly. The wizards then searched for available participants, visiting them in a spatial order that seemed navigationally efficient. On approaching a participant, the interaction pattern would proceed as in the Wizarded Interaction Flow (Fig. 5). After each interaction, the researchers would record the participants’ mood reports, exercise compliance/type, and if snacks were taken. They could also write memos or notes to remember particular features of the previous interactions. When needed, they would also refill the snack basket, or adjust hardware to ensure it was functioning appropriately. After three weeks of participating in the study, the participants were asked to fill out the exit survey as their final task.

Fig. 7

Experiment Schedule. There were 28 trials over 14 days, resulting in 91 total interactions. Three of the 28 trials were cancelled due to the wizards’ time conflicts

Fig. 8

Distribution of interactions over trial days. The average number of interactions per trial day was 6.5 (var = 10.3). The first week of interactions had fewer participants since some participants were recruited during that week

Fig. 9

Distribution of interactions over participants. The number of times each participant interacted with the robot by the end of the experiment. The average number of interactions per participant was 6 (var = 9.9). Participant 13, 14, and 15 were recruited after the experiment had started by word of mouth

Data Collection

The multi-channel data collection process in this study was referenced from existing ethnographic data collection methods [14, 23, 65], such as interviews, transcriptions of interactions and researcher observations. Similarly, this study included data logged by our technology, annotations by the wizards, data collected via participant surveys, and additions to the dataset via post-hoc video-analysis.

By the tech: During the experiment the robot’s onboard GoPro tracked video data about the interaction. The health coach interface also logged all wizard button presses, wizard typed statements, and the participant mood button responses.

By the wizards: During the interaction, wizards also annotated data like subject ID, trial day, and whether subjects participated in the health interactions, e.g., their mood ratings, what snack they took, and whether they did a particular exercise. After each trial, wizards also added field notes on the interaction, adding social observations like “another participant joined the interaction,” or task-relevant participant requests, e.g., someone wanting to do push-ups rather than squats. The wizards also noted aspects about the workplace like the location where the interaction happened, whether other people joined the interaction, whether the participant had previously been busy at work and unique occurrences like participants voluntarily helping the robot around tight spaces, participants joking with the robot, participants asking the robot to carry messages to other participants, etc.

Via the surveys: An end of experiment survey asked participants about their impressions of the robot, the impact of the robot on their health, and the impact of the experience on other factors related to their workplace, collecting five-point Likert scale responses to questions like “the robot was nice to have around,” as detailed in (Fig. 15 and Table 1), as well as open-ended questions where they could provide generalized feedback. The survey questions were crafted to ask participants about their work-related and health-related experience with ResolutionBot, with an emphasis on how well the robot meshed in with the social environment of the shared workplace. As these questions were delivered as part of a post-experiment survey, the researchers decided to ask one question per experimental concept to keep the survey from being too tedious for the participants. Future work should definitely expand on our methods and ask questions with both a positive and a negative valance. The entrance survey was used to obtain consent and to log the participants’ specific locations in the research site allowing ResolutionBot to approach the participant at their desired location.

Fig. 15

This figure presents the average of all the Likert Scale Responses (5-point ratings, ranging from Strongly Disagree (-2) to Strongly Agree (2)) asked in the post-survey questionnaire, indicating which question responses had significant correlations with the amount of , the amount of and with asterisks. Three ANOVAs (for exercise participation, snack participation, and mood change) were conducted across the ratings for each question (column). For example, if participants that exercised were more likely to have a different rating for “I enjoyed my workplace more” compared to a participant that had not exercised, this would mean that exercise participation has a significant impact on a participant enjoying their workplace (indicated by an asterisk)

Table 1

This table breaks down the 4 main experimental concepts for the exit-survey questions and shows the questions that fall into each concept as well as where they are discussed in Sect. 6. The questions were developed to check on participant attributions and reactions revolving around the 4 concepts

Experimental Concept	Survey Question	Section
Workplace Environment	I enjoyed my workplace more during the ResolutionBot study.	7.1
	I enjoyed my colleagues more during the ResolutionBot study.
Impact of Robot on Participant	ResolutionBot made me healthier.	7.2
	ResolutionBot made me happier.
	ResolutionBot made me more productive.
	ResolutionBot led me to connect with more people.
Attributions to Robot	The robot was useful.	7.3
	The robot was nice to have around.
	The robot was friendly.
Personal Commitment to Health	I was committed to mental health.	7.3
	I was committed to nutrition.
	I was committed to fitness.

Post-hoc data additions: Additional variables were added to explore data features that our wizards and team ethnographer (third author) observed in the video data. For example, noting whether the participant was alone or with other people. The non-subjective nature of these labels did not require multiple labelers (for the few reviewed, there was 100% agreement).

Data Analysis

We used quantitative and qualitative data analysis methods to evaluate interaction effects between the data we collected during each trial, as described above. The data collected and annotated by the wizards was evaluated to determine possible cause-effect relationships. The robot-centric ethnographic nature of the study (3) meant that we collected as much data as we could and ran analyses for all interaction effect permutations. All categorical data was coded using numerical labels for efficient quantitative analysis. Recorded videos from participant interactions were transcribed with timestamps and annotations detailing the stage in the interaction flow were included to aid in qualitative data analysis and grounded coding.

The first results Sect. 6, presents aggregate visualizations of user participation and highlights ethnography-inspired qualitative analysis through grounded coding. Aggregate participation refers to the number/percentage of interactions in which subjects participated in each of the robot health activities: mood reporting, exercise, and snack-taking. The anthropologist (third author) conducted grounded coding analysis [12] in three stages: open coding, axial coding and selective coding. Open coding, the first step of grounded coding, allowed us to develop ‘codes’, a.k.a. annotations for relevant concepts. In this stage, annotations such as ‘mood self-report’, ‘exercising’, ‘snack picking’, ‘robot joking’, and ‘off-script conversation’ were iteratively extracted from the recorded videos as being common themes in the interaction flow. Axial coding, the second stage of grounded coding helped group annotations into common categories or key words. Those categories included ‘laughed after ResoBot’s joke’, ‘pranking the robot’, ‘refusing exercise’, and ‘helping the robot navigate’ representing meta-concepts that were present in the data. In the final stage, these repeatable meta-concepts were defined and evaluated with independent reviewers to validate the theory building process. The grounded coding qualitative data analysis process yielded participant quotes and common behaviors categorized under common themes to identify significant concepts for robot communication development and assessment.

The second results Sect., 7, analyzes the effect of undertaking health activities on participant’s Likert responses to the end-of-experiment survey questions. Statistical analyses were run to determine the effect of health activity participation on participant’s Likert responses to the survey questions in Fig. 15. Parametric one-way ANOVAs are used to test for differences in the means of a normally distributed interval dependent variable across a categorical independent variable with two or more categories. After confirming normality using the Shapiro-Wilk Test, we used ANOVAs to evaluate the effect of ratio variables like “% Exercise Participation (mean)” on categorical variables like user ratings from the exit survey. We used the Kruksal-Wallis H Test when the data was non-parametric, i.e. non-continuous data such as the 4 different types of exercise (walking, jumping jacks, push-ups, and squats) and snacks (granola bar, orange, banana, and apple), which do not have any numerical order/interstitial values.

In the next section, we will delve into demographic results with regards health activity participation and participant mood change.

Health Activity Participation

The final experimental dataset included 15 participants (9 females, 6 males); all of who interacted with the robot at least once over a three week period with 14 trial days, resulting in 91 interactions in total. Fig. 10 summarizes how frequently our users participated in three health activities across the 91 recorded interactions, where 80% (N=73) of the participants reported their mood changes (mood at the beginning and end of each interaction), 71% (N=65) exercised, and 79% (N=72) took snacks.

Fig. 10

Distribution of people’s participation in health activities. Overall, 80% of interactions resulted in completed mood surveys (73 completions/91 offers), 71% of interactions had participants exercise (65 completions/91 offers) and 79% of interactions had participants take snacks (72 completions/91 offers)

This section analyzes whether and how participants participated in each of the three health activities: mood reporting, exercising, and snack taking. We report participant participation in the robot health activities as a percentage of overall interactions and the distributions of their participation, e.g., how many trials involved mood increases (29%) vs. decreases (0%). The following subsections detail these participation results supported by relevant examples from our qualitative analysis.

The Impact of ResolutionBot on Participant Mood

The mood reporting activity involved participant selection of how they felt (positive, neutral, negative) at the beginning and end of each ResolutionBot health interaction. 80% of interactions resulted in participants participating in the mood reporting activity. Because of the dynamic space in which the robot was operating, we observed that people would sometimes get distracted from that part of the interaction, e.g., starting other conversations with ResolutionBot, interacting with other people around them, or just ignoring that part of the script. However, most users participated in most interactions and these mood responses were most frequently positive, or trended positive from the beginning to end of the interaction.

Namely, interactions with ResolutionBot resulted in either mood increases or no mood change (Fig. 11), with most of the latter occurring when participants felt positively at the start of the interaction (Fig. 12). Mood increases occurred in 21 out of 73 (29%) of the interactions, while consistent mood occurred in 52 out of 73 (71%) of the interactions. Of the 73 recorded mood changes, none resulted in a mood decrease. Fig. 12 instead shows that all participants starting in a negative mood reported increased end-of-interaction mood, and 19 out of 22 interactions in which participants began with neutral mood, ended with positive mood. This supports the potentials for workplace service robots that positively impact worker experience.

Fig. 11

Overall mood change. Most people’s mood stayed the same (71%), however, when it did change it was always a mood improvement (29%). Of the participants that completed both mood reporting steps (N=73), 21 reported a mood increase, while 52 reported the same mood at the beginning and end of the trial

Fig. 12

Mood change (final mood - starting mood) instances by starting mood. As an example, one participant felt grumpy in the beginning of the interaction, however, their mood changed to happy at the end of their interaction with ResolutionBot. The participant further reported that they had a hard day that morning. No participant’s mood ever decreased after interacting with ResolutionBot

These mood results are consistent with our qualitative analysis. For example, in one case a participant who felt negatively at the beginning of an interaction, said, “It has been a tough day as I had to back up on some paperwork and class assignments as well.” After completing her interaction with the robot, however, she clicked the happy button on ResolutionBot. This change was consistent with observations of her nonverbal behaviors; at the beginning of the interaction, she bluntly answered ResolutionBot and was barely smiling. As the robot kept throwing jokes and suggesting exercise, however, more laughs and smiles appeared. At the end, not only did she smile but also expressed gratitude to the robot, saying, “Thank you, ResolutionBot!”

The Impact of ResolutionBot on Exercise Participation

Interacting with ResolutionBot helped participants’ exercise regularly. Out of 91 attempted visits (13 of which did not occur since the participant was busy), 65 interactions resulted in the participant exercising. Fig. 13 presents the breakdown of each exercise type completed. As exercise types were randomly asked and chosen in each trial, types of exercise were distributed unevenly, e.g., walking was conducted less than other types of exercise.

Fig. 13

Exercise participation summary. Overall, 71% of trials resulted in participants exercising (65 exercises completed, out of 91 attempted visits). jumping jacks (N=20), push ups (N=19), squats (N=18), walk (N=8), and no exercise (N=26)

Our qualitative data also showed positive impacts of ResolutionBot’s health activities on participants’ likelihood of exercising. In interviews, participants answered that they believed their interactions with ResolutionBot encouraged them to exercise at work, saying, “This little guy does make me do some physical movements in the middle of office hours. It is good to see you regularly! (Participant 1)” Overall, participants were willing to follow the workout plan led by ResolutionBot unless they were busy or were supposed to be doing something else during the robot’s visits. When refusing the robot’s exercise suggestions, they always tried to give the robot reasonable excuses, e.g, “I’m sorry, ResolutionBot, I like to workout with you, but really should leave for a meeting now. Can you come back and do an exercise with me later today? (Participant 3)” As participants became more accustomed to ResolutionBot’s health activities and the study, and hence ResolutionBot’s visits, was nearing an end, some participants wanted ResolutionBot to come visit them even despite tight schedules, saying “I don’t want to miss his visits. I should meet him more before he hibernates. (Participant 5)”

The Impact of ResolutionBot on Snack Participation

Our Snack Taking Summary (Fig. 14) shows people’s likelihood of taking snacks during their interaction with ResolutionBot. There are 19 instances in which participants did not take a snack, which is only 21% of the total number of interactions; there are 72 instances during which participants took snacks, which is 79% of the total number of interactions. As for participants’ preferences of snack types, granola bars were a lot more popular than other types of snacks; Fig. 14 shows the distribution of Granola Bar (N=43, 47%), Banana (N=14, 13%), Orange (N=9, 12%), and Apple (N=6, 7%), which shows that participants prefer granola bars more than any types of natural fruit.

Fig. 14

Snack taking summary. 79% of trials resulted in snack taking (72 snacks taken out of 91 attempts). Granola bar (N=43), banana (N=14), orange (N=9), apple (N=6), and no snack (N=19)

Our qualitative data also showed participants’ positive self-expressions regarding snack-taking. Participants seemed to be pleased and surprised by the robot’s snack offers, especially in their first interaction, saying, “Wow I didn’t expect that I would get a reward for push-ups! I like this concept! (participant 2)” In addition, participants were more likely to be attracted by granola bars; When participant 4 took a granola bar after an exercise, a bystander next to her expressed her willingness to join in the study, saying “Oh are you really giving her a granola bar? This is a good study project!”

Participant Experiences

The end-of-experiment survey asked participants questions about their workplace enjoyment, experienced emotions, robot attributions, and health commitment after all of their interactions with ResolutionBot. As shown in Fig. 9, each participant participated in an average of 6 interactions. This section analyzes the effect of health activities: exercising and snack taking and participant’s mood change on participant responses to the end-of-experiment survey questions.

Figure 15 shows the means and standard errors of the participant Likert-scale responses to these questions. Statistically significant correlations between participation categories and survey responses are reported using differently colored symbols: whether participants’ (purple) and (orange) and how much participants’ (green). The following subsections detail how health activity participation affected the workplace environment, participant happiness, health and productivity and participants liking ResolutionBot, as per survey responses.

ResolutionBot Positively Impacted the Workplace Environment

This subsection presents the impact of ResolutionBot’s health activity on participants’ enjoyment of their workplace environment and colleagues, which we analyze via Likert scale survey responses, corresponding to the first two columns of Fig. 15. We discuss the directionality of these results, statistical significance, and use qualitative analysis to support our interpretations.

Mean survey responses (Fig. 15) to the statements, “I enjoyed my workplace more during the ResolutionBot study,” and “I enjoyed my colleagues more during the ResolutionBot study” indicated high participant agreement with both statements. As these statements were rated as part of the exit survey, these participant impressions were formed after 3 weeks of interacting with ResolutionBot. This participant agreement shows that participants thought that having the robot around made their work life better.

Next, we wanted to understand whether a participant’s health activity participation impacted their enjoyment in their workplace. Three one-way ANOVAs analyzing the impact of exercise participation (), snack participation (), and mood change () on these survey responses demonstrated statistically significant positive correlations.

The numerical data shows that participants who had higher exercise and snack participation and who reported a higher mood increase were more likely to have higher Likert responses to these questions. For example, participants who responded with ‘Strongly Agree’ had taken a snack an average of 97% of the times that ResolutionBot had asked them to take one, participants who responded with ‘Agree’ had snack acceptance averages of 82%; whereas the rating of ’Neutral’ had snack acceptance averages of 18%.

Additionally, we wanted to investigate whether a participant’s health activity participation impacted how much they enjoyed their colleagues. A one-way ANOVA analyzing the impact of exercise participation () on this survey question’s Likert responses demonstrated a statistically significant positive correlation. For example, participants who responded with ‘Strongly Agree’ had exercised an average of 95% of the times that ResolutionBot had asked them to exercise, whereas the rating of ’Neutral’ had exercise acceptance averages of 70%. ANOVAs for mood change and snack taking were however not strongly correlated to a participant enjoying their colleagues.

From our qualitative analysis, we see that ResolutionBot increased human-human socialization. Participants seemed more relaxed and comfortable during exercising, as they started smiling after the robot’s jokes while also joking with ResolutionBot in turn. When participants were with other people (both in group interactions and as observers), participants and their colleagues often laughed together. Contributing to a shared enjoyment of their workplace, some colleagues even initiated conversations about what would be a better robot joke while the participant was exercising (Participant 11).

ResolutionBot Impacted Happiness and Productivity

This subsection presents ResolutionBot’s effect on participants feeling healthier, happier and more productive, which we analyze via Likert scale survey responses to the third, fourth, fifth and sixth columns of Fig. 15.

Mean survey responses (Fig. 15) to the statements, “ResolutionBot made me healthier,” “ResolutionBot made me happier,” “ResolutionBot made me more productive” and “ResolutionBot led me to connect with more people,” show that participants strongly agreed with ResolutionBot making them feel happier and didn’t agree with ResolutionBot making them feel healthier and more productive. Our current implementation of ResolutionBot was more successful at fostering happiness than making participants’ feel healthier. The mean in Fig. 15 shows that participants’ slightly disagreed with the statement “ResolutionBot made me healthier”. Some specific implementation choices like sugary granola bars as a snack choice could have impacted this sentiment.

Next, we wanted to investigate whether increased health activity participation might contribute to higher Likert self-reports of happiness. Two one-way ANOVAs analyzing the impact of exercise participation () and snack participation () on survey responses to “ResolutionBot made me happier” showed statistically significant positive correlations. Participants that exercised more reported feeling happier. On the Likert scale, participants that answered the question, “ResolutionBot made me happier” with the highest rating had exercised 100% of the times that ResolutionBot had asked them to exercise. Lower participant ratings on the 5-point scale for this question had lower exercise acceptance averages of 82%, followed by 30% for the neutral rating. Participants that took snacks more often reported feeling happier. On the Likert scale, participants that answered the question, “ResolutionBot made me happier” with the highest rating had taken snacks 100% of the times that ResolutionBot had asked them to take a healthy snack. Lower participant ratings on the 5-point scale for this question had lower snack acceptance averages of 84%, followed by 18% for the neutral rating. ANOVAs for mood change were not strongly correlated to happiness. However, no participant (out of 15 total participants) disagreed with this statement about ResolutionBot making them feel happier.

As a health coach, ResolutionBot offered 4 different exercises to participants. These different exercises seemed to have varying effects on participants’ mood increases (Fig. 16). Walking resulted in the highest mood increase compared to other exercise types, such as squats, jumping jacks and push-ups. A Kruskal-Wallis Test showed a statistically significant difference for the Mood Change among the 4 different exercises, (Fig.16). Due to some difficulties that ResolutionBot faced while walking, participants helped the robot navigate narrow aisles and messy lab areas. This led to unexpected opportunities for extra conversations, such as “You are helping me exercise and I am teaching you how to walk, ResolutionBot!” and “This is fun to walk together with you, ResolutionBot!”.

Fig. 16

The effect of the type of exercise undertaken by the participant on their reported mood change (final mood - initial mood). Despite walking being the only exercise that involved navigational and collision related issues, it resulted in the highest mood change. We believe that this might have been due to the collaborative and organic nature of walking when compared to other exercises that required ResolutionBot to be commanding and counting off participants

Healthier snack offerings such as fruits (e.g., bananas, oranges and apples) given out by ResolutionBot were more likely to increase participant mood. Despite granola bars being the most commonly taken snack, our results showed that people who took oranges and bananas had the highest average mood increase (Fig. 17. A Kruskal-Wallis Test showed a statistically significant difference for the Mood Change among the 4 different snacks, .

Fig. 17

The effect of the snack taken by the participant on their reported mood change (final mood - initial mood). Participants who took oranges or bananas had greater increases in their mood on average

Our qualitative analysis also replicated this tendency. Participants who mostly picked granola bars and rarely selected fruit options expressed mood increases only when they picked oranges or bananas. One instance of this was participant 7, who took granola bars in six out of seven trials. In his third trial, however, he habitually picked a granola bar at first, then looked at the nutritional information and said “hmm, this is too many calories for me this morning. I will go with a banana this time.” He said, “Hey Resolution Bot, can I exchange this?” Similarly, participant 11 who has chosen oranges and bananas most of her trials (four out of five), took a granola bar only once; this was a unique case as granola bar was the most frequent snack choice for other participants. Interestingly, participant 11 reported mood increase in every trials except the one in which she picked a granola bar.

Next, we wanted to delve further into participant sentiment about ResolutionBot not making them more productive. A one-way ANOVA analyzing the effect of exercise participation () on survey responses for this statement showed a negative correlation. In other words, participants appeared to think that the exercise time spent with ResolutionBot took away from their productivity. Interactions lasted an average of 5 minutes and 9 seconds, which could have served as a nice break but was also a literal distraction. Workplace productivity could certainly be an interesting and relevant potential to evaluate further in future work.

Participants Liked ResolutionBot

This subsection presents participants’ non health-related co-presence perceptions of ResolutionBot, which we analyze via Likert scale survey responses, corresponding to the seventh, eighth and ninth columns of Fig. 15. This section also considers the last three statements in Fig. 15, relating to participants’ commitment to mental health, nutrition and fitness.

Being a collaborative work environment, it can be relevant to investigate the effects of a service robot’s co-presence. As can be seen in Fig. 15, participants responded to the statement “The robot was nice to have around” and “The robot was friendly” with high levels of agreement on the 5 point Likert Scale (means of 1.3 and 1.5 respectively). The means show that participants also agreed that ResolutionBot led them to connect with more people and that ResolutionBot was useful.

Mean survey responses indicated participant agreement with considering the robot nice to have around. Two one-way ANOVAs analyzing the impact of exercise participation (, Fig. 15) and snack participation (, Fig. 15) on this question demonstrated statistically significant positive correlations. For example, participants who responded with ’Strongly Agree’ had exercise participation averages of 96%, participants who responded with ’Agree’ had exercise participation averages of 75% whereas the rating of ’Neutral’ had exercise participation averages of only 30%. However, participants’ mood change did not strongly correlate to participants’ finding the robot nice to have around. ANOVAs considering the impact of health activity participation on the other survey questions were not statistically significant.

Over the interactions, participants built an emotional bond with ResolutionBot. Participant 6, who was sufficiently bonded with the robot, expressed her sadness as she was supposed to leave campus and quit her study participation. She gave the robot a good bye card, saying, “don’t get rusty until I come back next year!” Participant 6 also asked the robot to tell her cohort a lyric on her behalf as a way of saying goodbye to ResolutionBot; she also asked the robot to give a farewell message to another cohort. Such bonds were also especially noticeable when interactions were experienced in more collaborative settings.

As a social agent, ResolutionBot reminded participants’ to exercise and take healthy snacks. However, its activities did not seem to translate to increase participant commitment towards mental health, nutrition or fitness. Fig. 15 shows how the participant ratings for “I was committed to mental health,” “I was committed to nutrition” and “I was committed to fitness” (the last 3 columns) had lower means in general, indicating that participants disagreed with these statements.

The ANOVA results and qualitative analysis imply that people enjoy ResolutionBot’s co-presence more when they have exercised more and have taken snacks more often. Increased participation in these helped ResolutionBot be considered ‘nice to have around’. In conclusion, the current implementation of ResolutionBot seemed to be more of a likable social presence, rather than a health activity enforcer. ResolutionBot did not seem to contribute to participants’ commitment towards being more healthy. However, participants that participated in healthy behaviors as asked by ResolutionBot had a greater liking for the service robot.

Discussion

This section discusses broad insights relative to our motivational research questions. The qualitative data and participant survey responses help us imagine a future wherein robots might form an integral part of our workplace experiences, much like delivery robots have now become a common feature of our sidewalks. In particular, we discuss the potential camaraderie benefits of co-present service robots in everyday human workspaces, the viability and benefits of service interaction customization to particular people and workgroup contexts, and the strong role that wizards can continue to play in our increasingly robot-integrated world post-COVID.

Service robots have the potential to positively impact workplace camaraderie: Out of 91 interaction instances, there were a total of 6 groups interactions. According to our analysis and observation, we noticed that group interactions can instigate human-human socialization which increase participants’ enjoyment in the shared workspace and with their colleagues. For example, participant 4 wanted her cohort to do a workout together; she asked ResolutionBot, “Can I ask my friend to do jumping jacks with me?” While she rarely smiled and laughed when she exercised by herself in her other trials, she seemed brighter and more talkative in this group interaction, as noted by the anthropologist. Meanwhile, group interactions also influenced participants’ willingness to engage in harder exercise types such as push-ups and squats. Harder exercise types usually generated participants’ sighs and refusals more than when doing easier exercises such as jumping jacks. However, exercising in groups to the cohorts’ cheering and clapping made participants smile and laugh during harder exercise types (Participant 1), indicating that they enjoyed these more than when doing them alone.

In a similar vein, participants’ lab memberships also appeared to induce more camaraderie, as they easily developed bonding towards the robot and their colleagues. Participants from the same lab group often worked in close proximity. Survey results from Fig. 15 show that ResolutionBot appeared to engender workplace and colleague enjoyment, creating a positive work environment. In addition to direct participants, participants’ neighbors and bystanders also seemed to be positively impacted by observing the service robot’s interaction with a person. In our qualitative analysis, this positive impact was supported by frequent smiles on the onlooker’s faces, memes about ResolutionBot that circulated around the research site (Fig. 18) and onlooker’s willingness to interrupt and participate in interactions. In the future, we believe that human-human socialization and group interactions can be catalysed by service robots in shared workplaces; and that this can be an compelling area for research.

Fig. 18

One of the memes about ResolutionBot that participants created and circulated around the lab. This meme pokes fun at ResolutionBot’s difficulties in getting across wires. People (both participants and onlookers) often helped ResolutionBot across wires and this became a topic of conversation throughout the lab

The ability to customize robot behaviors to particular contexts was tractable to do live; offering insights on how to design both autonomous and human-in-the-loop systems: Both quantitative and qualitative evidence from this study showed that participants reacted differently to the service robot in different social contexts. Some example contexts include when participants are working vs. when they are available, when they are interacting with a group vs. when they are interacting with ResolutionBot solo, when they are being observed by other individuals in their cohort, etc. These various situations can all necessitate different social robot behavior adaptations. Being able to customize the service robot’s behaviors for these participant preferences is likely to help the service robot’s interactions. Some of these customizations could include user preferences for exercises and snacks, current user state (whether they are available, working, etc.) and the interaction’s social context (group interactions, group memberships, being in the middle of a meeting, etc.).

As an example of such a customization, in one interaction, ResolutionBot communicated with a participant using a teleconferencing application on a laptop since this participant was traveling. This complex interaction arose from ResolutionBot attempting to initiate an interaction with a participant that happened to be teleconferencing with another participant. This interaction was interspersed with technical difficulties like doing ‘sitting’ push-ups and the impossibility of eating a snack virtually. Despite these challenges, the interaction proceeded successfully since the wizards were able to customize and adapt ResolutionBot’s interaction flow and design to this new environment. In fact, the intended participant helped facilitate this interaction by propping up the laptop to ResolutionBot’s head. Future roboticists can design human-in-the-loop (HITL) wizarded systems, like ResolutionBot, to allow service robots to adapt to various social contexts.

Not being able to successfully complete an exercise was sometimes embarrassing to the participant. Thus, perhaps future work with robot health coaches could explore how to minimize such anxiety and/or embarrassment, e.g., creating “Reassurance Robots” that teaching positive or compassionate self talk. Another interesting psychological effect was the lower net mood for participants taking the ‘unhealthy’ granola bar (Fig. 17). Perhaps participants felt like they were not living up to ResolutionBot’s expectations or their own goals. Future work in this area can further explore the importance of self-compassion, and how service robots can facilitate lowering self-judgement, in helping people acknowledge and stay with their goals, rather than beat themselves up about their decisions.

The Human-in-the-loop control increased the flexibility and fluidity of interactions: Being able to customize the service robot’s behaviors for participant preferences, live, helps surface some flexibility, e.g., user preferences for exercises and snacks, adapting to current user state (whether they are available, working, etc.) and assessing the interaction’s social context (group interactions, social relationships, being in the middle of a meeting, etc.). Anecdotally speaking, off task conversation also made the interaction more fluid and socially normative, contrasted with traditional robots’ algorithmic dialogue flows. As an example, during one interaction, ResolutionBot stopped and stared at the whiteboard near the students’ desks, commenting “Oh, this math is so easy!”. This got an initial laugh out of some of the participants, serving as an ‘ice-breaker’ to help ResolutionBot initiate an interaction with participants. The improvised, human-in-the-loop robot piloting system also allowed for unique openings and statements. In another interaction, a participant refused to exercise since they had already done ballet a few hours ago. The improvisational wizarding method enabled ResolutionBot to be more interactive and flexible in its conversation with the participant. ResolutionBot did a small little jig (rotated around at various speeds) and asked the participant to teach them ballet dancing. The human-in-the-loop wizarding enables robots to conduct normative conversation and helps us generate and build models for human-service robot behavior.

As a related mini-insight, qualitative analysis suggested that people (both participants and observers) helping the robot led to more collaboration which in turn made people feel happier. Extra work on part of the participants also led to unexpected opportunities for extra conversations, such as “You are helping me exercise and I am teaching you how to walk, ResolutionBot!” and “This is fun to walk with you, ResolutionBot!”

Additionally, the robot making mistakes was not always a big deal, especially when it was associated with its lack of navigational and collision-avoidance abilities. Communication delays, on the other hand, hampered the interaction until recovered by the human wizards. For example, people around the lab often helped ResolutionBot with technical difficulties, like getting stuck on wires, as the research site, being a laboratory, had uneven floors and obstacles that sometimes impacted navigation (Fig. 18). However, communication issues like misunderstood words could have been catastrophic without the wizards in the loop to recover from the situation. This underscores the value of gathering data with a human in the loop so we can see these expectations manifest and be solved as potential inspiration to future programming.

Improvisational wizarding also helps us integrate service robots into a social environment so as to consider the cultural situation of the robot [49, 67, 70]. ResolutionBot’s improvisational wizarding allowed it to maintain conversational fluidity and flexibly react to anything that a participant might do or say. Wizarding helps inform both autonomous applications [53] AND human in the loop systems, which have become pretty prevalent post the Covid-19 pandemic. Maygar et. al. implemented a WoZ interface that would learn from a human physiotherapist wizard in order to increase the level of autonomy of the robot, thereby, using improvisation wizarding as a stepping stone to future full autonomy. In the current world, there’s a multitude of both human in the loop systems and autonomous systems and the type of improvisational wizarding that ResoBot adopted, robot-centric ethnography, can play a significant role in designing contextual guidelines for future service robots in the workplace.

Conclusion

This study involved a three-week in-situ deployment of a wizarded robot health coach in a large shared research facility, in which the robot visited participants during their workday. Our results and discussion offer insights about the social expectations people have of service robots. Parallelling findings in social robotics, we found that our participants expected robots to have social capabilities that are traditionally present in general human-human interaction. For example, the ability to adapt to the unexpected, have off script conversations, build relationships and handle group interactions contributed to the fluidity, flexibility, and social integration of the system.

Our results generally indicate an openness to and enjoyment of workplace service robots, even with their potential for distraction, which bodes well for similar applications in the future. For example, participants reported that the robot was nice to have around and positively impacted the workplace (Fig. 15). In addition, they usually accepted ResolutionBot’s suggestions for exercising and eating healthy snacks, indicating an openness to the service robot application itself. Because of the wizards, the robot was also able to adapt to the unexpected, including customizing the workout program to the particular user.

The positive impacts that participants perceived in terms of enjoying their workplace and their colleagues, underscore the potential value of future workplace robots. The improvisational wizarding and ethnographic interpretations are an important part of this, as anthropology explicitly encourages researchers and participants to work together to identify common values. Being able to customize the service robot’s behaviors for participant preferences can help the service robot’s interactions. This ethnographic approach to designing service robots for human workplaces can allow incorporation of specific workplace/employee related details when customizing the robot’s interactions. For example, jokes might not be appropriate for certain workplaces, like official government buildings, and in such workplace environments, the service robot could be designed to be less humorous.

While this short duration study did not result in a perceived impact on participant health, it did serve as a source of enjoyment, happiness and human-human bonding that could be incorporated in designing everyday service robots, whatever their function. Future work could take a more longitudinal approach to evaluating the impact of social robots in shared workplaces. It may be that a longer study could demonstrate the health value of such a system. In addition, we would expect the cultural and social situation of the robots to be impacted by the longer-term relationships. Depending on the social design of the autonomous or wizarded system, it may be worth evaluating whether and in what situations the social bonding might want to take priority over the prescribed service function, and when (for example, when a person is busy) the robot might take a more efficient approach to the interaction.

This data was collected pre-pandemic, yet the implications of this work are highly relevant to both autonomous systems, as well as human-in-the-loop control systems, which are now becoming more prevalent. For example, autonomous versions of these systems could integrate our ethnographic social insights, such as this study’s finding that participants prefer to do harder activities (like exercising) in groups rather than in a solo interaction with the service robot, by seeking out group interactions in workplace environments. In addition, human-in-the-loop systems often use a combination of autonomy in general and humans where needed, thus the observed research concepts may be able to be used to help refine robot capabilities and interfaces, adding human insight to places where customization was most appreciated, such as the casual conversation that often occurred after the exercises were complete.

In social robotics, knowing what to program is harder than the actual programming, because it can be hard to identify what humans expect and value. Future work can continue to explore how combining improvisational wizarding, traditional HRI analyses, and ethnographic interpretations can rapidly source service robot insights and social expectations with the goal of seeding/improving future robot behavioral designs.