Engagement data of robotic students in a synchronous-hybrid course.

This article includes data on the engagement behaviors of seven robotically-embodied graduate students who attended class with an on-campus instructor, three robotically-embodied classmates, and two physically-embodied classmates; the classmates were not part of the study [1]. Purposive sampling was used to collect data from the target population of robot users. The present data were collected through video recordings of a class session and an online survey. Initial coding of the students' actions was based on an adaptation of the Telepresence and Engagement Measurement Scale (TEMS) [2], which was based on the National Survey of Student Engagement [3,4]. Thematic analysis was used to identify the potential determinants of robotic students' engagement. From these observations, we discovered that robotic students used their bodies differently than what they self-report, and their behaviors were dependent on classroom structure and actions of the instructor.

Data

The present data set contains both self-reported engagement data and classroom observations which acted as a proxy for an objective measure of engagement.

Observed engagement

The data referenced in this brief is presented as a single spreadsheet, which include the following: a codebook of codes and their meanings, examples of behaviors that were attached to that code; codes with timestamps for each participant and from two coders; frequency of codes by participant; and inter-rater agreement for each participant.

Self-reported embodiment, social presence, and engagement

Table 1 includes a summary of the students’ self-reported embodiment, social presence, and total engagement. Table 1 also shows how the individual participants compared to their counterparts. Items were scored based on a 5-point Likert scale.

Table 1

Self-reported embodiment, social presence, and engagement.

PID	Embodiment	Relative Embodiment	Social Presence	Relative Social Presence	Total Engagement	Relative Total Engagement
P1	3.5	High	4.2	High	5.0	High
P2	2.8	Low	3.3	Low	4.5	Moderate
P3	3.0	Moderate	3.9	Moderate	4.0	Low
P4	4.0	High	4.7	High	4.6	High
P5	3.3	Moderate	3.4	Low	3.9	Low
P6	3.3	Moderate	3.7	Moderate	4.3	Moderate
P7	2.9	Low	3.6	Moderate	4.0	Moderate

Sample items

Table 2 includes some representative survey items from the Telepresence and Engagement Measurement Scale (TEMS) [2].

Table 2

Sample items.

Variable	Sample items
Embodiment	“I had a good sense of how I appeared to others.”
Social presence	“I felt like I was with those who were physically present in my class.”
Cognitive engagement	“I tried to connect new information with what I already know.”
Affective engagement	“I had fun in class.”
Behavioral engagement	“I listened attentively to my classmates' contributions during class discussions.”

Experimental design, materials, and methods

Research context

Although not yet mainstream, robots in educational contexts have become increasingly common. They have been used by teachers to work with students who are not collocated [5], [6]. Two of the most compelling aspects of telepresence and mobile robot technologies are the sense of having a body and the sense of being with others. Both of these may be straightforward and consequently taken for granted in the traditional physical context; however, they become complicated in the context of robotic telepresence.

For example, take the following scenarios:

You are seated at a table with your colleagues in your department's conference room.

Your colleagues are in your department's conference room but you are seated at your dining table at home and are piloting a robot in the conference room.

What is so compelling about scenario number two is that seemingly straightforward questions such as “where are you?” and “who are you with?” produce complex answers. While scenario number one begets answers such as “I am in the conference room” and “I am sitting with my colleagues,” scenario number two could produce either “I am at home by myself” or “I am with my colleagues” or a combination of both. The responses one is likely to get for scenario two depend on several factors, such as the form and function of the robots, the adoption of the robot technology, and the identification of the robot as the extension of oneself (or lack thereof). As a result, the sense of being becomes best described as a robotic variation of Schrödinger's cat, both simulanteously in the conference room and not in the conference room.

This sometimes-messy sense of presence brought forth by the inclusion of robots presents unique challenges in educational contexts, instructional practices, and educational studies. For example, measuring student engagement in terms of behavioral outcomes, such as raising one's hand to ask questions in the class [7], is based on the assumption that learners possess a body, are familiar with its function, and can use it to accomplish goal-directed behaviors. But in robotic telepresence contexts, the robotic body that's present may not be one that the user identifies as their own, which may inhibit the level of observed engagement [1]. Teachers are one group that may be affected directly, as their visual assessment of students' behavior in the class is used to make instructional decisions. If these behaviors are not representative of the students' perceptions or intent, then the teacher may be making decisions based on incorrect or incomplete data. Likewise, researchers may be vulnerable to the same problem as teachers when collecting observational data on robotic students. As such, the present study may be used to illustrate the disparities between what can be observed and what is reported by participants in educational contexts that involve technology-mediated presence, also known as telepresence.

Sample

The data being shared in this article comes from a study that examined the impact of having a robotic body on students' engagement in the classroom. The study used a mixed-methods design, which employed survey research and thematic analysis of a video recording of doctoral students’ engagement in a synchronous-hybrid PhD course. The course met synchronously and asynchronously, the format of which alternated each week. For the synchronous meetings, the course used a combination of robots and video conferencing, although the present study focuses specifically on a session with robots. The class consisted of an on-campus instructor, two on-campus students (nonparticipants), and ten online/robotic students (three nonparticipants). The subjects in this study were seven of the robotic doctoral students enrolled in the course. The data were collected during a session when the online students in the class attended remotely by using two types of robots built by Suitable Technologies, the Beam and BeamPro. Data were collected in the middle of the semester to allow the students to become familiar with using the robots.

Technology

The Beam robot has a height of approximately 1.34 m, with a footprint of 0.42 m × 0.31 m, and a diagonal display size of 10 inches. The BeamPro robot has a height of approximately 1.59 m, with a footprint of 0.66 m × 0.51 m, and a diagonal display size of 17 inches. Each type of robot included two cameras, which provided a first-person view and a view of the ground near their robot's wheelbase to assist with navigation. The students were given an orientation on how to use the robot with their computer at the start of the course.

Data collection

The participating students completed a survey that collected their self-reported social presence, embodiment, and engagement. This survey was an adapted form of the Telepresence and Engagement Measurement Scale (TEMS) [2]. The engagement items on the TEMS were created based on previously published work on engagement, such as the National Survey of Student Engagement [3], [4]. The instrument focused on cognitive, affective, and behavioral engagement as they were the three most frequently cited dimensions of student engagement in the existing literature [8]. The survey was administered approximately halfway through the semester.

These students were recorded during one class period at approximately halfway through the semester to capture a typical day. During a typical class session, the students started the class together in a whole group with the instructor and did various housekeeping tasks. They then broke up into instructor-assigned small groups and would reconvene for the whole-class discussion. After the whole-class discussion, each class session would adjourn with small-group discussions.

The class session was recorded with the use of three Axis M1054 cameras mounted on tripods that were placed on three tables arranged around the room like the vertices of a triangle that bordered the center table where the students and instructor were gathered. These cameras were networked via ethernet with an HP ProCurve J9077A switch, and connected to a Windows computer where the Media Recorder 3.0 application was used for recording. The cameras were repositioned during each of the transitions between whole-class and small-group format. Each participant's behavior was reviewed on camera and given a code based on the kind of engagement behavior observed by coders. Coded behaviors were divided by embodiment type, whether it was verbal or nonverbal and whether it was on screen or with the robotic body, and by action types, such as whether it was an attention-seeking or an emotive action. For example, a combination of multiple codes could be used to describe a student who raises their hand on the screen and turns their robot to get the attention of their classmate, as this would involve nonverbal and robotic embodiment, and would be considered an attention-seeking action.

The analysis phase included both open coding and a priori codes generated based on behavioral proxies for the items on the survey. Two coders independently reviewed footage for each participant in the first stage. In the second stage, the two coders shared their codes and discussed differences in interpretation. A third coder was used to reconcile disagreements between the two coders to achieve the target of 85% interrater-agreement [9].

Significance

This data illustrates that even widely adopted instruments utilized in educational psychology studies, such as the National Survey of Student Engagement, may not be accurate in technology-mediated presence contexts, such as when students are using robots to attend class. The present dataset can be used to inform the design of self-reported and observed data in many telepresence contexts, even those without robots. For example, video conferencing, which also involves mediated presence, could be studied with a similar approach as the present study.

Specifications table

Subject area	Education
More specific subject area	Student engagement and educational technology
Type of data	Table (excel file)
How data was acquired	Online survey and classroom video recording
Data format	Analyzed data with a codebook
Experimental factors	Participants who were using either a Beam or BeamPro robot
Experimental features	The associated study measured student engagement in a synchronous-hybrid graduate course with an on-campus instructor, on-campus students, and off-campus students embodied as robots.
Data source location	Public R1 university located in the United States
Data accessibility	Data is included in the supplemental materials associated with the present article, and the related research article:https://doi.org/10.1016/j.compedu.2018.11.008
Related research article	Lei, M., Clemente, I. M., & Hu, Y. (2019). Student in the shell: The robotic body and student engagement. Computers & Education, 130, pp. 59–80.

Value of the data• Illustrates the variation of how students might use their robot bodies in the classroom. • Demonstrates the variability of how robotic students might interact with their robotic and nonrobotic counterparts in educational settings. • Data may be used to assist the development of future surveys and observational protocols for studying human-robot interactions in higher education and other educational settings. • Demonstrates that students' sense of presence may cause discrepancies between self-reported and observable measures of cognitive and noncognitive variables in contexts involving telepresence technologies.