Implementation of learning by design in a synchronized online environment to teach educational robotics to inservice teachers.

Teaching educational robotics is of growing interest in K-12 settings. Yet, immense efforts are needed to move the field forward by framing the teaching of robotics with pedagogically sound theories as well as appropriate instructional design models and strategies. To meet this need, the authors designed and implemented an online educational robotics course for inservice teachers who had little or no prior experience in teaching robotics, by applying instructional design factors as well as teaching and facilitation strategies derived from the learning by design (LBD) framework. Action research employing mixed methods was carried out to examine the effects of instructional design factors implemented in the online educational robotics course. An online survey indicated that the participating teachers increased their self-efficacy in robotics, use of problem-solving and collaboration strategies, and confidence in robotics knowledge and teaching. In addition, by the end of the course, the participating teachers demonstrated sufficient robotics content knowledge. As revealed in their reflective essays, they also developed learning strategies, such as case-based reasoning, sketches, trial and error, and evaluating capacity while completing the robotics open-ended project. Further, they realized the constraints of learning educational robotics online and the benefits of collaboration. This study sheds light on the design components of a robotics course grounded in LBD that are effective for preparing teachers in an online environment to implement robotics in their classrooms. © Association for Educational Communications and Technology 2022.

As Next Generation Science Standards (NGSS Lead States, 2013) in the United States emphasize engineering, technology, and applications of science as one of the disciplinary core ideas, efforts to bring robotics to K-12 classrooms have received increased attention among teachers. However, challenges exist related to creating developmentally appropriate ways of delivering STEM content and the skills necessary for students to succeed in the twenty-first century (Bers, 2008; Bers & Portsmore, 2005; Kim et al., 2015; Rogers & Portsmore, 2004). In particular, the adoption of robotics in the existing K-12 curriculum has faced concerns due to limited teacher knowledge about this field and a lack of teaching strategies. Thus, in order to incorporate educational robotics into K-12 settings, a pedagogically sound curriculum needs to be designed and teachers need to be appropriately trained. Among the growing body of research exploring the role of robotics in education, most studies to date have focused on the effectiveness of robotics tools and specific implementation strategies without considering an instructional design model that can lead to the development of successful robotics curricula (Noh & Lee, 2020).

Educational robotics, when combined with sound pedagogical approaches, promotes students’ thinking skills and knowledge through problem-solving tasks (Dorotea et al., 2021). Since it is generally teachers who design learning activities to promote their students’ target skill sets and content knowledge, it is imperative to establish practical guidelines derived from an instructional design model that promotes teachers’ development of knowledge in robotics, higher-order thinking skills, and meta-level reflective thoughts to support their instructional decisions (Han et al., 2019; Noh & Lee, 2020). Due to the paucity of research in this area, teachers tend to rely on websites or manuals on how to use robotics technically along with sample lesson plans. Although these resources can be helpful, they are rarely designed based on instructional design principles, leaving teachers without pedagogically sound strategies to guide and support their efforts.

Further, while online instruction has historically offered efficient and convenient ways to access learning regardless of learners’ geographical location or timescale (Regmi & Jones, 2020), there is a shortage of research on delivering robotics instruction online. Teachers who are not familiar with robotics lack prior knowledge and self-confidence about integrating robotics in teaching activities (Dorotea et al., 2021). To adequately support and prepare teachers to integrate robotics in pedagogical contexts, it is essential that teacher training programs expose teachers to effective and feasible robotics problem-solving tasks. These challenges have motivated us to investigate ways to incorporate robotics in an online course for teachers with little or no prior robotics experience.

Purpose of the study

The purpose of this study was to (a) extend our understanding of teaching educational robotics to inservice teachers and (b) examine the effectiveness of a newly developed robotics curriculum based on learning by design (LBD). LBD combines a systems design approach with dimensions of problem-based learning (PBL) and case-based reasoning (CBR) (Kolodner et al., 1998) to promote problem-solving and collaboration skills. While this framework has frequently been adopted in traditional in-person classrooms, it is underrepresented in online environments.

The primary aim of our online course was to equip teachers with educational robotics knowledge to support the application of robotics in their classrooms. To evaluate the effectiveness of the course, the study examined teachers’ level of content knowledge, their changes in interest and self-efficacy in STEM and robotics, and the use of problem-solving and collaboration skills related to educational robotics. We also examined teachers’ confidence in their knowledge about and their teaching of robotics because these dimensions have been identified as being significantly correlated with educational robotics knowledge and problem-solving skills (Piedade, 2021). Finally, we explored teachers’ experiences in learning educational robotics in an online environment. To those ends, we sought to answer the following four research questions:

RQ 1. To what extent did the use of the LBD approach in the online robotics course promote teachers’ robotics content knowledge?

RQ 2. To what extent did the use of the LBD approach in the online robotics course promote teachers’ use of problem-solving and collaborative working strategies?

RQ 3. To what extent did teachers demonstrate different levels of interest, self-efficacy in robotics tasks, and level of confidence in their knowledge about robotics and their teaching with robotics after taking the online robotics course?

RQ 4. What were the teachers’ experiences and reflections about learning robotics through the LBD approach in an online environment?

Literature review and conceptual framework

Educational robotics

As a discipline, the goal of educational robotics is to introduce students to robotics and programming interactively from an early age. Robotics in the educational field encompasses applications of robots for learning and teaching; however, a clear definition of the purpose and scope of educational robotics is often missing (Scaradozzi et al., 2019). Because of inconsistent use of the term robotics, we adopted the definition developed by Angel-Fernandez and Vincze (2018):

Educational Robotics is a field of study that aims to improve the learning experience of people through the creation, implementation, improvement, and validation of pedagogical activities, tools (e.g., guidelines and templates) and technologies, where robots play an active role and pedagogical methods inform each decision. (p. 41)

Angel-Fernandez and Vincze’s (2018) definition emphasizes the role of pedagogy, which could cover the use of robotics in education as a learning object, learning tool, and learning aid. Robotics in education is not exclusive to the twenty-first century. For example, Papert (1972, 1980) implemented Logo programming and the concept of computation in education in the 1970s. Since then, educational robotics has become an integral part of education, especially with the growing focus on STEM disciplines and interdisciplinary integration with other subject areas (Malinverni et al., 2021; Ospennikova et al., 2015).

We argue that robotics in education must emphasize the development of not only content knowledge but also interdisciplinary thinking skills, such as problem-solving, teamwork, logical reasoning, and creative design that are necessary across the curriculum. As such, robotics construction kits can be used as a supplementary tool for learning other domains and developing multidisciplinary skills applicable in mathematics, science, and engineering (Benitti & Spolaôr, 2017; Bers et al., 2014; Yuan et al., 2019). The use of such kits encourages learners to construct and program robots to control and solve authentic and multidimensional problems by applying abstract STEM knowledge (Bers, 2008).

However, despite all the available tools, applications, activities, and interest in robotics education, K-12 schools face difficulties in finding teachers who can develop a curriculum using coding, robotics, and computing concepts (Yadav et al., 2016). Especially when it comes to robotics, studies report that teachers have insufficient knowledge to incorporate robotics into their classroom teaching (Benitti & Spolaor, 2017; Mataric et al., 2007).

Dorotea et al. (2021) showed that knowledge and self-confidence are indicators of effective application of educational robotics by inservice teachers to improve their use in students’ learning activities. Because a lack of content knowledge or experience in applying robotics projects may be correlated with low self-efficacy and lack of confidence in teaching with robotics (Piedade, 2021), our primary goal in the present study focused on teacher training, to first help teachers to gain content and pedagogical knowledge.

Several researchers have investigated the effectiveness of preservice teacher education to teach STEM with robotics (Jaipal-Jamani & Angeli, 2017) and to integrate robotics into the classroom (Bers, 2008; Kim et al., 2015; Yuan et al., 2019). Similarly, the present study focused on supporting inservice teachers in learning robotics knowledge through robotics design challenges using problem-solving skills and collaboration, promoting their self-efficacy and knowledge in robotics to bring robotics into their classrooms.

Learning by design as a conceptual framework

Research has shown that robotics enables learners to engage in real-world tasks and construct knowledge through learning-by-doing, which is the premise of constructionism theory (Han & Bhattacharya, 2001; Papert, 1980). That is, to support the knowledge-building process, constructionism promotes learner-centered environments that provide opportunities to investigate, create, and solve problems consisting of real-world tasks (Han & Bhattacharya, 2001). As such, it highlights the “construction” of knowledge by “constructing” external artifacts. However, it fails to explicitly emphasize a critical element in robotics—the design process. This is where LBD (Kolodner et al., 2003) comes in, as an implemented form of constructionism that values the construction of meaning and, at the same time, emphasizes the process needed to create a rich context for learning, where the learner is a designer who creates, programs, or participates in diverse forms of designing (Han & Bhattacharya, 2001).

LBD enables learners to become more responsible for the design process by designing, sharing, piloting, evaluating, and modifying their work through hands-on activities that are aligned with NGSS goals, which aim to promote the engineering design process (EDP) in K-12 education (Ziaeefard et al., 2017). Specifically, the EDP highlights the problem-solving processes that can be used to connect and apply science concepts to the construction of robots through problem identification, research, brainstorming, testing, evaluating, and revising (Grubbs, 2013; Howard et al., 2008; Hynes et al., 2011).

The robotics design process shares many similarities with the EDP (Howard et al., 2008), a cyclical process that includes problem definition, information gathering, solution generation, analysis of the solutions, and test/implementation to develop a sound solution to a design problem (Ertas & Jones, 1996; Lumsdaine et al., 1999). By reflecting on problem-solving processes according to the EDP, teachers will better appreciate how ideas, approaches, and tools are applied in solving design challenges (Purzer et al., 2015) and, therefore, be better prepared to share this knowledge in the classroom.

To ensure effective and engaging robotics-infused lessons in K-12 classrooms, teacher education needs to consider incorporating the design processes discussed in LBD. To date, studies targeting teacher education programs in robotics have been limited, and those that do exist are usually grounded in constructive pedagogical methods rather than the design processes required in robotics (Jaipal-Jamani & Angeli, 2017; Papanikolaou et al., 2008).

Furthermore, given the increased attention to online learning, we aimed to provide instructional guidelines for delivering robotics in online environments. As a type of teacher professional development, the adaptable online robotics course proposed here could expand opportunities to provide authentic learning with robotics and engage teachers in effective online learning of robotics content and implementation of LBD learning theory.

Application of LBD components in the design of an online robotics course

While designing and implementing the online robotics course, we applied “critical components of LBD” (Kolodner et al., 1998, pp. 20–21) to educate teachers about robotics to improve their robotics content knowledge, skills, application, and confidence for future implementation. We categorized the critical components into two parts: Guiding Components for Instructional Design (LBD-A) and Teaching and Facilitating Strategies (LBD-B).

LBD-A: guiding components for the instructional course design

The online robotics course was primarily guided by case-based reasoning (CBR), a unique dimension of LBD (Kolodner et al., 1998), to develop a series of robotics activities (Kolodner et al., 2003). CBR is not a specific strategy applied to a certain task, but is a methodology that promotes overall reasoning and learning through tasks. The use of CBR, therefore, involves presenting diverse learning contexts (Kolodner, 1992) that include the authentic and engaging context of learning. The course developed for this study consists of multiple design tasks intended to serve as cases to enable teachers to develop and apply analogical thinking for subsequent open-ended problem-solving tasks. And finally, a launcher activity, provided in the form of an open-ended real-world problem-solving task, was designed to engage teachers in real-life contexts where they could explore and apply the robotics knowledge and skills they acquired through the cases.

In sum, we selected three critical components for the LBD-A section to guide the overall structure of the course: (a) authenticity, (b) case-based reasoning, and (c) a “launcher” activity—an open-ended final project where teachers were able to apply acquired knowledge and reasoning. Table 1 describes how these components informed our instructional design strategy for the online robotics course.

Table 1

LBD-A: guiding components for the instructional design of the online robotics course and its application

Guiding components for instructional course design (Kolodner et al., 2003)	Application of the guiding components for the online robotics course
(a) Authentic, engaging design-and-build activities that enliven students’ interest in science	We provided authentic tasks to engage teachers in robot design and building activities
(b) Case-based reasoning and use of analogical reasoning with concrete cases to inform design decisions (e.g., experience with toy cars, paper-based cases, online case libraries)	We provided a series (libraries) of purposive robotics activities to promote analogical reasoning in science and robotics concepts to inform future design decisions and applications
(c) “Launcher” activities to introduce students to the science skills and complex cognitive and social skills critical to doing, learning, and applying science	We provided “launcher” activities as the final project to engage students in robot design and functions, coding, and science content concepts

LBD-B: guiding components for teaching and facilitating strategies

For the second part, LBD-B, we implemented the remaining components of LBD as the teaching and facilitation strategies: scaffolding, feedback, group discussion, exploration, and reflection. The seven-week course consisted of three modules of robotics lessons and activities, mainly about gears since an understanding of how gears work is essential and critical to building various types of robots using sensors. Therefore, the tasks in these modules explored gears and robotics structures involving building and coding through design challenges using tangible LEGO kits.

Each teacher was provided with the LEGO WeDo 1.0 kit and guided to download its software from the web on their computer, so they could participate in class from home via Zoom. Teachers were also required to read articles provided each week and the book Lifelong Kindergarten: Cultivating Creativity Through Projects, Passion, Peers, and Play by Mitchel Resnick (2017).

The critical components of LBD-B guided our design of teaching and facilitation strategies to promote teachers’ planning and logical thinking procedures for developing design solutions. The LBD was originally developed in the context of science: yet, we attempted to apply the core components to robotics design tasks, as detailed in the following Table 2.

Table 2

LBD-B: guiding components for teaching and facilitating strategies

Guiding components for teaching and facilitating strategies (Kolodner et al., 2003)	Application of the guiding components for the online robotics course
(a) Multiple contexts for design activities	Teachers were provided with real-world scenario-based activities to design robotics that could elicit multiple design thoughts. The examples from the LEGO WeDo 1.0 kit allowed teachers to build a series of robots and experiments using programming
(b) A balance of constrained, scaffolded challenges with more open-ended design tasks (Crismond, 1997)	Constrained tasks promote the collection of analogous examples to generate ideas based on the features and properties of prior examples (Smith et al., 1993). Open-ended tasks promote novel creations by applying useful approaches and ideas acquired from prior experiences (Smith et al., 1993). A balance between constrained and open-ended design tasks is effective for novice teachers to successfully produce designed artifacts. In order to execute this balanced design successfully, the instructor provided scaffolding strategies such as sketching (Dym et al., 2005) and flowcharting (Chambers et al., 2007; Smetsers-Weeda & Smetsers, 2017). Therefore, the instructor developed various constrained tasks with worked examples and open-ended design tasks for teachers to use cumulatively as cases for developing reasoning about robotics and coding concepts
(c) Rich, varied feedback for designers, through real-world testing, peer and expert analysis of conceptual designs, comparisons to expert model-case solutions, and comparisons between peer designs and extant expert design guidelines	Small groups and whole-class synchronous meetings and discussions were delivered. The breakout rooms and synchronous meetings supported students’ exploration, brainstorming, and ideation (i.e., brainstorming through sketching, flowcharting, robot structure, coding, and designing solutions). Constructive feedback was provided both in a small group and a whole-class meeting
(d) Well-orchestrated approaches to generating classroom discussions and collaborative work, including gallery walks and pin-up sessions for studio-like design discussions (Schön, 1991)	An online gallery walk was realized. After completing robot building tasks, teachers were required to demonstrate their projects online in a synchronous meeting room, share peer and expert analyses of the design, and compare solutions. Teachers had an opportunity to collaboratively reflect on and provide feedback to one another while comparing and contrasting their designs and solutions. The whole class presented their work during the studio-like presentation to share student-generated inquiry, findings, and questions
(e) Experimental and exploratory laboratory work that supports the design challenge and engages students in investigation. Many times, students design and create ways of exploring key science concepts	In the context of robotics, open-design challenge tasks allowed teachers to explore and investigate key robotics and coding concepts while designing and creating a myriad of robotics structures utilizing tangible kits
(f) Support for process thinking and self-monitoring and a strong reflective component through the use of design diaries, “design dials,” reflective discussions, publication of lessons learned, and optional sophisticated, but easy-to-use, software in support of the wide variety of learning, problem-solving, discourse, and understanding activities in which students engage	The teachers were encouraged to draw the design and share it with peers and the instructor. The teachers video-recorded each step of their robot designs and took screenshots of their coding. They shared the design steps, thinking processes, and trial-and-error experiences, and demonstrated their understanding of the structure and core mechanical system of the robot

Table 3 presents the three modules and the final activity based on the LBD-A (Table 1) and LBD-B components (Table 2) that made up the course. Module 1 was delivered for the first three weeks, followed by Module 2 delivered for another three weeks. Finally, Module 3 was delivered in the seventh week along with the final activity for the gallery walk and reflection.

Table 3

Teaching modules based on LBD-A and LBD-B components

Method

Participants

Fourteen teachers (11 female and 3 male) were enrolled during fall 2020 in a Programming, Robotics, and Engineering course. The course is required for the M.Ed. in Curriculum & Learning: STEAM Program at a mid-sized state university in New Jersey that has a long-standing teacher preparation program. The first author as the instructor designed and taught the course; each session was taught synchronously for 2 h and 40 min via Zoom, once a week for seven consecutive weeks. This class time was used for lectures, whole-class discussion or hands-on activities, and small-group discussions and hands-on activities in breakout rooms. The final week included the presentation day for the launcher activity.

All participants were NJ inservice teachers: four high school teachers (three math and one physical science teacher), five middle school teachers (three math and two science teachers), four elementary teachers, and one preschool teacher. The teachers’ race and ethnicity were one Asian, two Hispanic, nine White, and two teachers of mixed race. Their age ranges were: two teachers above 41 years old, one 31–35 years old, nine 26–30 years old, and two 20–25 years old. Twelve of the teachers were currently teaching at NJ public schools (one public school is a charter school) and two teachers were teaching in private schools. Only one teacher had previous experience with LEGO WeDo Robotics via a professional development workshop. The WeDo 1.0 Educational Robotics kits were given out before the robotics sessions started, and teachers downloaded and installed the software on their computers. As used earlier, we refer to the participants in this course as teachers.

Data sources

This study employed action research with a mixed-methods research design (Creswell, 2012) to better understand and answer the research questions using both quantitative and qualitative data, consisting of a robotics content knowledge assessment, an online survey, and a reflection essay assignment. These data sets were analyzed to examine demonstrated robotics content knowledge, self-efficacy, self-confidence, and teaching capabilities in robotics.

Robotics content knowledge assessment

A robotics content knowledge assessment was administered in the seventh week of the course synchronously on Zoom and proctored by the first two authors, with no time limit. The assessment consisted of seven items measuring knowledge about gears, axles, and application of robot structures for a total of 11 points: 1 point for each of five multiple-choice items and a maximum of 3 points each for two open-ended items.

The multiple-choice items asked fundamental concepts about gears, such as the direction of the gears, their speed, and the difference between the number of teeth. The two open-ended items were modified based on the LEGO WeDo™ Resource Set developed as part of the Simple Machines Set (The LEGO Group, 2020). These questions assessed teachers’ problem-solving ability to apply knowledge of axle and gear mechanics using LEGO robot structures. To rate these questions, the first two authors developed the following rubric: 0 (incorrect), 1 (partial understanding), 2 (correct answer with weak reasoning), and 3 (correct answer with clear understanding), leading to a total possible score of 6. The first two authors analyzed and compared responses to each open-ended question together to grade responses.

Survey

At the end of the course, teachers were invited to respond to an online survey consisting of 22 items. The first 16 items were adapted from a study by Jaipal-Jamani and Angeli (2017), in which they modified items from the Robotics and GPS/GIS in 4-H Project (Nugent et al., 2009). These items were designed to assess (a) interest in STEM and robotics (four items), (b) teacher self-efficacy, a self-assessment of the respondent’s own ability to achieve specific objectives related to robotics tasks (three items), (c) use of problem-solving strategies in a robotics context (five items), and (d) use of collaborative skills in a robotics context (four questions).

The next six items in the survey were selected from Sinay et al. (2016) to assess teachers’ self-confidence in their knowledge level of robotics and ability to teach robotics as follows: (a) three questions about confidence in robotics knowledge and (b) three questions about confidence in teaching robotics. All responses were measured using a five-point Likert scale except for the three confidence items, which used a self-rating scale from 0 (no confidence) to 10 (completely confident). The 22 items are listed in the Appendix, along with internal consistency reliability evidence of Cronbach’s α from the 14 teachers who responded to the scale. The overall Cronbach’s α was 0.85 on the pre-assessment and 0.88 on the post-assessment.

To determine changes in participants’ behavior and attitudes after taking the course, two sets of 22 items each were included in the survey for comparison: one set as a retrospective approach, asking about teachers’ perceptions before the start of the course, and the other set asking about their perceptions at the end of the course. A retrospective approach is useful for minimizing the effect of possible overestimation in the pre-survey due to participants’ lack of clear understanding of knowledge and what the pre-survey might ask (Howard et al., 1979; Raidl et al., 2004).

Reflective essays

The prompt of the reflective essay was as follows: “Please describe how the final project was processed, reflecting on the strategies you used, difficulties you experienced, accomplishments you achieved, and how you felt about the project.” The participating teachers were given three days to submit this essay after doing a gallery walk presentation. The directions indicated that the assignment should be two pages long, single-spaced, and submitted via the Blackboard Learning Management System.

Data analysis

Survey responses

To compare differences between pre- and post-responses, due to the small sample size, we applied Wilcoxon signed-ranks tests, a nonparametric counterpart of paired sample t-tests. To determine the magnitude of the differences, we calculated Hedges’ g effect sizes because Cohen’s d statistic tends to overestimate population effect sizes when using a small sample size (Borenstein et al., 2009).

Reflective essay

At the end of the course, all teachers’ reflective essays were downloaded and saved to be reviewed by the first two authors of the study. A thematic analysis was employed to identify, analyze, and develop repeated patterns and themes (Braun & Clarke, 2006) in the text-based data. During the preliminary coding process, emerging categories regarding teachers’ thought processes while doing the robotics projects in an online environment were recorded independently by the first two authors. Later, they met to discuss and compare similarities and differences in the recurring themes. Through this iterative process, categories of teachers’ reflections were developed. The interrater reliability reached 88% of agreement. Disagreement between the two coders was resolved by a second examination and discussions between them.

Results

Teachers’ demonstrated knowledge about educational robotics

The robotics content knowledge assessment measured the fundamental concepts about gears and applications of gears and axles within robot building. The mean of the five multiple-choice items (Questions 1 to 5) asking about fundamental gear concepts was 4.86 (SD = 0.36) out of a total possible score of 5. Twelve teachers answered all items correctly; two answered one item incorrectly. The incorrect item involved the three-dimensional image of gears that were vertically connected as used in the robot models in Module 1; all other items presented two-dimensional gear images that are connected in parallel.

The first open-ended question, Question 6, probed if the teachers applied the gear and axle structure that transmits the horizontal torque of the gear into a vertical torque that drives the wheels, as seen in Fig. 1. The answer that correctly addressed the need for two gears, one crown gear and one leading (spur) gear with two axles, and how the gears should be placed, received the full 3 points. Two points were assigned for partially correct answers, in which the teachers failed to discuss the position of two gears and only mentioned that the two gears needed to interlock. If teachers only mentioned the use of one gear and failed to identify the correct placement, 1 point was given. A completely incorrect answer was graded as 0. The mean of Question 6 was high, M = 2.43 (SD = 1.02) because 10 teachers provided specific descriptions of what type of gears was to be used and how the gears were to be placed.

Fig. 1

Image of the correct gear structure for question 6

The second open-ended question, Question 7, asked teachers to identify the effective combination of the gears to reduce the power required to activate the movement. Eleven teachers received the full 3 points; the remaining three teachers received 1 point, resulting in M = 2.79 and SD = 0.43. Overall, the mean of the robotics knowledge test was M = 10.07 with SD = 1.21 with a minimum score of 8 and a maximum of 11, indicating that, on average, teachers demonstrated a sufficient understanding of robotics content knowledge.

Teachers’ interest, skills use, self-efficacy, and confidence

Interest in STEM and robotics

Table 4 shows the results of the changes in teacher perceptions as a result of participating in the online robotics course. No significant difference was found in teachers’ interest in learning STEM and robotics between pretest and posttest scores. Teachers came with high interest (M = 4.42, SD = 0.43) in taking the course, and their interest stayed the same until the end of the course (M = 4.52, SD = 0.55). The high mean values of teachers’ interest in STEM and robotics indicated that they were highly motivated and that the course kept their interest high until the end.

Table 4

Changes in teacher perceptions of STEM, robotics, strategies, and teaching (N = 14)

Perceptions	Pre-assessment		Post-assessment		Wilcoxon signed-ranks test		Hedges' g
	M	SD	M	SD	Z	p
Interest in STEM and robotics	4.42	0.43	4.52	0.55	− 0.95	0.341	0.31
Problem-solving strategies in robotics	4.21	0.41	4.46	0.50	− 2.46	0.014	0.77
Collaborative working	4.09	0.52	4.27	0.62	− 1.72	0.085	0.48
Self-efficacy in robotics	3.62	1.15	4.19	0.70	− 2.45	0.014	0.69
Confidence in knowledge of robotics	2.10	1.32	4.21	0.53	− 3.20	0.001	1.80
Confidence in teaching robotics	3.17	3.15	7.67	1.74	− 3.20	0.001	1.64

Problem-solving strategies and collaborative working skills

The facilitation strategies provided by the instructor promoted the use of step-by-step analysis, planning, and error-correction strategies, which are directly related to problem-solving in a robotics context. A statistically significant difference for use of problem-solving skills was found between pre- and post-assessments with a moderate effect size of Hedges’ g = 0.77 (see Table 4). This survey finding is also supported by the qualitative data collected from the reflection paper discussed below.

The online robotics course provided a series of group activities to plan, brainstorm, design, discuss, and build together. The teachers preferred engaging in collaborative activities and using collaboration strategies, as reflected by the statistically significant difference with the second-largest effect size, Hedges’ g = 1.64. The significantly improved preference for collaboration was further supported by the qualitative data collected from the reflection papers.

Self-efficacy in robotics

The four self-efficacy questions asked teachers to judge their ability to achieve specific objectives or complete tasks to solve robotics projects. These questions are not specific to their belief in their general teaching ability or knowledge, but are more general about their ability to use educational robotics. The statistically significant difference between pre- and post-assessments indicated that teachers’ self-efficacy increased, with a moderate effect size of Hedges’ g = 0.69 (see Table 4).

Confidence in teaching robotics and knowledge of robotics

There was a statistically significant increase in teachers’ confidence in teaching robotics, with a large magnitude of the effect size, Hedges’ g = 1.64. Teachers’ knowledge of robotics also showed a statistically significant difference between pretest and posttest scores (see Table 4). The largest Hedges’ g, 1.80, indicates that the course had the biggest impact on improving teachers’ confidence regarding knowledge of robotics among the six types of perceptions listed in Table 4.

The significant increase in the level of the teachers’ confidence in their knowledge and ability to meaningfully implement robotics in their classrooms and engage students in in-depth learning indicates that they had acquired sufficient knowledge about robotics. This finding is further supported by the robotics content knowledge assessment on which all teachers achieved high scores without any exceptional outlier (M = 10.07, SD = 1.21).

Teachers’ thought processes based on their reflection papers

Table 5 shows the coding results from the reflection papers along with frequencies and accompanying teacher quotes. The results coalesced into two categories: (a) strategies teachers used and (b) teachers’ reflections regarding constraints in learning in an online environment.

Table 5

Categories and example quotes emerged in the reflection papers

Category	Examples quote	Frequency (%)
1. Strategies used
a. Cases	“This would require us to use a rotating mechanism similar to what we used when making the goalkeeper in one of the tutorials. Instead of moving a goalkeeper back and forth, we would be moving a door” “The Lego goalie had similar movements that mimicked what we needed for our door’s performance: sliding side to side. This past project helped us create our initial design for the motor” “This idea was based on the birds project we did in class, and the fact that we made the birds spin in opposite directions—exactly what my door needed to do”	10 (71.4%)
b. Sketch	“I also sketched a code for each automatic door drawing, hoping to help guide me as I progressed, or at least lend itself to another code if the construction portion did not work out” “I sketched a diagram of a door with ridges or teeth on top that would connect to the gear that would be connected to the motor. I drew the motion sensor on the left to detect when an object/person needed the automatic door to open” “I had three sketches of different automatic doors”	12 (85.7%)
c. Trial and Error	“We were both hopeful that our design would be a success, and we were excited when a simple code moved our door successfully. However, this ‘trial code’ required some adjustments to ensure it opened at the correct speed and would still be able to close” “My partner and I spent most of this time silently trying to figure out a multitude of ways to create our models with the occasional comment. I tried different axles and blocks. Therefore, I was working with having the motor vertically” “Several modifications were made prior to testing. Testing the door necessitated various ‘tweaks’ to ensure the gears engaged effectively”	13 (92.8%)
d. Evaluating capacity	“We came up with two options for a design: The door would either have to swing open or slide open. After analyzing the LEGOs in our possession, we realized that some designs were not feasible. Thus, we came to the conclusion that the second option would be more ideal after taking into consideration the LEGOs available to us” “How do I build an automatic door using LEGOs and the coding program given? My constraints of this project were limited to the pieces I had and my background knowledge from being in the course thus far” “The constraints became evident early in this process. The Lego Kit didn’t have a wide variety of pieces, and we had to work with what we had” “The restriction of the limited number of pieces of LEGOs meant that I had to get creative about the use of LEGOs”	12 (85.7%)
2. Reflections
a. Constraints in learning in an online environment	“We had limited LEGOs available, and we could only work together virtually over Zoom. We realized later that Zoom is a mirrored display, so this made our designs backwards” “I noticed that my partner’s code had been done in reverse since our doors were reflections of each other’s “We were working together through Zoom; we actually ended up with projects that were a reflection of the others”	3 (21.4%)
b. Benefits of collaboration in learning educational robotics	“I built my own model but was fortunate enough to work with a classmate via FaceTime and collaboratively use the engineering design process to construct a functional LEGO door” “I enjoyed working with a partner because it allowed us to bounce ideas off each other, but it is challenging”	4 (28.5%)

The teachers illustrated their thought processes and analyzed the learning procedures they used while completing the robotics activities. They reported using four types of strategies: cases, sketch, trial-and-error, and evaluating ability. Teachers reported how they used the robot-building cases to construct reasoning to solve the open-ended launcher activity and how scaffolding strategies prompted them to tackle the challenges while problem-solving the launcher activity at the end of the course. Specifically, the teachers confirmed that sketching a design, a code, and a diagram helped them design solutions and take the necessary steps to complete the goal (Dym et al., 2005). This reinforces the survey finding of significantly higher scores on the use of problem-solving strategies because teachers eventually learned how to make plans and analyze the steps required to achieve the final goal by drawing the coding flowchart (see Fig. 2) and sketching the design for the robot building (see Fig. 3). They also found that trial-and-error strategies were effective in robotics projects while working collaboratively.

Fig. 2

Examples of coding flowcharts

Fig. 3

Examples of sketching

In addition, teachers gained the ability to evaluate working conditions to conceive of possibilities and constraints. For example, they considered multiple design alternatives and, with a team member, discussed competing constraints before selecting an optimal design. One of the constraints was the limited number of LEGO pieces available to them. Because the course was conducted online, the teachers could not access additional pieces but had to complete the project with the pieces provided at the beginning. Nevertheless, the online environment enabled teachers to develop problem-solving skills to overcome the constraints.

In regard to working robotics in the synchronous online environment, several teachers mirrored the display on Zoom. As a result, in one case, the code was viewed in reverse order from the partner’s view, so the auto door they designed was not moving as planned. The group finally discovered the problem and realized that it was due to the mirrored display.

Discussion

Robotics content knowledge and use of related skills

Robotics education is deeply rooted in problem-solving and creative design processes that encourage students to design solutions, work collaboratively with peers, and address and tackle real-world problems by using tangible tools (Alden & Tramonti, 2020). When bringing robotics into education based on LBD, teachers become designers, who develop solutions and reasoning (Kolodner et al., 2003) to purposefully devise solutions and apply them to computational artifacts, such as robotics and robot coding.

The results of the present LBD-implemented robotics course supported the premise that learning activities that view teachers as designers enhance their learning of robotics content knowledge and the use of skills necessary in STEM contexts, such as problem-solving and collaboration (Kolodner, 1992; Kolodner et al., 2003). The analysis of demonstrated robotics knowledge and participants’ reflection on the robotics projects showed that they used various skills that were discussed as being useful in LBD when tackling robotics tasks, such as analogical reasoning, planning by drawing or flowcharting, utilizing trial-and-error strategies, and evaluating problems (Crismond, 1997; Kolodner, 1992; Kolodner et al., 2003).

Teacher self-efficacy and confidence in educational robotics

The teachers who served as participants of this study also reported significantly increased self-efficacy related to robotics knowledge and confidence in their ability to apply robotics in their teaching while also being able to help students who encountered difficulties in tackling robotics problems. These thoughts were supported by the participating teachers’ high scores on the demonstrated robotics content knowledge. This finding echoes the results of Piedade (2021) in that teachers’ content knowledge and confidence in the knowledge of robotics and teaching with robotics are all correlated dimensions. Especially in STEM disciplines, subject content knowledge in combination with higher teacher self-efficacy and confidence levels is crucial for the successful implementation of educational robotics activities in the classrooms (Dorotea et al., 2021; Schina et al., 2021).

The survey responses and reflection papers further confirmed that the teachers realized a way to apply problem-solving strategies and how collaborative work online could benefit their learning while engaging in robotics projects. Their teaching experiences and knowledge promoted their adoption of strategies in their own classrooms to varying degrees, as reflected in their increased confidence about teaching reported in the post-survey. Moreover, during the breakout discussion sessions, teachers discussed how they might teach robotics on their own without prompts from the instructor and how they might differentiate the strategies they had learned in the course, such as scaffoldings, problem-solving, creativity, and collaborative strategies, and apply these skills in their own classrooms. This confirms previous findings that the implementation of educational robotics activities is effective in promoting participants’ problem-solving, critical thinking, creative thinking, and teamwork-related skills (Eguchi, 2014; Lin et al., 2009; Mauch, 2001). The participants of this study, the inservice teachers, developed skills useful in completing robotics projects and, moreover, critically and creatively analyzed how to apply these skills in teaching robotics.

Teachers’ reflection on their robotics learning and use of strategies

In the reflection papers, the participants described how scaffolding strategies and the use of a series of cases helped them develop understanding and reasoning to solve robotics tasks. This finding corroborated the assertion that when learning is guided by case-based reasoning, teachers are provided with a series of cases that they can use as a resource for evaluating the solutions and applying them in other circumstances (Yuan et al., 2019). As Kolodner et al. (2003) noted, by being presented with analogous cases that engage them in gradually more complex robotics concepts ranging from constrained to open-ended tasks (Crismond, 1997), teachers can learn to adapt old solutions and merge pieces of solutions developed in previous cases. Further, the cases can inform new designs by providing information about potential problems learned from the previous cases.

According to their collaboration strategy scores on the survey, the teachers expressed appreciation for working collaboratively in an online environment, as also reflected in the comments shown in Table 5. Online collaborative work was new to the teachers; however, they found it helpful to discuss ideas together, spend time working with their LEGO pieces, and discuss trial-and-error steps they encountered while being synchronously connected. In short, the collaboration provided opportunities to discuss and share planning steps and ongoing procedures.

This finding echoed that of Eguchi’s (2014) study of an undergraduate in-person educational robotics course. Specifically, Eguchi noted that students who were originally not in favor of working in groups, mainly due to the potential unfairness of group work, gained a new, positive perspective toward group learning through robotics activities. Similarly, in the present study, one student asked to work alone; however, she later reported in her reflective essay that she wished she had a partner to work with: “The only piece missing from my experience was the ability to work with a team. Perhaps I would have spent less time being frustrated at the coding part had I been able to discuss it with a partner. I was, however, able to learn from my failures and implement new possibilities in the process.”

Limitations of the study and directions for future research

It is important to note that the findings reported here are partially attributable to the high motivation teachers brought to the course because they were enrolled in the M.Ed. in Curriculum and Learning: STEAM Program. This also explains the high pre-scores on the interest items. The teachers’ high interest and motivation prior to taking the course could act as a driving force leading to their meaningful engagement in online robotics courses and their evaluation that the strategies could help the instruction of their target audiences.

The goal of the study was to investigate real-world settings rather than controlling participants in a lab setting. The small sample size and the lack of a control group may limit the generalization of the study. It is recommended, therefore, that future studies be conducted with a larger sample and with diverse learner groups, including a control group. Replication of this study with different teacher participants could also help extend the generalization of the findings; nevertheless, as action research, the present study illustrates how to design an online educational robotics course for meaningful teaching practice and student learning (Mills, 2011), while showing that the design of robotics activities based on appropriate learning theories can benefit teachers in terms of their content knowledge, confidence, and teaching skills. A future study could also involve teachers’ actual lesson plans or activity designs pertaining to educational robotics in their classrooms.

This study did not employ a pretest for the robotics content knowledge; therefore, a comparison between pretest and posttest scores was not conducted. However, the teachers showed increased confidence in their robotics knowledge and achieved high scores on their one-time knowledge assessment. It is recommended that future studies include pre- and post-assessments. Despite the lack of a pre-content knowledge test, the qualitative findings of the study provide practical guidelines and components to consider when designing robotics courses based on LBD, especially in an online setting.

Recommendations for teacher professional development

The results of this study have implications for practice, specifically related to professional development. To increase teachers’ knowledge of and ability to use educational robotics with their students in K-12 classrooms, it is important to introduce robotics in teacher professional development programs. As shown in this study, although teachers may be struggling to learn educational robotics at first, the LBD approach provides ample opportunities for viewing worked examples, planning using appropriate tools, brainstorming, and reflecting with peers and the instructor, while developing knowledge and confidence.

Online instruction is an effective tool for overcoming time and geographic constraints, providing ongoing support, and bringing in content with emergent tools, knowledge, and skills. The findings from this study provide insights into how to teach educational robotics in an online environment effectively when the projects are based on a framework that ensures active and engaging learning. This suggests that the use of online learning modalities can be a prudent option for instructors wishing to continue their professional development in subjects involving hands-on learning such as educational robotics.

Thus, online courses for inservice teachers should be designed to increase their efficacy and confidence in robotics, not just robotics content knowledge, for future application of robotics in their classrooms. As Eguchi (2014) suggested, educational robotics help promote 21st-century skills among young students. It is time, therefore, to move forward to implement educational robotics in teacher education programs in the United States so future teachers can prepare children for the workforce, including the knowledge and skills needed for the twenty-first century.

In conclusion, the authors hope that this work provides a feasible option for engaging and preparing teachers to enhance their knowledge and confidence in educational robotics to design and implement instruction for their K-12 students.

Construct	#	Item
Interest in STEM and robotics (Cronbach’s α = 0.58 on the pretest and 0.92 on the posttest)	1	I like using scientific methods to solve problems
	2	I think careers in science, technology, engineering, or math are interesting
	3	I like learning about new technologies like robotics
	4	I would like to use robotics to learn mathematics or science
Self-efficacy in robotics (Cronbach’s α = 0.75 on the pretest and 0.76 on the posttest)	5	I believe that I could work with a robot in a science investigation
	6	I am confident that I could learn how to make a robot do something that I had not done before today
	7	I would use robotics in my classroom teaching
Problem-Solving Strategies (Cronbach’s α = 0.55 on the pretest and 0.84 on the posttest)	8	I use a step-by-step process to solve problems
	9	I make a plan before I start to solve a problem
	10	I try new methods to solve a problem when one does not work
	11	I carefully analyze a problem before I begin to develop a solution
	12	In order to solve a complex problem, I break it down into smaller steps
Collaborative working (Cronbach’s α = 0.35 on the pretest and 0.73 on the posttest)	13	I like listening to others when trying to decide how to approach a task or problem
	14	I like being part of a team that is trying to solve a problem
	15	When working in teams, I ask my teammates for help when I run into a problem or do not understand something
	16	I like to work with others to complete projects
Confidence in knowledge of robotics (Cronbach’s α = 0.90 on the pretest and 0.77 on the posttest)	17	I have sufficient knowledge about robotics for use in teaching and learning at the beginning of the robotics course
	18	I have sufficient knowledge of LEGO WeDo coding as it applies to robotics at the beginning of the robotics course
	19	I have sufficient knowledge of the engineering design process as it applies to robotics at the beginning of the robotics course
Confidence in teaching robotics (Cronbach’s α = 0.87 on the pretest and 0.94 on the posttest)	20	At the end of the robotics course, I feel confident that I had the skills necessary to use robotics for classroom instruction
	21	At the end of the robotics course, I feel confident that I could engage my students to participate in robotics-based projects (using LEGO WeDo)
	22	At the end of the robotics course, I feel confident that I could help students when they have difficulty with robotics