Piloting an E-Learning Applied Behavior Analysis Course for Caregivers of Children with Autism in the Czech Republic.

The telehealth model can allow for cost-effective supports in areas where limited applied behavior analysis (ABA) services are available. As a result, e-learning programs for parents of children with autism have become part of the telehealth paradigm. In the Czech Republic, one of the many places where there are limited ABA supports and no available ABA e-learning courses for families, the development of an e-learning course that is freely available, and linguistically and culturally appropriate, is needed. It was anticipated that this course could mirror some of the free e-learning materials available to English-speaking caregivers and provide foundational knowledge to later support more intensive ABA services. Therefore, a pilot study was developed to assess an ABA e-learning course for caregivers in the Czech Republic with the aim of validating its efficacy and assessing whether a synchronous component was needed for the best outcomes. The materials were not meant to replace best practices in more intensive caregiver training but provide foundational skills for later more successful local services. Results demonstrated that a total of 33 caregivers completed the course, making statistically significant gains in their ABA knowledge and rating the course highly. Interestingly, the addition of the synchronous component only impacted the knowledge gain outcome. Suggestions for supporting caregivers in e-learning course completion and directions for future research are additionally explored.

Introduction

Globally, it has been demonstrated that families of children with disabilities are typically met with increased health and economic demands as well as decreases in quality of life (e.g., Lin et al., 2009; Schertz et al., 2016). This was further brought to light during the Covid-19 pandemic where extended time at home revealed that families were struggling with behavioral issues and increased household stress (Shorey et al., 2021). It was clear from Shorey et al. (2021) review that more needs to be done to support families of children with autism and related neurodevelopmental disabilities (NDD) through educational endeavors that explicitly train parents how to implement strategies. Since applied behavior analysis (ABA) is an empirically validated science that can support the needs of children with autism (e.g., Virues-Ortega, 2010), caregiver education on strategies from the science of behavior are important. However, access to needed ABA-based parent training services can be scarce in many parts of the world (e.g., Kingsdorf & Pancocha, 2020). This can result in caregivers remaining uninformed about best practices, turning to unvalidated approaches, or spending extended periods of time on service waitlists. To meet the needs of families in areas with limited services, given restraints due to locale, funding, or a pandemic, parent training services provided via telehealth technologies have surfaced (e.g., Gerow et al., 2021). These telehealth services have aimed at supporting the use of ABA in the home through e-learning or education for caregivers (e.g., Dai et al., 2018), coaching for real-time applications of interventions (e.g., Marino et al., 2020), and even the direct delivery of services by interventionists from afar (e.g., Ferguson et al., 2020). There have been increases in telehealth usage for families of children with autism in certain parts of the world (e.g., White et al., 2021). However, in other areas, like Europe, there are still lags in access to telehealth, and ABA in general (Kingsdorf & Pancocha, 2021). Creating increased access to telehealth services for families of children with autism is necessary. Even resources that focus on parent education, as opposed to training, can help provide a foundation that is likely to support better service outcomes, and give increased options to caregivers struggling with finding validated information and services.

Validated ABA Telehealth Services

In recent years ABA-based telehealth services have demonstrated their ability to deliver treatment at a lower cost (Lindgren et al., 2016), change parent behavior (e.g., Yi & Dixon, 2021), and train individuals to deliver behavior analytic interventions (Tomlinson et al., 2018). Recent comparisons have been made between e-learning and in-vivo ABA training for parents, as in the work of Blackman et al. (2020), and demonstrated comparable results in the learning outcomes (i.e., parent–child interactions and knowledge of ABA) of participants in both groups. The works of Dai et al. (2021) and Yi and Dixon (2021) have provided further evidence of feasibility and acceptability for ABA-based parent services at a distance by increasing: parent knowledge in ABA as demonstrated through curriculum-based assessment gains, parent self-efficacy, and parental adherence to training protocols. With this service delivery model growing in popularity, new guidelines for ABA telehealth service delivery are being suggested to support treatment integrity (e.g., Rodriguez, 2020). However, to date, no one standard model of service delivery has been established; especially not in Europe.

ABA-based telehealth services can take on several different formats. The two main categories are divided based on time, presence, and mode of participation. The first category is synchronous, or real-time support, which can involve one-on-one caregiver and practitioner meetings or one-on-one service delivery with videoconferencing or audio-only meetings between the ABA practitioner and the child. This synchronous didactic support involving instruction or coaching may use real-time video conferencing tools like Skype (e.g., Pantermuehl & Lechago, 2015), FaceTime (Boutain et al., 2020), and the Google platform (e.g., Pennefather et al., 2018). The second category is asynchronous support, where the practitioner and child or caregiver do not meet at the same time. Under the asynchronous model there may be the sending and receiving of videos or electronic learning materials. This model can look like teaching, training, or education, with the transfer of information sometimes using common file storage applications like Dropbox for video sharing (e.g., Neely et al., 2020) or store-and-forward technologies requiring low bandwidth where video recordings from live events and historical data are shared via a secured platform (e.g., Smith et al., 2017). To summarize, telehealth is considered to encompass training, teaching, or coaching services provided using online delivery and/or two-way audio-visual communication (Rispoli & Machalicek, 2020). Therefore, training parents via an e-learning platform in services related to behavioral health is a type of telehealth. This model can aim to increase knowledge of ABA concepts and/or aim to increase practical skills (e.g., functional analysis, functional communication training).

While there is some research to support all these variations for service delivery, little work has been done comparing e-learning telehealth methodologies to determine which generate the best outcomes. In fact, as demonstrated in reviews of the telehealth literature, it is typical that a combination of asynchronous and synchronous technologies is employed (e.g., Kingsdorf & Pancocha, 2021). Furthermore, most of the work has looked at using service delivery models that involve some telehealth services component which is being delivered directly to the child (e.g., Wacker et al., 2013). While there are certainly benefits to systematically including the child in the services, this is not always possible. When access to telehealth-competent practitioners is limited because of a region’s specific linguistic and cultural needs, telehealth services must take more of a training and education approach. This education approach could also set the stage for more effective services down the line when they become available to caregivers, as we know that caregiver education in ABA impacts the successes of their children, remote or in-person (e.g., Bearss et al., 2015; Strauss et al., 2012). Consequently, ABA parent education services, even at the remote level, are considered an important component of an ABA telehealth model (Pollard et al., 2017). As a result, numerous teaching and training strategies for reaching families from a distance have surfaced.

Supporting Strategies in ABA Telehealth Delivery

Teaching and training strategies in ABA telehealth delivery, which focus predominantly on educating caregivers, have included components such as providing parents with video recordings (Dai et al., 2018), using telephone booster sessions (Bearss et al., 2018), scripted role-play over a virtual private network (Fisher et al., 2020), and developing specialized e-learning content using common learning management systems (e.g., Moodle; Roberts et al., 2019). As previously mentioned, telehealth research involving training parents to implement specific strategies with their children during distance sessions often incorporates multiple strategies. Likewise, telehealth focusing on education and changes in caregiver-only behaviors have commonly involved treatment packages. In the work of Pennfather et al. (2018), real-time video meetings were added to online asynchronous instruction. Phone interviews were similarly added to the asynchronous training program evaluated by Dai et al. (2018). These synchronous components proved valuable, demonstrating favorable changes in parental stress and knowledge (Dai et al., 2018; Pennefather et al., 2018). Additionally, any research that has focused on making telehealth training comparisons between the intervention and waitlist control groups, as in the works of Dai et al. (2020), Marino et al., (2020), and Jang et al., (2012), has demonstrated that e-learning for caregivers is more effective than nothing or services as usual; with caregivers benefiting from e-learning as increases in ABA knowledge, improvement in the perception and management of their children’s behaviors, and the reduction of parental stress. In the Czech Republic, where limited ABA services are available, it is anticipated that similar outcomes would be observed. However, it has not yet been investigated whether an e-learning course in ABA that is culturally and linguistically appropriate for Czech caregivers can result in increases in knowledge, be perceived favorably, and have the same outcomes with or without synchronous components.

Research Aim

Telehealth can supplement or allow for cost-effective services in areas where more intensive ABA intervention is not currently available. While e-learning programs for parents of children with autism do exist on the internet through companies like Relias Academy (Relias Academy & Inc., 2021), for example, these are available in English, with the population of the United States in mind. As previously noted, ABA services in general, and telehealth specifically, are less available in Europe. Despite the demonstrated need for ABA services in the Czech Republic (Kingsdorf & Pancocha, 2020), there are no available ABA training courses for caregivers. In fact, the emergence of ABA in the Czech Republic is quite new. There were no behavior analysts in the country before 2016. However, since then, the field has grown exponentially by establishing ABA professional organizations, holding ABA conferences, training various professionals in ABA, and passing laws surrounding the profession and its practice. There are currently 19 BCBAs in the country who are working to make ABA services more available to families in need. However, they still struggle with large caseloads and reaching those outside of major cities. Therefore, the decision was made to begin the development of ABA-based telehealth services for families in the country through piloting a freely available, linguistically, and culturally appropriate, e-learning course. The aim of the study was to fill this need for initial services, validate the efficacy of the e-course, and assess whether a synchronous component was needed for the best outcomes. Consequently, the decision was made to conduct a pilot study on an open access university run parent training course, making comparisons across e-learning asynchronous and synchronous variations.

Method

Participant Recruitment

Caregivers of children with NDD were recruited for participation in the study. The e-learning course was advertised in the local language on the websites of the Center for Applied Behavior Analysis at Masaryk University (www.ped.muni.cz/caba) and at the website of a local autism organization Modry Kruh (Blue circle) (http://modry-kruh.cz).

Participants

One hundred and seventy-three potential participants responded to the initial advertisement for the e-learning course. However, not all of those who responded were deemed eligible to participate (i.e., not all of them were the primary caregiver to a child with autism, some were teachers or other professionals). For inclusion, the participants needed to be the primary caregiver of a child with a diagnosis of autism (as per self-report) and have no prior experience with ABA. Due to the Czech Republic’s limited availability of ABA professionals to provide support throughout the course, it was decided that a maximum of 60 caregivers could be accepted for the study. Therefore, 60 randomly selected eligible caregivers were invited to complete a further battery of intake materials (i.e., the demographic questionnaire, EIPSES, and BPI-01). The caregivers then provided consent for study participation, by completing a form which described the study, risks, and their rights. Following consent, participants were then randomly assigned to either: (1) Group 1, the asynchronous e-learning course with one-on-one online synchronous meetings (consultations) throughout or, (2) Group 2, the asynchronous e-learning course only, and sent an email with an overview of the course, instructions, and a password to take the first pretest. However, they were not assigned in equal numbers. One third was randomly assigned to Group 1 and two thirds were randomly assigned to Group 2. Therefore, 20 caregivers were assigned to Group 1, and 40 caregivers were assigned to Group 2. The decision to make the group with the synchronous component smaller was made based on the limited number of local practitioners available to provide the synchronous support (consultations). Of the 60 caregivers that were assigned the e-learning course, only 14 caregivers from Group 1 and 19 caregivers from Group 2 completed the course. It is unclear why only 33 of the initially invited 60 caregivers logged into the e-learning course. However, it is hypothesized that the course taking place during the summer, with many people being on holiday, impacted their ability to access the learning material. It is also possible that those who were initially interested were not fully aware of the service format until receiving information on how to access the material. Additionally, deadlines for participation in the course were set and access closed after a certain window, so those who did not start on time where unable to join later. This article presents and analyzes the data for those 33 participants. They were all living in the Czech Republic, had a native language of Czech, and identified as female. Most of the participants were also married (91%), believed themselves to be in the middle-income range (85%), and had a male child with a diagnosis of autism (76%). Additional participant details can be found in Table 1.

Table 1

Participant demographics

	Group 1	Group 2	Combined
	n = 14 (42.4%)	n = 19 (57.6%)	N = 33 (100%)
Parent characteristics
Parent age, average in years (SD)	38.1 (6.6)	37.4 (4.0)	37.7 (5.2)
Parent gender—females	14 (100%)	19 (100%)	33 (100%)
Mother’s education
Vocational certificate	1 (7%)	0	1 (3%)
Completed high school	7 (50%)	4 (21%)	11 (33.3%)
Bachelor degree	1 (7%)	0	1 (3%)
Master or advanced degree	5 (36%)	15 (79%)	20 (60.6%)
Marital status
Married/living with a partner	12 (85.7%)	18 (95%)	30 (90.9%)
Divorced
Single parent	2 (14.3%)	1 (5%)	3 (9.1%)
Income level
Low	3 (21.4%)	2 (10.5%)	5 (15.2%)
Middle	11 (78.6%)	17 (89.5%)	28 (84.8%)
High	0	0	0
Child characteristics
Child age, average in months (SD)	68.8 (71.5)	60.3 (40.2)	63.9 (54.8)
Child gender
Males	12 (85.7%)	13 (68.4%)	25 (75.8%)
Females	2 (14.3%)	6 (31.6%)	8 (24.2%)

Parental age is presented in years; child’s age is presented in months

E-Learning Materials

The e-learning course was hosted by Masaryk University in the Czech Republic. The University’s equivalent to an institutional review board approved the course and the research plan prior to enrolling caregivers. The University operates a proprietary learning management system (LMS) for accredited courses. However, it is also open to the public, allowing anyone to create an account to access certain publicly available courses.

The materials uploaded into the platform included: video recordings of audio-visual presentations of a behavior analyst delivering a lecture using a PowerPoint presentation, supplemental materials (e.g., locally created short videos and reports), activity sheets, and curriculum-based assessment (pretests and posttests). The activity sheets and assessments were all created in Google Docs and then uploaded into the system (as PDFs or individual test questions within the platform’s test software). Examples of the activity and homework sheets are provided in Appendix A. Questionnaires that were part of the assessment battery were created using Google Forms. The synchronous online meetings were done outside of the LMS and held using Zoom. Although the Health Insurance Portability and Accountability Act (HIPAA) is not a concern for providing ABA services in the Czech Republic, Zoom does comply with requirements to support the confidentiality of participant information (Zoom, 2021).

E-Learning Modules Development

The online content for both groups was the same. The course contained eight learning modules on the topics of: (1) introduction to ABA, (2) reinforcement, the three-term contingency, and preference assessment, (3) pairing, conditioning, and instructional control, (4) functions of behavior, (5) prompting, (6) naturalistic learning and motivation, (7) communication, and (8) ABA services and telehealth. More details on the subject matter of each module can be found in Table 2. The content for the e-learning course was created based on existing parent training materials. The researchers reviewed online training materials for caregivers available in the English language through organizations such as Relias Academy (Relias Academy & Inc., 2021), the Autism Focused Intervention Resources and Modules (AFIRM) from the National Professional Development Center for Autism Spectrum Disorders at the University of North Carolina at Chapel Hill (National Professional Development Center, n.d.), and Boston Medical Center’s autism program (Boston Medical Center, 2021). The above materials were chosen for review since they were the freely accessible ABA parent training materials most widely available to English-speaking caregivers. A goal of this e-course pilot was to model similarly free and widely available materials but make them appropriate for Czech caregivers. The RUBI Network Parent Training curriculum (Bearss et al., 2018) was also reviewed. The decision was made to review this curriculum as it had been previously used with caregivers in the region. After the review, a list of topic areas was created by the head behavior analyst (first author). It was then presented to the local Czech ABA practitioners for an informal focus group. Via a video conference, the second author met with the group of local ABA practitioners and presented the list of topics. The topics were discussed, and the group was asked to evaluate whether the topics were part of the training normally provided to caregivers. A goal of developing this e-learning course was supporting the existing behavioral practices of local behavior analysts. Therefore, their input guided the development of curricular targets for the course. They selected the topics which most aligned with their training practices in home and clinic-based settings. The head behavior analyst then created materials on the eight chosen topics in English. The head behavior analyst had been part of the local behavior analytic community for over three years but was not fluent in the Czech language. Therefore, the materials were later reviewed, edited, and translated into the local language by local behavior analysts and PhD students in behavior analysis. During the revision and translation processes steps were taken to support the cultural relevance of the materials. The local behavior analysts and students used example scenarios that reflected common activities engaged in by Czech families, ensured that names and materials reflected those traditional in Czech culture, and adjusted terminology that was not logical when directly translated. For example, sample videos represented ethnicities of children in the region (e.g., Czech, Slovak, Roma, and Vietnamese), used traditional food items (e.g., koláče, a traditional Czech cake, as opposed to an American treat like a brownie), showed children engaging in preferred toys or activities seen in the region (e.g., sitting and listening to fairytales, playing with toys of traditional Czech characters such as Krtek as opposed to American characters from Sesame Street), and seeing challenging behaviors in the context of traditionally targeted early-childhood skills (e.g., changing clothes and shoes independently when arriving at the kindergarten).

Table 2

Parent E-learning module content

Module	Module title	Module content	Activities completed during the video presentation	Homework activities
1	Introduction to applied behavior analysis (ABA)	The meaning of behavior in the context of ABA The definition of ABA Applications of ABA	Selecting observable behaviors Identifying misconceptions about ABA	Observing and collecting data on their child’s behavior
2	Reinforcement, the three-term contingency, and preference assessment	The three-term contingency (antecedent-behavior-consequence) relationship Positive reinforcement Negative reinforcement Reinforcers Preference assessments The difference between reinforcement and bribery Guidelines for using reinforcement effectively	Diagramming three-term contingencies Selecting reinforcement contingencies in applied examples Selecting potential reinforcers in a scenario	Selecting potential reinforcers for their child
3	Pairing, conditioning, and instructional control	Using reinforcement Pairing A procedure for implementing pairing A procedure for gaining instructional control The relationship between using preference assessments, pairing, and gaining instructional control What it means to be contingent	Describing statements to be used during pairing procedures Describing statements to be used when working to gain instructional control	Charting their own behavior during play sessions with their child
4	Functions of behavior	Strategies for preventing challenging behaviors The main functions of behavior The use of extinction as planned ignoring Additional uses for, or types of, extinction	Hypothesizing the function of a behavior Stating the steps of using extinction	Developing and implementing an individualized plan for responding to one of their child’s challenging behaviors
5	Prompting	Functions of behavior Functionally equivalent replacement behaviors The meaning of skill acquisition Where prompts fit into the three-term contingency Different types of prompts The process for using prompts	Selecting prompts	Developing and implementing an individualized plan for teaching their child one new skill
6	Naturalistic learning and motivation	The definition of motivation in the context of ABA Assessing motivation The concept of incidental teaching The concept of natural environment teaching (NET) Different types of prompts The process for using prompts in the natural environment	Selecting teaching targets Describing how to capture a natural learning opportunity	Developing and implementing an individualized plan capturing natural learning opportunities with their child
7	Communication	Functions of behavior Functionally equivalent replacement behaviors The ABA approach to language development/the concept of verbal behavior Mands The process of teaching mands The procedure of using NET Selecting functional mand training targets	Charting the missing components of a mand Selecting targets for functional communication training	Developing and implementing an individualized plan for teaching their child one new mand
8	ABA services and telehealth	The definition of synchronous ABA telehealth services The definition of asynchronous ABA telehealth services The technology needed for accessing ABA telehealth services The time commitment needed for accessing ABA telehealth services The behaviors that are appropriate for accessing ABA telehealth services The safety of ABA telehealth services The process of receiving ABA telehealth services	Assessing compatibility with telehealth services	Assessing their child’s compatibility with telehealth services

E-Learning Modules General Content

There was a total of eight modules which built upon one another and needed to be completed in order (i.e., the next module did not open to allow access until the posttest in the previous module was completed). Caregivers took about one week to complete each module. Each module contained a pretest, overview of module content, video presentation, worksheets, homework, posttest, and additional resources as a copy of the PowerPoint presentation and supplementary local materials. Caregivers moved through the modules at their own pace, but they were only available for 10 weeks. Caregivers did not receive any individualized feedback on asynchronous coursework except the numerical score on the quiz at the end of each module and did not have any other data collected on their behaviors.

More specifically, the caregivers moved through the modules by selecting one of the eight modules within the course overview page and seeing: the module title, a link to the pretest, an overview of the module content, a link to the video presentation, a link to download the PowerPoint only version of the presentation, links to worksheets that needed to be printed and used while watching the video presentation, links to any supplementary local materials, links to the homework worksheets, and a link to a posttest. An example of the module format is provided in Fig. 1. To begin each module the caregiver would click on the link to the pretest, complete the pretest, then move on to the learning material. They would download the worksheets that accompanied the video and then watch the video presentation. During the video presentation a local practitioner presented the learning material in laymen’s terms, using examples relevant to Czech families. At certain points within the video the presenter would stop, and after explaining the concept and providing examples, ask the caregiver to pause the video and complete an exercise (e.g., diagram an example behavior use an ABC data collection form). The caregiver would restart the video and review the answer to the exercise. At the end of each video presentation the caregiver was instructed to complete a homework assignment with their child using a skill taught during the session (e.g., observing their child and making a list of potential reinforcers). The activities and assignments were specific to each topic. Activities included: selecting observable behaviors, identifying misconceptions about ABA, diagramming three-term contingencies, selecting reinforcement contingencies in applied examples, selecting potential reinforcers in a scenario, describing statements to be used during pairing procedures, hypothesizing the function of a behavior, stating the steps of using extinction, selecting prompts, selecting teaching targets, describing the steps in capturing a natural learning opportunity, charting the missing components of a mand (request), selecting targets for functional communication training, and assessing compatibility with telehealth services. Homework assignments also accompanied each module, where the caregivers were expected to apply the material that they had learned after completing the online module. Homework assignments included activities such as: observing and collecting data on their child’s behavior, selecting potential reinforcers for their child, charting their own behavior during play sessions with their child, developing and implementing an individualized plan for their child’s challenging behavior, teaching a new skill or mand, capturing a natural learning opportunity, and assessing their child’s compatibility with telehealth services. More details on the module topics can be found in Table 2.

Fig. 1

Example of the module presentation in the e-learning system

Group 1: Asynchronous E-Learning with Online Synchronous Meetings (Consultations)

The group of caregivers assigned to the asynchronous group with the online synchronous meetings (consultations) had the most individualized support. One-on-one video meetings, or consultations, were scheduled at the end of each module with a local ABA practitioner. The local practitioners (all Czech or Slovak nationals with native languages of Czech or Slovak) had completed graduate level coursework in behavior analysis, been working in the field of behavior analysis for at least 2 years, and were either certified in behavior analysis or working toward certification. Since there was a total of eight modules in the series, a total of eight synchronous meetings were held. Each meeting lasted about 45 min. The aim of the meetings was not to teach the material. Rather, the sessions were meant to function as opportunities for the material to be reviewed, homework assignments to be discussed, and questions to be answered. During video meetings, the practitioners followed a fidelity checklist composed of goals that was tailored to each module. An example of a checklist can be found in Appendix B. All the checklists, and resultantly all the synchronous meetings, followed the same general format: the ABA practitioner briefly reviewing each of the main topics from the module, reviewing the activity and homework assignments, checking for additional questions, and assigning any needed tasks (e.g., completion of a missing homework assignment). However, to support individualization, they were largely guided by the needs of the caregivers (e.g., if during the consultation session on naturalistic teaching the caregiver wanted to discuss their child’s specific behaviors that were targeted for teaching, the practitioner supported them).

The ABA practitioner completed the fidelity checklist during the session to self-assess their own behavior during the sessions by rating the degree to which synchronous meeting goals were attained. Scoring options included: 0 points = goal was not achieved, 1 point = goal was partially achieved, 2 points = goal was fully achieved, N/A = goal was not covered in the session. Following each session, a fidelity score was calculated by summing the number of points, dividing the number of points by the total number of available points, and multiplying by 100. All consultation sessions were recorded and 20% were checked for fidelity by an additional observer. The fidelity scores’ mean was 93% (range 80–97%).

Group 2: Asynchronous E-Learning Only Group

The group of caregivers assigned to the asynchronous only group completed the e-learning module series on their own. After the initial assessment they were granted access to the module series. They were instructed to complete the module series at their own pace, but that access would only be available for a total of 10 weeks. Prompts were built into the course for them to complete the pretest, posttests, and social validity assessments at the correct times. If they emailed the course moderator (person in charge or handling passwords and access), they received a response. They may have received emails periodically from the course moderator. However, they did not have any one-on-one real-time interactions with individuals regarding the course. Additionally, their assignments from within the learning modules or homework assignments were not graded or checked. They were only given numerical feedback on their tests after each module, which were graded by the LMS.

Measures

Various measures were used to evaluate the effects of the interventions. The overall course curriculum-based assessment (pretests and posttests), Early Intervention Parenting Self-Efficacy Scale (EIPSES; Guimond et al., 2008), and Behavior Problems Inventory (BPI-01; Rojahn et al., 2001) were conducted before and after the entire module series. Curriculum-based assessments (pretests and posttests) specific to each module were conducted before and after each individual module. The Treatment Evaluation Inventory Short Form (TEI-SF; Kelley et al., 1989) was conducted after each individual module. The overall social validity scales (developed for this study) were conducted at the end of the entire module series. As with all materials, all measures were translated into the language of the caregivers. All the forms, except for the curriculum-based assessments, were converted into online forms. Links to the forms were sent to the caregivers for them to independently access and complete.

Curriculum-Based Assessments (Pretests and Posttests)

Curriculum-based assessments were used throughout the study to assess caregivers’ gains in ABA-specific knowledge. The researchers created a test bank of questions targeting all eight modules. All the questions were multiple choice and contained four answer choices. The test bank had a total of 80 questions, 10 questions from each module. Before beginning the module series, the caregivers took a 20-question pretest. The questions were randomly selected by the LMS test software from the test bank. Each test taken by each person each time was different, since the LMS selected items from a test bank at random. The caregivers took a posttest at the end of the module series. Just as with the pretest, the posttest at the end of the series was 20 randomly selected questions. Pretests and posttests were also used in each individual module. In the individual modules, the pretests and posttests had the same questions. All the tests were timed, instructed to be completed independently, calculated as the number correct, and did not provide specific feedback to the caregivers upon completion (only a score, not which questions were correct/incorrect). The testing procedures were the same for caregivers regardless of group assignment. An example of a test question (translated into English) can be found below:

Which type of prompt involves you showing your child how to do something before he tries to do it himself?

Verbal

Physical

Model

Timed

Early Intervention Parenting Self-Efficacy Scale (EIPSES; Guimond et al., 2008)

The EIPSES is a short self-efficacy measure that is often used in parent training (e.g., Dai et al., 2018). The measure was given to the caregivers pre and post completion of the module series. The self-report assessment was used to have the caregivers measure their own perceptions of their abilities as caregivers. We used a revised version based on the work of Dai et al. (2018). It included 10 items that were rated using a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). The list of questions on the scale can be found in Appendix C.

Behavior Problems Inventory (BPI-01; Rojahn et al., 2001)

The BPI-01 is a 52-item behavior rating instrument used to assess behaviors most associated with children with NDD (Rojan et al., 2001). It has caregivers provide an assessment of the frequency (i.e., never, monthly, weekly, daily, hourly) and severity (i.e., mild, moderate, severe) of their children’s behaviors across the categories of self-injurious behavior, stereotypic behavior, and aggressive/destructive behavior. The form yields a total of six numeric scores: one across the frequency and severity for each of the three behavior categories.

Treatment Evaluation Inventory Short Form (TEI-SF; Kelley et al., 1989)

The TEI-SF is a 9-item assessment using a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). It was used to assess the social validity perceptions of the caregivers on each of the individual training modules. They completed our version of the scale, which was based on the work of Dai et al. (2018), after each of the eight modules. The list of questions in the form can be found in Appendix D.

Overall Social Validity Scales

The overall social validity scales were completed by caregivers at the end of the complete module series. The scales were developed by the researchers based on the work done in similar studies (i.e., Parmanto et al., 2016; Griffiths, 2020; Tsami et al., 2019). They used a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). The list of the questions that were used in the analysis can be found in Table 7.

Table 7

Social validity

N = 33	Group 1	Group 2	p
1. Access to this information improved my knowledge of ABA	4.86	4.89	0.75
2. The topics covered in the modules were relevant to my family	4.64	4.58	0.72
3. Participating in the module series fit easily into my family’s routine	3.93	4.16	0.39
4. I applied the information taught in this module series	4.43	4.42	0.97
5. I noticed changes in my child’s behavior as a result of applying strategies taught in this module series	4.07	4.26	0.49
6. I plan to continue using the information that I learned about in this module series	4.57	4.68	0.57
7. I want to continue learning more about ABA	4.43	4.47	0.87
8. The online learning system was easy to use	4.71	4.68	0.86
9. Telehealth is an acceptable way to receive ABA services	4.21	4.11	0.72
10. I would use telehealth services again	4.50	4.74	0.27
11. Overall, I was satisfied with this ABA telehealth system	4.71	4.74	0.90
Social validity total score	4.46	4.52	0.67

Statements were rated on a five-point Likert-type scale, where (1) indicated strongly disagree and (5) indicated strongly agree

Results

Curriculum-Based Assessments (Pretests and Posttests)

The scores on the overall curriculum-based assessment were calculated for each caregiver pre and post the completed module series. Caregivers could reach a score from 0 to 20 correct answers. Caregivers from Group 1 improved by 44.7% on average, while those in Group 2 increased their knowledge by 33.4%, (see Fig. 2). A paired-samples t test was conducted to compare the knowledge scores separately for each group at intake (pretest) and after the end of the e-learning program (posttest). For Group 1 there was a significant difference in the scores at pretest (M = 8.57, SD = 3.18) and posttest (M = 17.50, SD = 1.91); t(13) = − 10.69, p = 0.001. In case of Group 2, significant difference between the scores at pretest (M = 9.58, SD = 2.32) and posttest (M = 16.26, SD = 1.76); t(18) = − 12.36, p = 0.001, were also found. An independent sample t test was again used in comparing the difference in knowledge gain between the two groups. The test revealed a statistically significant difference in the knowledge gain scores between Group 1 (M = 8.93, SD = 3.13) and Group 2 (M = 6.68, SD = 2.36); t(31) = 2.36, p = 0.025. Refer to Table 3 for complete knowledge data comparison.

Fig. 2

Curriculum-based assessments

Table 3

Curriculum-based assessments

	Group 1	Group 2	Combined	p
	n = 14	n = 19	N = 33
Treatment	Consultations	Asynchronous
Pretest knowledge M (SD)	8.57 (3.18)	9.58 (2.32)	9.15 (2.72)
Posttest knowledge M (SD)	17.50 (1.91)	16.26 (1.76)	16.79 (1.90)
Knowledge gain M (SD)	8.93 (3.13)	6.68 (2.36)	7.64 (2.89)	0.025*

*Indicates statistically significant difference between groups in the mean knowledge gain

Early Intervention Parenting Self-Efficacy Scale (EIPSES)

The EIPSES scores representing self-perceived parental efficacy (evaluated on a 5-point Likert scale) were calculated for individual questions of the EIPSES questionnaire. Table 4 shows how caregivers in each group improved or worsened on each question from pre to post completion of the e-learning course. Using independent sample t-tests, no statistically significant difference between the two groups was found for any of the individual EIPSES questions. Similar results were found in the case of the EIPSES total scores. Group 1 had an overall mean EIPSES score of 3.51 at baseline and 3.59 after the completion of all modules. The change was slightly positive (0.08 points), but very small. Similar results were found for Group 2 (3.52 points at baseline and 3.63 at posttest), with a 0.13 point increase. No statistically significant differences were found in the total EIPSES score change between the two groups (t(31) =  − 0.21, p = 0.834).

Table 4

EIPSES item-level and total scores change between pretest and posttest

Statements (shortened)	Group 1,	Group 2	p
	n = 14	n = 19
1 If my child is having problems, I would be able to help	− 0.36	0.16	0.20
2 When my child shows improvement, it is because I am able…	0.00	0.00	1.00
3 If a professional working with my child had difficulty……	0.57	0.05	0.15
4 Children will make the most progress if a professional works	− 0.21	0.21	0.27
5 Even a good parent may not have much impact	0.14	0.00	0.67
6 If my child learns something quickly, it would probably be because	0.00	0.32	0.23
7 On most days, I can handle most of the ups and downs	0.21	0.05	0.66
8 I worry that I am not a good enough parent due to outside demands	0.43	0.21	0.57
9 Over the past three months, I can see the progress that I have made	0.50	0.32	0.31
10 Whether a child is born with a disability or not, the child's parents	− 0.50	− 0.05	0.66
Total score EIPSES mean difference	0.08	0.13	0.83

Full statements are presented in Appendix C. Statements were rated on a five-point Likert-type scale, where (1) indicated strongly disagree and (5) indicated strongly agree. Items 4, 5, and 8 were reverse coded. Positive mean difference indicates increase in parental efficacy

Behavior Problems Inventory (BPI-01; Rojahn et al., 2001)

Scores on six BPI dimensions were analyzed to determine the extent to which caregivers’ self-reported behaviors of their children changed pre to post course completion. Table 5 summarizes the average change (improvement) for the six BPI dimensions for each group. Negative scores indicate reduction in problem behavior occurrence or severity. Group 1 had an average total BPI score change of − 15.14. Group 2 had an average total BPI score change of − 6.84. Using independent sample t-tests, no statistically significant difference between the two groups was found for any of the dimensions, including the total change (t(31) =  − 1.01, p = 0.323). However, both groups improved on all dimensions of the BPI except for Group 2 on aggression severity. Both groups reported the most significant decrease on the stereotypy occurrence and stereotypy severity sub-scales of the BPI.

Table 5

Average BPI dimensions and total scores change between pretest and posttest

N = 33	Group 1	Group 2	p
SIB occurrence	− 1.71	− 0.58	0.32
SIB severity	− 1.21	− 0.44	0.22
Stereotypy occurrence	− 5.71	− 4.32	0.75
Stereotypy severity	− 3.29	− 1.39	0.47
Aggression occurrence	− 1.71	− 0.47	0.33
Aggression severity	− 1.50	0.21	0.11
Total BPI score change	− 15.14	− 6.84	0.32

Treatment Evaluation Inventory Short Form (TEI-SF; Kelley et al., 1989)

To examine the validity of individual modules of the course, we looked at how caregivers rated each lesson of the modules’ e-learning program. Table 6 summarizes the average ratings for each module within each group. The total scores (on a 5-point scale) for all modules showed that caregivers in both groups rated the course highly (Group 1, M = 4.31, Group 2, M = 4.44). Differences between the groups in total scores were statistically insignificant t(31) =  − 0.49, p = 0.634, suggesting that groups did not differ in their overall evaluation of the course. Regarding the individual modules, both groups rated Module 8 less favorably and Group 1 also rated Module 1 with lower scores (4.17 on a 5-point scale).

Table 6

TEI-SF mean ratings for each e-learning module

N = 33	Group 1	Group 2	p
1. Introduction to applied behavior analysis	4.17	4.44	0.42
2. Reinforcement, the three-term contingency	4.50	4.73	0.19
3. Pairing, conditioning, and instructional control	4.45	4.64	0.17
4. Functions of behavior	4.41	4.42	0.97
5. Prompting	4.41	4.74	0.06
6. Naturalistic learning and motivation	4.51	4.52	0.94
7. Communication	4.62	4.58	0.85
8. ABA services and telehealth	4.18	4.21	0.92
TEI-SF score for all modules	4.31	4.44	0.63

Statements were rated on a five-point Likert-type scale, where (1) indicated strongly disagree and (5) indicated strongly agree

Overall Social Validity Scales

After the last module, participants were asked to complete a social validity questionnaire with 11 analyzed questions. Participants rated the content of the course, its applicability in their everyday lives, willingness to continue training in ABA, and usability of the LMS system on a 5-point Likert scale. A mean score was calculated for each group. Group 1 scored M = 4.46 (SD = 0.44) and Group 2 scored M = 4.52 (SD = 0.36). Based on the results of the independent samples t test, it was concluded that the differences between the groups in social validity scores were statistically insignificant t(31) = − 0.435, p = 0.667. The results suggest that the online e-learning program had high social validity for all the participating caregivers (M = 4.5 on a 5-point scale), regardless of group assignment. See Table 7 for more details on the social validity scores.

Discussion

The current study focused on piloting an ABA e-learning course for caregivers and assessing whether a synchronous component was needed for the best outcomes. The e-learning course was not meant to replace best practices in more intensive caregiver training but provide foundational skills for later more successful local telehealth services. At the time, local ABA professionals in the region did not have the capacity or skills to provide more individualized and child-centered ABA telehealth services. However, there were upcoming projects in the works to better prepare the professionals for providing e-coaching ABA telehealth services to Czech families in the future (Kingsdorf et al., under-review). Establishing this educational component beforehand aimed to be in alignment with the research by Bearss et al. (2015) and Strauss et al. (2012), who have demonstrated value in educating caregivers in ABA to promote greater improvement in the behaviors of their children during direct ABA services. The relative efficacy was evaluated for the groups with and without synchronous components by assessing the following areas prior to and immediately following training: knowledge base concerning ABA, perceived self-efficacy of caregivers, challenging behavior in their children, and social validity.

Overall, the e-learning course increased ABA knowledge and was viewed as socially significant by all caregivers. Content knowledge greatly improved in both groups with the first group (consultation group) displaying the greatest increase. Although the difference between the two groups was statistically significant, both groups made very large gains. These results are consistent with the assessment of knowledge in the research conducted by Yi and Dixon (2021) on a telehealth training curriculum and the comparison of in-vivo and online training done by Blackman et al. (2020). With both groups making large gains, though, the results suggest that an online training model focusing solely on building theoretical knowledge may rely on the provision of asynchronous interactive content only. This conclusion is supported by Kingsdorf and Pancocha's (2020) comparison of the efficacy of online versus in-person lectures in ABA for pre-service teachers. However, an important point to consider, is that although the course increased the participants' knowledge, since the caregivers did not receive any direct coaching or feedback on how they were implementing procedures, we do not know how this knowledge translated into practice. Therefore, it would require adopting direct child and parent outcome measures as well as an extensive synchronous training model to make predictions as to the overall efficacy of this e-learning course.

In addition to the outcomes regarding the e-learning modality, there are other interesting areas to consider from this work. In our pilot, originally 60 participants were recruited. However, only 33 of them started the training program; 70% from the synchronous group (Group 1) and 48% from the asynchronous group (Group 2). Clearly, motivating participants to move forward with online courses is challenging (Martinez, 2003). It could be the case that the promise of the short synchronous meeting provided increased motivation for course participation. Additionally, although not analyzed in this paper, lack of interest in participation may have been impacted by demographic characteristics of the participants. Unfortunately, extensive data on important variables such as education, racial, ethnic, cultural, and linguistic diversity (RECLD), and socio-economic status were not collected and analyzed in this study. We did aim to collect cursory information on household income and education, but only have complete data available for those who completed the course. Those demographic data did reveal that most caregivers who completed the course had a university education. This may align with the work of Martinez (2003), who found that participants having the necessary background and experience (i.e., experience with post-secondary education) faced less challenges when adjusting to a relatively new online learning environment. Participation by those caregivers in higher education likely increased the probability that they would have the necessary level of academic and social integration described by Tinto's (1975) retention model (e.g., family background, learning strategies, and individual attributes).

Considering individual attributes that can be measured, like education and RECLD characteristics, are undoubtedly important. However, success in online training may be additionally impacted by variables that cannot be easily assessed. It has been reported that synchronous contact supports a feeling of connection to instructors and affects attitude and motivation (Francescucci & Rohani, 2019; Martinez, 2003). This could again be a reason behind the increased success rate of those in Group 1. However, another hypothesis is that those who were successful in the course had more developed self-management skills. According to Martinez (2003), people with an internal locus of control are motivated and trust their abilities to make a difference and achieve desired outcomes. On the other hand, people with an external locus of control feel that they must rely on sources outside their control. The concept of self-efficacy coined by Bandura (1997), referring to one's belief in their capacities to achieve a goal, is based on the internal locus of control. Strengthening these behaviors within the skin might be targeted in future work by adding components from acceptance and commitment training as a precursor to coursework. In the work of Yi and Dixon (2021), for example, caregiver adherence to online training protocols was enhanced by addressing the self-management of thoughts and feelings through the components of mindfulness training, promoting values-driven behaviors, and psychological flexibility building. Therefore, these might be seen as important prerequisite skills to build before expecting independent online learning from caregivers.

While measures of self-assessment were targeted within this research, they did not explicitly address behaviors within the skin (unobservable or covert behaviors). Rather, assessment of caregiver self-efficacy regarding the applications of tactics targeted within the course was evaluated. However, in this research, the EIPSES outcomes exhibited no significant development, neither between pre- and post-assessment nor between groups. In contrast, the research of Nefdt et al. (2010) reported that parents felt more confident after completion of their online self-directed learning program. Blackman et al. (2020) found that in-vivo training led to greater increases in self-efficacy measures than online training. However, even in the online modality, perceived self-efficacy increased. Further investigation into the self-efficacy beliefs of caregivers included in our training is recommended, namely discovering how self-efficacy influences retention, and understanding which components of the training influence self-efficacy beliefs. This knowledge might help to optimize training outcomes. It may also be beneficial to look at the intersection of embedded behavior change procedures targeting covert behaviors, online training participation, and self-efficacy.

Despite the small changes in some scores, like those of self-efficacy, the assessment of overall social validity showed that the training was perceived as very useful to all participants, with no significant differences between the groups. Even though there was no assessment of the application of the information taught in the course (e.g., the caregivers were not observed for their usage of positive reinforcement with a child), the TEI-SF assessment was completed after each topic and had questions which supported the application of the curriculum by the caregivers. Notably, the two lowest rated modules were Module 1: Introduction to applied behavior analysis, which focused on the theoretical introduction of ABA, and Module 8: ABA services and telehealth, which provided information on the technical aspects of ABA delivered in different telehealth formats. Both had smaller practical components for use by the parents with their children. Both modules also included the most professional terms from the field of ABA and telehealth. Some parents reported that they did not plan to use telehealth ABA services in the future and therefore found the content irrelevant. On the other hand, highly rated modules by both groups (4.5 and above) included Module 2: Reinforcement, which included content focused on reinforcement assessment and delivery, and Modules 6 and 7, focusing on naturalistic learning, motivation, and communication. These modules included substantial practical elements, which could be used immediately by the participating parents (e.g., creating a list of preferred items, delivering reinforcement effectively, building motivation, teaching new skills in the natural setting, and creating communication opportunities). This appreciation of the applied aspects of the module content was echoed in the overall social validity assessments, too. The caregivers highly rated (4.5 and above) their willingness to use the acquired information further (question 6) and willingness to use telehealth services again (question 10). Overall, this aligns with the work of other research on caregiver training, demonstrating that the needs of caregivers of children with autism can be supported through practical ABA-based e-learning trainings (e.g., Dai et al., 2018; Blackman, 2020).

This aligns with our, although limited, outcomes on the BPI-01, as well. It revealed a decrease in observed challenging behavior tendencies that was notable but not significant. The greatest decrease occurred for stereotypical behaviors (e.g., rocking, hand flapping). Preferences for stereotypical behavior are often linked to a self-stimulatory function and tend to appear because of inadequate (or low level of) environmental stimulation. It might be hypothesized that the training had a positive effect on interactions between caregivers and children throughout the day and the caregivers' abilities to prepare meaningful activities, thus increasing environmental stimulation. However, with this study’s severe limitation of missing direct data collection on the behavior of the caregivers or their children, this hypothesis is only cursory.

Implications and Future Research

Overall, while the outcomes of this pilot study are valuable, there is certainly room for further exploration. First, the number of caregivers who started the course was much lower than anticipated. From this study we learned that a caregiver’s interest in a free online course does not guarantee participation. Future research should aim to create and sustain larger groups for comparison by using strategies such as antecedent manipulations like incorporating reminders, consequent manipulations such as incentives, building self-management skills before participation, and/or removing the variable of limited time access. Furthermore, to aid future course development, more information should be collected on the prerequisite skills needed for caregivers to be successful in online learning. Something that was not done here, but which should be taken into consideration in the future, is investigating and programming to support the RECLD characteristics of the caregiver population’s learning needs. While we aimed to make the course linguistically and culturally appropriate for the Czech population, we did not assess our match between our perceived and actual participating population. Overall, the conclusions from this pilot can help drive future work on e-learning applications for caregivers of children with autism, with the added benefit of validating low-cost ABA-education applications. They also set the stage for our later work with caregivers in the region. As noted, the goal of this study was not to replace more intensive caregiver training services. Rather, it was supporting the work of local practitioners by better preparing parents to be active participants in their child’s ABA telehealth services, when more intensive services become available (e.g., local practitioners have the skills and capacity to provide other telehealth services). In that same vein, future work should investigate the relationship between caregivers’ participation in the e-course and their later role in their children’s ABA services, their willingness to pursue more intensive ABA services, and observable behaviors in both them and their children.

	Question	Strongly disagree	Disagree	Undecided	Agree	Strongly agree
1	If my child is having problems, I would be able to think of some ways to help my child	1	2	3	4	5
2	When my child shows improvement, it is because I am able to make a difference in my child's development	1	2	3	4	5
3	If a professional working with my child had difficulty, I would be able to offer some useful suggestions	1	2	3	4	5
4	Children will make the most progress if a professional rather than a parent works with them	1	2	3	4	5
5	Even a good parent may not have much impact on whether children make progress in their development	1	2	3	4	5
6	If my child learns something quickly, it would probably be because I've done a good job teaching him	1	2	3	4	5
7	On most days, I can handle most of the ups and downs of being a parent	1	2	3	4	5
8	I worry that I am not a good enough parent due to outside demands placed upon my time and energy	1	2	3	4	5
9	Over the past three months, I can see the progress that I have made in becoming a better parent	1	2	3	4	5
10	Whether a child is born with a disability or not, the child's parents can still do a lot to help him/her	1	2	3	4	5

	Question	Strongly disagree	Disagree	Undecided	Agree	Strongly agree
1	Acceptable: I find these strategies to be an acceptable way of dealing with my child’s challenging behaviors	1	2	3	4	5
2	Willing to use: I would be willing to use these strategies if I was working to change my child’s behaviors	1	2	3	4	5
3	Child responded well: I believe that my child would respond well to the use of these strategies	1	2	3	4	5
4	Like: I like the strategies introduced in this module	1	2	3	4	5
5	Considered effective: I believe that these strategies are likely to be effective	1	2	3	4	5
6	Long term improvement: I believe using these strategies is likely to result in permanent improvement	1	2	3	4	5
7	Able to try out: I am going to use these strategies on my own with my child	1	2	3	4	5
8	Clear: I found the directions on the use of these strategies clear	1	2	3	4	5
9	Overall positive: overall, I have a positive reaction to these strategies	1	2	3	4	5