Effects of an Adapted Telehealth Training Curriculum for Registered Behavior Technicians®.

Objectives: We evaluated a telehealth training curriculum to prepare Registered Behavior Technicians® (RBTs®) to conduct caregiver coaching on behavior analytic skills via telehealth.
Methods: A non-concurrent multiple baseline across therapists design was used to evaluate the training procedures delivered by Board Certified Behavior Analysts® (BCBAs®). All therapists (RBTs) worked with one family during the baseline and training phases and an additional session was conducted with a different family during the novelty probe condition to see if therapists were able to perform the newly acquired skills without BCBA support.
Results: Seven therapists who received behavioral skills training on the curriculum reached mastery criteria within three sessions. All RBTs were able to provide caregiver coaching via telehealth with high degrees of fidelity and were able to independently conduct a telehealth session with a new family without support from the BCBA. Conclusions: Results of the current study provide support for the use of the telehealth training curriculum to prepare RBTs to conduct ABA services via telehealth.

Applied behavior analysis (ABA) is a science focused on the way in which environmental contingencies promote or inhibit certain behaviors. These principles have been used to develop a variety of evidence-based practices used to serve individuals with autism spectrum disorder (ASD; Roane et al., 2016; Wong et al., 2015) by addressing interfering behavior that can impede an individual’s learning and daily functioning. Specifically, therapy based on the principles of ABA has been used to increase prosocial behavior such as language, social skills, and academic tasks (Virués-Ortega, 2010) and decrease challenging behavior such as aggression (Heyvaert et al., 2014) and self-injurious behavior (Hagopian et al., 2015). ABA service providers, or therapists, typically conduct treatment in clinic, home, or educational settings while working directly in-person with clients. However, since the USA declared a national emergency in response to COVID-19, stay-at-home orders and other containment measures resulted in behavior analysts pivoting ABA therapy services to the telehealth modality. Additionally, board regulations and insurance companies adjusted their guidelines for professional behavior analysts for services and supervision to continue via telehealth (ABA Billing Codes Commission, 2020; Behavior Analyst Certification Board, 2020a).

Telehealth is defined as the provision of health care services from a professional to a client at a remote geographic location through use of two-way audio and visual communication technologies (e.g., Zoom®; Vismara et al., 2012). With recent advances in technology and these communication systems becoming more readily available within the community, a variety of healthcare fields (e.g., psychiatry, behavioral health; Tuckson Reed et al., 2017) have adopted telehealth as a service delivery model. Research on behavior analytic services delivered via telehealth has existed within behavior analytic literature for the past decade (Schieltz & Wacker, 2020). However, prior to 2020, much of this literature focused on outcomes for the interventionist rather than the client (e.g., procedures used, fidelity outcomes; Ferguson et al., 2019; Neely et al., 2017; Tomlinson et al., 2018; Unholz-Bowden et al., 2020) and had not yet been deemed an evidence-based practice (Ferguson et al., 2019; Tomlinson et al., 2018). Nonetheless, with the COVID-19 pandemic serving as a catalyst for additional research on this topic, recent reviews have investigated the impact of telehealth ABA services at the client level, finding support for the use of telehealth to conduct behavior assessments and provide interventions aimed at increasing or strengthening behavior (Neely et al., 2021). Additionally, as providers have experienced rapid transition from in-person service delivery to telehealth service delivery models, additional papers have been published on the practitioner experience during this transition (Pollard et al., 2021), support provided through research on available technologies (Zoder-Martell et al., 2020), and systematic training procedures for telehealth practitioners (Neely et al., 2022).

These training procedures are vital to telehealth ABA services, as most research supports synchronous (live) coaching from a behavior analyst to the interventionists working directly in-person with the client (Neely et al., 2021). Two previous systematic reviews (Neely et al., 2017; Unholz-Bowden et al., 2020) evaluated research on telehealth caregiver coaching and found commonalities regarding the interventions taught and coaching procedures associated with improved interventionist (i.e., caregiver) treatment fidelity outcomes. However, Neely et al. (2017) brought attention to the lack of reported fidelity for these coaching procedures across studies, which was also reported in follow up reviews (Ferguson et al., 2019; Tomlinson et al., 2018). Without high levels of coaching fidelity, one cannot confidently interpret levels of interventionist fidelity. Thus, it remains unclear whether low levels of interventionist fidelity may be attributed to certain factors related to the intervention itself (e.g., complexity of the intervention) or were moderated by degrees of coaching fidelity.

An important variable that may impact interventionist-fidelity outcomes are the characteristics of the therapist-trainer providing caregiver coaching. Tomlinson et al. (2018) noted a lack of reported trainer characteristics in previous research and found that when these characteristics were reported, most were professionals with prior ABA experience and for six out of 20 papers reviewed, all trainers were Board Certified Behavior Analysts® (BCBAs). In a study by Lerman et al. (2020), authors report on the experiences of 18 practitioners who provided caregiver coaching via telehealth from four different sites across a 4-year period. As part of the study, Lerman et al. found that caregiver coaching skills do not automatically transfer from the in-person modality to telehealth modality. Factors that have been found to enhance trainer fidelity when coaching interventionists share similarities across both in-person and telehealth coaching modalities, but certain barriers may be present and harder to overcome when services are provided in a telehealth service modality (Lerman et al., 2020). Specifically, behavior skills training (BST; Miltenberger, 2008) has been used to promote degrees of interventionist fidelity across a variety of behavioral assessments and interventions; however, certain elements of BST such as modeling may be difficult to incorporate via telehealth as it may require the addition of a client-confederate in the session or a prerecorded video model. In addition to this, there are a variety of other barriers that are more present in a telehealth service environment (e.g., technical issues, challenges with remote viewing; see Lerman et al., 2020, for more detail) as compared to in-person services, and as such, require more intensive training procedures for therapists preparing to conduct services remotely. Neely et al. (2022) extended the research by Lerman et al. (2020) by developing a telehealth training curriculum to prepare four in-service BCBAs to deliver remote caregiver coaching. While these advances are necessary given the rapid growth of telehealth as a service delivery model, most research to date has focused on training BCBAs rather than direct therapists (i.e., Registered Behavior Technicians®; RBTs).

According to the Behavior Analyst Certification Board (BACB), there are almost twice as many RBTs (n = 82,978) as BCBAs (n = 42,272) currently certified (BACB, 2021), and as such, RBTs are often front-line interventionists. Given the lack of adequate practitioners to provide caregiver coaching via telehealth (Neely et al., 2022) and substantial support for this model of service delivery (e.g., Neely et al., 2021; Unholz-Bowden et al., 2020), research should focus on how to prepare all levels of practitioners in the telehealth modality. Additionally, with telehealth ABA services rapidly expanding, preparing RBTs to conduct these services with high degrees of fidelity is pivotal to meeting the consumer demand. However, before RBTs are able to provide high-quality telehealth services, a rigorous technician training system that includes standard practices and fidelity expectations (Pollard et al., 2017) is needed in order to yield positive effects for clients who either opt-in or require behavior analytic services via telehealth for a variety of reasons.

The purpose of the current study was to (1) train RBTs to lead an ABA service session via telehealth with BCBA supervision, and (2) evaluate therapists’ ability to peform the newly acquired skills with a novel family without BCBA support. Specifically, the research questions were as follows: (1) Can BST and the telehealth training curriculum published by Neely et al. (2022) be utilized to prepare RBTs to accurately conduct telehealth ABA services? (2) Will RBTs be able to perform the newly acquired skills with a novel family without any additional coaching? We acknowledge that independent caregiver coaching by RBTs is not within the scope of the current RBT Task List (2nd edition, BACB, 2018). Thus, in accordance with Center for Disease Control and Prevention social distancing guidelines (2022) and the BACB Ethics Code for Behavior Analysts (2020b), the current study extended caregiver coaching methods to involve RBTs supervised by BCBAs, and it should be noted that BCBAs were always present during caregiver coaching sessions. In addition, this study included therapists enrolled in a master’s program, studying to become BCBAs, who would ultimately be charged with conducting telehealth sessions independently post-certification.

Method

Participants

This study was conducted in the context of a larger host project initiated during the on-set of the COVID-19 pandemic (March 2020). A total of 25 therapists participated in the larger host project which provided telehealth services to families unable to attend in-person services at the clinic due to stay at home orders during the pandemic. Therapists that were part of the host project included individuals who were working 1:1 with clients, providing ABA services to individuals with autism in the clinical setting. At the conclusion of the project, therapists were recruited for this research from the host project and were included in this research if they (1) were RBTs or therapists working toward becoming a BCBA, (2) completed all the training sessions, and (3) consented to include their data for publication. A total of seven therapists met the inclusion criteria and consented to include their data in this study (see Table 1). Therapist characteristics are shown in Table 1 and denote the therapist’s age, gender, race/ethnicity, highest level of education at the time of the study, practitioner credentials, RBT certification date (month/ year), and years of experience in ABA. Prior to participating in this research study, none of the therapists had prior experience independently coaching caregivers on behavior analytic procedures; however, they had modeled skills for caregivers in the clinical setting while their supervising BCBA provided direct instruction to caregivers during in-person caregiver coaching sessions. Each therapist had a different level of experience in the field of ABA. Five therapists were currently in graduate-level training programs to become a BCBA, one therapist completed all coursework to become a BCBA and was accruing supervision hours at the time of the study (CJ), and one participant was completing undergraduate coursework to become certified as a BCaBA (MW).

Table 1

Therapist demographics

Therapist	Age in years (gender)	Race/ethnicity	Highest level of education	Credentials	RBT certification date(month/year)	Years of ABA experience
TX	29 (M)	White/Hispanic	Bachelors	Working toward RBT	9/2020	> 1
MW	27 (F)	White	Bachelors	RBT	12/2015	6
LQ	26 (F)	White/Hispanic	Bachelors	Working toward RBT	1/2021	> 1
CJ	27 (M)	Non-White/Black	Masters	RBT	9/2021	4
SS	24 (F)	White/Asian	Bachelors	RBT	9/2019	2
SD	24 (F)	White/Hispanic	Bachelors	RBT	6/2020	> 1
LI	27 (F)	White	Masters	RBT	2/2020	> 1

F, female; M, male.

During baseline and the RBT training phase, each therapist worked with one family for the purpose of this study. In addition, each therapist worked with one new family during the novelty probe condition. Families who participated in the telehealth services were either currently receiving in-person clinical services that were interrupted due to COVID-19 or recruited from autism clinic waitlist(s). Before participating in this specific telehealth program, caregivers were informed that through participation in this telehealth program, they would be coached by therapists who were in training. All participation was voluntary. If caregivers did not consent to the study, they were referred to one of the BCBAs to participate in telehealth sessions that did not involve a therapist currently receiving telehealth training. Additionally, novel families were included in the study to probe for therapists’ maintenance and application of the newly learned skills. Novel families were those that were currently receiving ABA services via telehealth at the local ABA clinic. Family characteristics are shown in Table 2 and denote the child’s age, diagnosis, relationship to caregiver, service status at the time of the study, the child’s target behavior addressed, and intervention taught to caregivers.

Table 2

Family demographics

Therapist	Family	Child age in years	Child diagnosis	Caregiver relationship to child	Race/ethnicity	Service status	Target behavior	Intervention
TX	1	5	ASD	Mother/ Father	Non-White/Hispanic	Currently Served	Elopement, aggression	Pairing, FCT
	2	12	ASD	Mother	White	Waitlist	Aggression	Behavior contract
MW	1	3	ASD	Grandmother	Non-White/Hispanic	Currently served	Communication	Picture exchange
	2	5	ASD	Mother	Non-White/Hispanic	Currently served	Feeding	Shaping
LQ	1	13	ASD	Mother/father	White/Hispanic	Currently served	Aggression	FCT with signal training
	2	10	ASD	Mother	Non-White/Hispanic	Currently served	Task refusal	Token economy
CJ	1	3	At risk for ASD	Mother	White/Hispanic	Currently served	Communication	NET
	2	14	ASD	Grandmother	Black	Currently served	Task refusal	Behavior contract
SS	1	14	ASD	Mother/father	White/Hispanic	Wait list	Stereotypy	Signal training, self-monitoring
	2	3	At risk for ASD	Mother	Not provided	Wait list	Communication	Picture exchange, NET
SD	1	3	ASD	Mother	Non-White/Hispanic	Currently served	Aggression Tantrum	FCT
	2	8	ASD	Mother	Non-White/Hispanic	Currently served	Skill acquisition	Token economy
LI	1	14	ASD	Mother	Non-White/Hispanic	Currently served	Communication	NET
	2	13	ASD	Mother	Non-White/Hispanic	Currently served	Tolerating presence of communication device	Shaping, differential reinforcement

FCT, functional communication training; NET, naturalistic environmental teaching.

Prior to each therapist starting with the family, the BCBA trainer met with the families to identify the area of need and what target behaviors they wanted to focus on during the telehealth intervention. The BCBA trainer then determined an appropriate intervention that the therapist would coach the caregivers to implement (see Table 2). Each therapist was familiar with all skills they were coaching caregivers to implement from prior clinical or coursework experience. The BCBA trainer reviewed the intervention with the therapist and answered any questions the therapist had regarding the intervention or strategy they would be coaching the caregivers to implement prior to the first telehealth session with the caregiver.

Given that specific technology equipment was required for telehealth services, families were required to have internet access, but a technology lending library was available and used for families and therapists who did not have access to the necessary materials (i.e., tablets, wireless earphones, and headsets with a microphone). That is, researchers did not exclude therapists or families from the study if they did not possess the required technology equipment and materials. However, only one family utilized the technology lending library and checked out wireless earphones. All other families and therapist had all technology required.

BCBA Trainers

The second, fourth, and fifth authors served as BCBA trainers and trained all the therapists. The sixth author served as the expert trainer and supervised the BCBA trainers. At the time of this study, the sixth author, a BCBA-Doctoral level, had 9 years of experience implementing ABA therapy and 6 years of experience conducting sessions via telehealth. She trained the second, fourth, and fifth authors (BCBA trainers) prior to the study through a different training project. The second author was credentialed as a BCBA for 1 year and had 1 year of experience in providing services to clients via telehealth. The fourth author was credentialed as a BCBA for 3 years and had 2 years of experience in providing services to clients via telehealth, and the fifth author was credentialed as a BCBA for 3 years and had 2 years of experience in providing services to clients via telehealth.

Procedures

All sessions were conducted via video conferencing using the Zoom® platform. The therapists (i.e., RBT, RBT in training), BCBA trainers, and the families were all at separate sites (e.g., home, clinic) throughout the duration of the study. Each individual (e.g., trainers, therapists, and families) joined the coaching sessions using a device with a built-in webcam and microphone (i.e., tablets, laptop, or cell phones) and internet connection. All therapist training sessions, telehealth sessions with clients, and feedback sessions were recorded (with the exception of one client who did not consent to recording their sessions) for data collection purposes. Both the BCBA trainer and therapist were present during every telehealth session with the families.

Experimental Design

A non-concurrent multiple baseline across therapists design was utilized for the current study and therapists were randomly assigned their leg of intervention. In each multiple baseline evaluation, the first therapist had three baseline sessions (TX and CJ), the second therapist had four baseline sessions (MW and SS), and the third therapist had five baseline sessions (LQ, SD, and LI). The intervention phase (i.e., BST) was implemented once baseline data were stable and predictable and the intervention had demonstrated an effect for the previous therapist.

Baseline

Prior to baseline sessions, therapists had access to a brief description of the client including their age, gender, current communication repertoire, presence/absence of challenging behavior, current treatment plan, and telehealth schedule. The BCBA trainers then asked the therapists to complete the pre-session preparation steps (e.g., setting up a HIPAA-compliant Zoom® meeting, drafting an email to the caregiver). The BCBA trainer evaluated the therapist’s performance on the pre-session preparations using the caregiver coaching fidelity checklist. Due to the therapists having no prior experience conducting sessions via telehealth, the researcher team utilized a gating procedure to advance the therapists through the training. The gating procedure was implemented to ensure high levels of procedural fidelity for the services rendered while simultaneously allowing for the therapist skills to be shaped by the BCBA trainers before implementing with caregivers. Specifically, therapists needed to obtain a score of 100% on the pre-session preparation section of the caregiver coaching fidelity checklist (e.g., meeting set-up, email caregiver) in order to progress to the later sections of the checklist (e.g., live session with a family). All of the therapists failed at least one step of the pre-session preparations during baseline and no live sessions with family members were conducted during baseline.

Training

The BCBA trainers utilized BST to train the therapists in the caregiver coaching procedures. Following completion of baseline, the BCBA trainers provided therapists with all training materials consisting of a short, narrated PowerPoint® about the telehealth intervention, three research articles to read (Ferguson et al., 2019, Lerman et al., 2020, and Neely et al., 2017), the caregiver coaching fidelity checklist, and the sample of potential challenges and solutions when providing telehealth services (Lerman et al., 2020, Table 1). After the therapists reviewed the materials, they set up a meeting with their BCBA trainer to complete the initial training session. During the initial training session, the BCBA trainer reviewed the caregiver coaching fidelity checklist as well as the sample of potential challenges and solutions when providing telehealth services table with the therapist. Then, the trainer reviewed the therapist’s baseline performance and provided performance feedback on steps completed correctly or incorrectly, modeled a complete telehealth session with the target family, and answered any questions from the therapist. The initial therapist training sessions lasted between 45 min to 1.5 h and were all conducted via Zoom®.

After the completion of the initial therapist training session, the therapist conducted the telehealth session with the caregiver from start to finish; however, the gating procedure described previously was still in effect. Pre-session preparation materials (setting up the HIPAA-compliant Zoom® meeting and email to caregiver) were sent to the BCBA trainer to review prior to sending to the caregiver. If the therapist did not obtain a score of 100% on the pre-session preparation section of the caregiver coaching fidelity checklist (e.g., meeting set-up, email caregiver), they were not permitted to conduct the caregiver coaching session with the family. Instead, the BCBA trainer would conduct the session with the family and then provide additional training and feedback to the therapist during a later meeting, prior to the next appointment with the family. Each caregiver coaching session lasted between 30 min to 1 h depending on the client receiving the services. Additional therapist training and feedback sessions between the therapist and BCBA trainers were conducted outside of telehealth sessions with clients and lasted between 10 and 30 min.

All therapists completed the pre-session preparation procedures; thus, the sessions with the family progressed as follows. During the first session following intervention (e.g., BST), the therapists implemented the telehealth caregiver coaching session and the BCBA trainer modeled steps that were missed or implemented incorrectly. For the first session, the BCBA trainer had their camera and microphone on and were active participants in the session. That is, if the therapist was missing a step on the fidelity rubric, the BCBA trainer would perform the step as a way to directly model it for the therapist. During sessions two and three, the BCBA trainer was not as actively involved and was not visible to the caregivers. The therapist implemented the session and the BCBA trainer provided immediate feedback on missing/incomplete steps via the chat feature in Zoom®. Caregivers were unable to see the feedback that was being provided to the therapist. If the textual feedback did not improve therapist performance, the BCBA trainer would increase the prompt level and again be more visibly present in the session by completing the step as a way to model for the therapist, and then fade accordingly across sessions. That is, the BCBA trainers faded the level of support provided to therapists across caregiver coaching sessions. However, if during any part of the session there was an immediate risk of harm to the family (e.g., the child engaged in problem behavior that was dangerous), the BCBA trainer would immediately step in and model the expected behavior. Following each session, the BCBA trainer reviewed the rubric with the therapist and provided written feedback (in the form of an email) to the therapist. During each session, caregivers were aware that the BCBA trainer was present in the meeting and would step in to complete part of the caregiver coaching if the therapist was unable to complete the coaching without prompting. Mastery criteria was defined as a minimum of three sessions with at least one session at 90% accuracy and an increasing trend based on visual analysis.

Novelty Probe

After therapists reached mastery criteria in the intervention condition, they conducted a novelty probe with a new family. The purpose of this probe was to measure whether the therapist would be able to conduct a telehealth session with a novel family in which direct training from the BCBA trainer was not provided. As with the first family, a brief description of the client and the target intervention was provided to the therapist; however, at this point, the therapist implemented the telehealth caregiver coaching session without a modeled session.

Measures

The dependent variable was the percentage of steps performed correctly by the therapist on the caregiver coaching fidelity checklist provided in Table 3. The checklist was replicated and adapted from the coaching fidelity checklist published in Neely et al. (2022), which is also available as an online supplemental file in the original publication. The checklist was separated into three sections: pre-session planning, pre-session set-up procedures, and session procedures and followed the published checklist. However, the research team did make some modifications to reflect the role of an RBT during a telehealth session and these changes are as follows. First, in Coaching on a priority skill, numbers 4, 5, 7, and 8 were removed. Step numbers 4 and 5, providing descriptive feedback, were removed because these were included in step 3, coach the caregiver to implement the program using immediate feedback. Step 7, introduce one skill at a time, and step 8, shape caregiver behavior over the session, were removed as the therapists were only teaching one skill to each caregiver. In Other session skills, two items were added (sharing computer screen when applicable, demonstrating professionalism matched to caregiver style and preferences) in order to provide therapists with a means to share graphs and data with caregivers and ensure that communication styles and services were contextually fit the family and local community resources. Professionalism matched to caregiver style/preference was defined as the therapist following the caregivers lead with use of ABA terminology and overall mannerisms.

Table 3

Caregiver coaching fidelity checklist

Category	Expected components/ behaviors
Pre-session planning	1. Include link to Zoom meeting with correct day/time and a protected measure (e.g., password or waiting room enabled). Additionally, video should be “on” for both host and participant and join before host option is enabled
	2. Provide caregiver with phone number or other modality of real-time communication
	3. Provide caregiver with instructions for session (e.g., materials needed, directions for environmental arrangement, etc.)
	4. Provide caregiver with agenda for the session (e.g., list of session activities)
Pre-session set-up procedures	1. Check internet and ensure stable connection
	2. Check speaker, mic, and camera to ensure operational
	3. Prepare room for lighting and neutral background
	4. Prepare room to ensure confidentiality (e.g., lock door to room)
	5. Adjust camera to include all trainers
	6. Prepare devices (e.g., computer, tablet, and phone) by minimizing all windows except those needed for session and muting all sounds/notifications
Session procedures	Session introduction
	1. Check of systems (e.g., “can you hear and see me ok”)
	2. Provide a “warm” greeting
	3. Provide praise and descriptive feedback for any improvements made between sessions (e.g., “thanks for fixing your mic, I can hear you great”)
	4. Review previous correspondence/ meeting
	5. Ask caregiver about any progress/ challenges implementing the intervention(s) outside of session
	6. Ask caregiver if they have any questions
	7. Review the appointment plan and priorities for the appointment
	8. Review termination criteria (i.e., safety precautions)
	9. Review a distractor activity for the child (“reset/control”) that will allow the caregiver to talk with the [therapist] and criteria for when to use the distractor
	10. Confirm session priorities prior to beginning coaching
	Coaching on a priority skill
	1. When introducing a new goal or changing a program, provide verbal and written instruction (without technical language), modeling/video-modeling, and role-play
	2. If not introducing a new skill or after introducing the new skill, prompt the caregiver to initiate the program (e.g., FCT or incidental teaching)
	3. Coach the caregiver to implement the program (e.g., FCT or incidental teaching) using immediate feedback
	4. Ask if the caregiver has any questions and answer the questions the caregiver has from the coaching instructions
	Session wrap-up
	1. Conclude the session with a recap of key concepts and next steps
	2. Plan for subsequent meeting based on clinically relevant variables, caregiver preferences, and logistics teaching
	3. End meeting professionally and successfully (e.g., end meeting at a good moment, not when a caregiver is asking a final question; successfully logs out of meeting)
	Other session skills related to telehealth challenges
	1. Remedy any caregiver technical issues
	2. Remedy any remote viewing issues
	3. Remedy disruptions in the client’s environment
	4. Remedy issues related to client or caregiver behavior
	5. Remedy issues related to incorrect responding by caregiver
	Other session skills
	1. Allow extra time during pauses to allow for processing, response time, and internet delays
	2. Takes notes on caregiver comments (e.g., to link content to caregiver comments, to help with report/recommendation writing, to signal listening)
	3. Share computer screen when applicable (reviewing data, presenting new procedure rubrics, etc.) *
	4. Demonstrates professionalism, matched to caregiver style and preferences *

*Denotes skills different from Neely et al. (2022)

Each step was marked as either correct, incorrect, or not applicable. Correct responses were defined as the therapist completing the step correctly and independently, without support from the BCBA trainer. Incorrect responses were defined as steps completed by the therapist but requiring assistance of the BCBA trainer through modeling. Not applicable was circled if the step did not apply to the current session. The BCBA trainers collected the primary fidelity data during live telehealth sessions and trained graduate students collected reliability data from screen recordings of each session. For the therapist whose client did not consent to video recording, a second rater collected reliability data during the live sessions. Before collecting primary or reliability data, the independent observers were trained on the data collection procedures by the expert trainer until they reached 90% reliability for one session.

Trained graduate research assistants collected interobserver agreement (IOA) for a minimum of 40% of baseline sessions, 33% of intervention sessions, and 100% of novelty probe sessions for each participant (e.g., 40% of baseline sessions for TX, 40% of baseline sessions for MW), with the exception of CJ’s novelty probe. The researchers were unable to collect reliability data during the novelty probe session for CJ as the session recording was corrupted. IOA was calculated using exact agreement method (Cooper et al., 2020) and average scores attained were 100%, 96.6% (range, 88.2–100%), and 96.1% (range, 91.2–100%) for baseline, intervention, and novelty probe sessions, respectively.

Procedural Fidelity of BCBA Trainer

Procedural fidelity was collected for 100% of training sessions and 33% of intervention sessions for each participant. Raters collected procedural fidelity using procedural task lists developed by the expert trainer including pre-preparations for meeting, session procedures, and post-session procedures. Pre-preparations for meeting included items pertaining to whether or not the BCBA trainer asked the therapist if they had watched the required training videos, reviewed the caregiver coaching fidelity rubric, highlighted any steps completed correctly in baseline, reviewed role play script, and solicited and answered therapist questions. The session procedures included if the BCBA trainer waited 5 s before prompting the therapist, used most-to-least prompting across sessions, and interjected only if the client’s behavior is severe and dangerous. Post-session procedures included if the BCBA trainer began with positive feedback, provided corrective feedback, reviewed each step of the rubric, solicited and responded to therapist questions, and provided verbal and written feedback to the therapist.

Trained graduate research assistants collected procedural fidelity of the BCBA trainers by watching recorded videos of the sessions. Items on the fidelity rubric were marked ( +) if the behavior was observed and then a ( −) if the behavior was not observed. The total was summed, divided by the total number of expected behaviors, and multiplied by 100 to obtain a percentage. The resulting procedural fidelity was 100% across all trainers.

Data Analysis

The data for the therapist’s performance on the telehealth fidelity checklist were summarized in graphic form using a multiple baseline across therapists’ design. To assess the difference between therapist’s performance before, during, and after receiving the telehealth training intervention, data were summarized in regard to the percentage of steps completed correctly out of the total number of steps on the fidelity checklist across each session.

Results

Figures 1 and 2 present the results for all therapists. Six out of seven therapists scored 0% during all baseline sessions. One therapist, MW, increased to 6% independence during the second baseline session and remained at that level until intervention started. Once intervention started, all seven therapists met or exceeded the mastery criteria of 90% independence. All therapists’ data show increasing trends for acquisition of skills and therapists reached mastery criteria within three sessions. Moreover, six of the seven therapists had two consecutive data points above 80% independence during the intervention phase.

Fig. 1

Treatment results for TX, MW, and LQ

Fig. 2

Treatment results for CJ, SS, SD, and LI

During the novelty probe, six therapists conducted a telehealth session with a new target family and performed at greater than 90% independence. Specifically, MW, TX, LQ, SS, and LI all performed the novelty probe session with 100% accuracy to the new target client while SD and CJ performed at 97% and 79% correct, respectively. However, it should be noted that during the novelty probe with CJ, the internet connection was lost and unable to be reconnected, so the BCBA trainer finished the session to prevent a hold in services for the client. This resulted in a lower novelty probe score for CJ. Overall, all therapists met mastery criteria in three sessions and were able to perform the newly acquired skills in a caregiver coaching session with a novel family.

Approximately 2 years after the study was completed, a follow-up questionnaire was sent to all therapists to gain information on their current positions and use of telehealth within their practice. Six out of seven therapists responded. In regard to their current certification, four therapists are now BCBAs (SS, LQ, LI, and SD), one is a BCaBA (MW), and the other is a RBT (TX). Two therapists (MW and SS) still conduct telehealth ABA services and reported serving five to 20 families per year. In fact, MW is responsible for conducting all telehealth sessions at her clinic with ongoing supervision from a BCBA trainer that participated in the current study. Of the two therapists who still conduct telehealth, one still uses the telehealth fidelity checklist provided in the current study while the other does not. All of the BCBA trainers from the current study still provide ongoing telehealth ABA services and work with approximately five families per month.

Discussion

The purpose of this study was to evaluate and expand upon previous research on training behavior analysts to conduct caregiver coaching services via telehealth through use of a telehealth training curriculum to prepare RBTs to provide these services. To date, there has been little research in the area of therapist training to provide caregiver coaching on interventions using telehealth services, and those that have, have done so with BCBAs (Lerman et al., 2020; Neely et al., 2022). The current study resulted in seven RBTs being trained to implement caregiver coaching procedures, reaching 100% fidelity within three sessions. Additionally, all therapists were able to demonstrate these skills in working with a novel family. Specifically, the therapists were able to create and set up a HIPAA-compliant Zoom session, coach the caregiver in specific interventions as supervised by a BCBA and resolve any challenges that occurred during sessions.

In this study, we trained RBTs to implement caregiver coaching sessions via telehealth. Although BCBAs should still oversee caregiver coaching sessions, the outcomes of this study speak to the effectiveness of the training curriculum and provide support for further investigation of a curriculum to prepare practitioners in the telehealth modality. Given the positive outcomes demonstrated in the current study, these findings support previous research demonstrating the positive effects of explicit instruction in training practitioners to transition behavior analytic skills from in-person services to a telehealth service delivery model (Lerman et al., 2020; Neely et al., 2022). In light of the COVID-19 pandemic, practitioners rapidly transitioned to telehealth service delivery models with limited guidance and explicit training on the requisite skills needed to effectively do so. Thus, this training curriculum has the potential to serve as a model for universal training requirements for all practitioners delivering services via telehealth.

This study also utilized a pyramidal training procedure as all of the BCBA trainers were previously trained by the expert trainer using the same curriculum. Thus, the results extend previous research and provide preliminary support for the use of a pyramidal training model to prepare RBTs to provide therapy based on the science of ABA via the telehealth modality (Neely et al., 2018). In the current study, RBTs were trained to high levels of fidelity with minimal time required from the expert trainer (BCBA-D). Given the lack of trained practitioners to provide caregiver training via telehealth (Lerman et al., 2020), and the emerging critical need for this model of service delivery (Cox et al., 2020), the use of a pyramidal training model may prove to be a viable solution to bridge the gap between limited resources and high demand for ABA services (BACB, 2021), while also creating more opportunities for individuals to receive technician-delivered telehealth services regularly.

Lastly, results of the current study support findings from previous research on the use of telehealth to expand the availability of ABA services. Telehealth has been used as a modality for providing services to families outside of the USA (Sivaraman et al., 2020) and to underserved communities (Barkaia et al., 2017). The current study aimed to address the global health crisis and help slow the spread of COVID-19 by providing families with alternative treatment options for those who wanted to opt out of in-person treatment due to personal and/or health concerns. Given that all supervision and training was conducted remotely, we likely prevented further transmission of COVID-19 without sacrificing continuity of services for families in need.

Limitations and Future Research

Although the current study had many positive outcomes, it is not without limitations. Most notably, although the primary focus of this study was to replicate and demonstrate the effects of the telehealth training curriculum (Neely et al., 2022) on preparing RBTs to conduct telehealth ABA, we did not collect procedural integrity at the interventionist-level across all families. As such, we cannot draw conclusions about the effectiveness of the therapist-delivered coaching on the interventionist level of skill acquisition for the prescribed behavior analytic procedures. Future research should collect and report fidelity measures across all levels of personnel including the BCBA trainer, RBT therapist, and interventionist’s implementation of the behavior analytic skill with the client. Second, the modifications made to the training procedures, specifically the gating process, may have artificially deflated the baseline performance. That is, the therapists may have performed some of the steps of the coaching session with fidelity during baseline if allowed to conduct that part of the session. Therefore, there is a lack of a functional relation due to differences in measures during baseline versus intervention phases. However, the study does provide preliminary support toward the use of the BST curriculum in training RBTs to use telehealth as a service delivery method. Moreover, it should be noted that this study was conducted during the COVID-19 pandemic and provision of services to the families was priority. Therefore, the research team decided to employ this gating procedure to ensure clinical quality for the families and supervision quality for the therapists.

Given the increasing prevalence and popularity for telehealth delivered behavior analytic services, the current study has several implications for future research. First, we attempted to prepare therapists for challenges that may arise during telehealth sessions by reviewing common challenges and solutions (as outlined by Lerman et al., 2020, Table 1) and rehearsing these scenarios through role play. Rehearsal is an integral component of BST; however, it is unknown whether all of the elements of the training (e.g., checklist, modeling, feedback) are essential components to the telehealth training curriculum. Research has shown that BST can result in generalization of the learned skills to the natural environment, and it is possible that provision of the checklist (without rehearsal of the challenges and solutions that may arise during telehealth) could result in substantial improvements in fidelity. Future research might consider conducting component analyses of the trainings to determine which components are necessary or automating pieces of the training (e.g., video model of a session rather than live model) to better promote large-scale technician training formats. Second, the current study conducted only one novelty probe after the training condition to test for generalization of therapist skills. Given there was an absence of this novelty probe in baseline, we cannot conclude whether or not the RBT training procedures truly improved, or impacted therapists’ generalization of these skills. Instead, the novelty probes provide support toward the use of the newly acquired skills with novel families. Future research should focus on more rigorous methods of generalization by evaluating therapist’s skills with novel families before and after receiving training, as well as across multiple clients and behavior analytic treatment programs (including assessments and interventions).

Taken together, results of the current study provide strong support for the use of the telehealth training curriculum to prepare RBTs to conduct ABA services via telehealth. Future large-scale investigations of this curriculum have the potential to substantially increase the number of practitioners able to provide telehealth ABA services, ultimately resulting in more families accessing and benefiting from behavior analytic services.