Brief Report: Assessment of a Caregiver-Implemented Intervention for Improving Social Communication Skills in Toddlers and Young Children with Autism.

As early identification of autism improves, there is a critical need for interventions to support the development of social communication skills in toddlers. Caregiver coaching and parental involvement is crucial for improving outcomes and providing children with adequate hours of planned active engagement. This pilot study assessed a 4-week intervention for individual caregiver-child dyads. Eight toddlers 21- to 45-months of age participated. Standardized assessments were collected at four study visits to assess autism symptomatology, language development, and both caregiver knowledge and engagement. Results demonstrated the feasibility of the intervention. Social communication, receptive and expressive language all improved as measured by direct assessment. Caregiver knowledge and caregivers' subjective feelings of engagement with their toddlers also improved.

Autism spectrum disorder (ASD) is characterized by deficits in social communication and the presence of restricted and repetitive behaviors (American Psychiatric Association [APA], 2013). Social communication deficits may manifest as difficulty using nonverbal communication, engaging in reciprocal social exchanges, and developing or maintaining relationships (APA, 2013). ASD can be reliably diagnosed as young as 18 months (Zwaigenbaum et al., 2016), and the National Research Council recommends early intervention, in which the child is engaged in developmentally appropriate educational activities related to specified objectives beginning as soon as concerns are identified (2001). Early screening and referrals for these types of interventions are imperative to support the development of positive social behaviors in very young children (Hyman et al., 2020).

A variety of evidence-based interventions targeting core ASD symptomatology exist, including Early Start Denver Model, Learning Experiences and Alternative Program for Preschoolers and Their Parents (LEAP), Naturalistic Intervention, Pivotal Response Training, Picture Exchange Communication System (PECS), and Discrete-Trial Teaching (Wong et al., 2015). Several of these practices have demonstrated efficacy in supporting the development of specific behaviors such as joint attention (Krstovska-Guerrero & Jones, 2015; Schertz & Odom, 2007; Schertz et al., 2013), spontaneous social communication (Fuller & Kaiser, 2020; Wetherby et al., 2014), and spontaneous verbalizations (Gillett & Leblanc, 2007). Despite their effectiveness, available interventions frequently require rigorous time commitments from families consisting of at least 15 hours per week, for 12 weeks. Time demands have been described as a barrier to intervention access, particularly for those in low-income or underserved communities (Carr & Lord, 2016). Additionally, inconclusive findings have emerged regarding the relationship between treatment dosage and study outcomes (Hampton & Kaiser, 2016).

Caregiver training and participation during intervention presents a promising solution to this challenge. Naturalistic interventions and caregiver-implemented interventions are also considered evidence-based practices for young children with ASD (Steinbrenner et al., 2020; Wong et al., 2015). As such, training and coaching of caregivers is often a key feature of interventions designed to support young children with ASD (Dawson et al., 2010; Pickles et al., 2016; Wetherby et al., 2014; Zwaigenbaum et al., 2015). Caregiver-coaching has been shown to increase gains in expressive and receptive communication outcomes for toddlers (Pickles et al., 2016), improve implementation ease, and promote generalization of learned skills (Zwaigenbaum et al., 2015). Additionally, the inclusion of caregivers in ongoing interventions allows children to more easily achieve the recommended 25 hours per week (Estes et al., 2014; Landa, 2018; Maglione et al., 2012; National Research Council, 2001) of systematically planned and developmentally appropriate active engagement. Ongoing investigation of the effectiveness and acceptability of caregiver-delivered training programs also has important implications for families in underserved communities, where there may be fewer opportunities to access early autism interventions (Hine et al., 2020).

In addition to the lack of availability of less time-demanding interventions, there is also a significant gap in the literature regarding effective early intervention for children ages three and younger. Steinbrenner et al. noted that only 9% of studies in the 2020 review of evidence-based practices for individuals with autism focused on children birth to 35 months of age (Steinbrenner et al., 2020). Our pilot study included children eligible to receive early intervention services prior to entering the public education system (at age three).

This pilot study examined the efficacy of an intervention designed to teach caregivers strategies to increase their child’s social behavior and communication skills within everyday play-based routines. As previous research has shown, child outcomes may be improved when caregivers receive individual, rather than group, coaching (Wetherby et al., 2014), which is the approach used in the current study. The intervention is also less time-demanding compared to presently available programs. The condensed time and focused targeted skills taught may be a motivating factor in caregiver commitment. The primary goal of this study is to assess the feasibility of teaching caregivers the skills needed to integrate targeted objectives into everyday activities using a low-burden format. Secondary aims sought to evaluate the effectiveness of the intervention by: (1) evaluating change in caregiver knowledge and engagement, and (2) evaluating change in ASD symptoms, with a focus on social communication behaviors targeted throughout the intervention. We hypothesized that caregiver knowledge of concepts covered (e.g., supporting language and engagement, creating routines and taking turns, positive reinforcement) would improve following the intervention, and that caregivers would report improved engagement with their children. In addition, we hypothesized that ASD symptoms and social communication would improve as a result of the intervention.

Methods

Participants

Eight toddlers and children between 21 and 45 months of age (M = 31.75, SD = 8.43; 2 female) whose caregivers reported developmental delays enrolled in this study to receive the Spring into Action Together (SIAT) intervention (Table 1). Five had confirmed ASD diagnoses at enrollment, one was diagnosed shortly after concluding study participation, and two were referred by early childhood educational programs due to social and communication delays. One primary caregiver was identified to participate in all sessions and to complete caregiver questionnaires and interviews; seven mothers and one grandmother participated. In terms of education, all mothers had obtained at least partial college training or an associate’s degree (n = 3 partial college, n = 3 standard college, n = 2 graduate/professional training). All fathers had obtained at least partial college training or an associate’s degree (n = 4 partial college, n = 2 standard college, n = 2 graduate/professional training). The average annual household income bracket was $100,000–$149,999. All children were required to screen positive (at risk) on the Modified Checklist for Autism in Toddlers-Revised, with Follow-Up (M-CHAT-R/F; Robins et al., 2009) to participate; 11 other toddlers were screened and excluded (n = 3 MCHAT-R score below eligibility; n = 2 not on stable treatment; n = 4 lost to follow up; n = 2 not available for in person visits). All participants were required to keep ongoing treatments stable for 12 weeks prior to the baseline visit for this study. This study was approved by the Mount Sinai Program for the Protection of Human Subjects and informed consent was obtained from all caregivers.

Table 1

Baseline participant characteristics

Child age in months, M (SD)	31.75 (8.43)
Female, N (%)	2 (25%)
ADOS-2 Total Scorea, M (SD)	18.63 (5.29)
Social Affect, M (SD)	15 (5.42)
Restricted and Repetitive Behavior, M (SD)	3.63 (1.41)
MSELb: Receptive Language, M (SD)	25.5 (10.49)
Expressive Language, M (SD)	28.5 (10.01)
Vineland-3: Communicationc, M (SD)	60.63 (9.75)
Receptive Languaged, M (SD)	7.38 (2.56)
Expressive Languaged, M (SD)	5.5 (2.73)
Developmental Level
MSEL: Early Learning Compositee M (SD)	65.63 (12.85)
Gross Motor Age Equivalentf M (SD)	20.75 (3.85)
Visual Reception Age Equivalent M (SD)	23.13 (8.58)
Fine Motor Age Equivalent M (SD)	21.75 (5.55)
Receptive Language Age Equivalent M (SD)	16.5 (8.05)
Expressive Language Age Equivalent M (SD)	17.38 (7.09)

ADOS-2 Autism Diagnostic Observation Schedule-Second Edition; MSEL Mullen Scales of Early Learning

aTotal Score on the ADOS-2 is the total sum of Social Affect and Restricted and Repetitive Behavior scores, where a higher score indicates a greater number and severity of ASD symptoms

bScores on the MSEL are represented as T-scores (M = 50, SD = 10)

cVineland-3 Communication Domain is presented as a standard score (M = 100, SD = 15)

dVineland-3 Subdomain scores for Receptive and Expressive Language are presented as V-scale scores (M = 15, SD = 3)

eMSEL Early Learning Composite is presented as a standard score (M = 100, SD = 15), derived from the Cognitive T Score Sums (the sum of Visual Reception, Fine Motor, Receptive Language, and Expressive Language T Scores)

fMSEL Age Equivalent scores are presented in months and provided for each domain

Procedures

SIAT is a 4-week caregiver coaching program implemented by trained therapists that consists of two 30-min sessions per week. Caregivers are present for the duration of each session. Caregiver coaching involved teaching caregivers strategies to engage their child using modeling, verbal feedback, and explicit examples of the impact of the interaction on the child. Sessions were led by either a board-certified behavior analyst (BCBA), one of two speech-language pathologists (SLP), or a speech-language pathologist clinical fellow (SLP-CF) under the supervision of a licensed SLP. SIAT was developed for toddlers and young children displaying social and/or communication delays, and provides concrete opportunities for caregivers to learn and practice routines targeting key skills including gesture use, engagement with a caregiver and an object or activity, and transitioning between activities. The intervention incorporates evidence-based behavioral strategies including prompting, utilizing natural reinforcers, and modeling (Schreibman et al., 2015; Steinbrenner et al., 2020; Wong et al., 2015), designed to provide caregivers with strategies to increase their child’s social behavior and communication skills. The curriculum was based on the Together You and Me curriculum developed at the University of Miami, Nova Southeastern Center for Autism and Related Disabilities (UM-NSU CARD), further adapted as Project G.R.A.S.P. (Alvarez-Tabio, 2019). The original curriculum was designed to be implemented in a small group setting or to individual caregiver–child dyads. In this study, the intervention was delivered to individual dyads to maximize direct, individualized coaching. Given the manualized nature of the intervention, strategies taught during each session were consistent across participants and designed to support the development of social, communication, and play skills. Each week included a standard series of play-based structured activities repeated across two sessions. The activities and materials were identical across participants and presented in the same sequence. Caregivers were provided with an information sheet at the beginning of each session which was verbally reviewed by the clinician. Social communication targets were determined with the caregiver and therapist during each session and were dependent on the child’s current communication level. The intervention did not include a target number of social behavior and communication opportunities for the caregiver; however, the number of times that the therapist had to model a targeted routine was standardized (no more than five times before encouraging the caregiver to engage in the same routine with the child). The goal for the caregiver was to provide as many opportunities to engage in the identified routine with their child as possible during the time allotted for that particular activity. Gesture use was a key target. Caregivers were given information about the importance of gesture use as well as developmentally appropriate expectations for use of gestures. Caregivers were then provided with verbal feedback on their implementation of concepts.

Participants were assessed at four study visits over a 12-week period. Study visits included: 4 weeks prior to the intervention (baseline; week 0), immediately before the first session (start of treatment; week 4), immediately following the last session (end of treatment; week 8), and 4 weeks post-treatment (follow-up; week 12). Table 2 shows data collected at each study visit (Table 2). Caregivers were asked to keep their child’s ongoing services stable for 12 weeks prior to enrollment, and asked not to change the type or quantity of services for the duration of the study period.

Table 2

Data collected across study period

	Baseline (week 0)	Start of treatment (week 4)	Intervention (weeks 4–8)	End of treatment (week 8)	Follow-up (week 12)
Caregiver questionnaires
Knowledge		√		√
Engagement		√		√
Fidelity checklists			√
ADOS-2	√	√		√	√
MSEL	√	√		√	√
Vineland-3	√	√		√	√
MCDI	√	√		√	√

ADOS-2 Autism Diagnostic Observation Schedule-Second Edition; MSEL Mullen Scales of Early Learning; MCDI MacArthur-Bates Communicative Development Inventory

Measures

The 10-item Caregiver Knowledge Questionnaire (Online Resource 1) assessed the caregiver’s knowledge of key concepts covered during the intervention and within handouts provided after each session (e.g., gesture use, positive reinforcement, creating routines, supporting language and engagement, generalizing skills into everyday experiences). Scores range from 0 to 100%. The 6-item Caregiver Engagement Questionnaire (Online Resource 2) asked caregivers to rate different types of interactions with their child on a five-point Likert scale. Scores range from 0 to 30. The Caregiver Knowledge Questionnaire was created to assess knowledge of target concepts, and The Caregiver Engagement Questionnaire was a tool included in the original curriculum.

The Autism Diagnostic Observation Schedule-Second Edition (ADOS-2; Lord et al., 2012) assesses communication, social interaction, play, and restricted and repetitive behaviors. Participants were assessed with the ADOS-2 Module 1 or Toddler Module based on age and language level. The Mullen Scales of Early Learning (MSEL; Mullen, 1995) provides raw scores and age equivalents in Gross Motor, Visual Reception, Fine Motor, Receptive Language, and Expressive Language domains. An Early Learning Composite reflects a child’s overall developmental functioning. For measuring change, we used raw scores to be maximally sensitive to movement over our short (4-week) interval. The Vineland Adaptive Behavior Scales-Third Edition (Vineland-3; Sparrow et al., 2016) assesses Communication, Daily Living Skills, Socialization, and Motor Skills in everyday settings. The Adaptive Behavior Composite was calculated to assess overall adaptive skills. Finally, the MacArthur-Bates Communicative Development Inventory (MCDI; Fenson et al., 2007) assessed receptive language, social communication (including gestures), and early vocabulary. To assess change over time, we focused on the number of phrases understood (out of 28 phrases), number of words understood (out of 396 words), and number of words produced (out of 396 words).

A fidelity checklist evaluated the fidelity of intervention administration and included items that measured whether the interventionist: (1) implemented each routine in the specified time interval, (2) provided opportunities and feedback to the child and caregiver, (3) reviewed key concepts, and (4) provided the caregiver with the handout for each session. Fidelity checklists were completed by trained research assistants from behind a one-way mirror during sessions or from taped sessions.

Results

All eight participants completed eight intervention sessions. Four participants were unable to complete the final follow-up study visit (week 12), two due to the COVID-19 pandemic, and two due to significant changes in ongoing treatment services as a result of starting preschool immediately upon intervention completion.

Caregiver Knowledge and Engagement

A paired-sample t-test was conducted to examine differences in caregiver knowledge pre- (M = 54.29%, SD = 21.5) and post-treatment conditions (M = 88.57%, SD = 13.45). The t-test revealed a significant difference between pre- and post-test scores, M difference = 34.29%, SD = 17.18, t(6) = − 5.28, p = 0.002. Following treatment, 85.71% of caregivers (n = 6) scored 90% or higher on the knowledge questionnaire following treatment; as compared to 12.5% (n = 1) meeting this threshold pre-treatment.

A paired-sample t-test was conducted to examine differences in caregiver-reported engagement with the child pre- (M = 25, SD = 3.35) and post-treatment (M = 28, SD = 1.79). The t-test revealed significantly higher engagement ratings post treatment (M difference = 3, SD = 2.37, t(5) = -3.11, p = 0.027).

Autism Symptomatology

Baseline ADOS-2 scores were all above the autism (n = 7) or autism spectrum (n = 1) cutoffs. ADOS-2 Total Scores trended down across the intervention, suggesting a decrease in ASD symptomatology. Specifically, Social Affect scores decreased across the study period (Fig. 1), though Restricted and Repetitive Behaviors, on average, remained constant. To examine change over time, change scores were calculated for each individual across the three intervals as described above. One-sample two-tailed t-tests were conducted on change scores, which showed significant improvement across the treatment interval for Social Affect (M difference = − 2.25, t(7) = − 2.55, p = 0.038, Cohen’s d = − 0.90) (Table 3). Between baseline and start of treatment (M difference = − 0.625, t(7) = -0.58, p = 0.582, d = − 0.20) and between end of treatment and follow-up (M difference = − 0.25, t(3) = − 0.52, p = 0.64, d = − 0.26), Social Affect scores did not significantly change. There was no significant change in Restricted and Repetitive Behaviors during any of the three intervals (all p values ≥ 0.7).

Fig. 1

Social Affect scores on the ADOS-2 across the study period, where a higher score indicates a greater number and severity of ASD symptoms. Social Affect scores decreased across the study period, with a significant change in ASD symptoms within the treatment interval

Table 3

Mean difference in change scores across study intervals

M difference	Baseline—start of treatment	Start of treatment—end of treatment	End of treatment—follow-up
ADOS: Total Score	− 0.63	− 2.5††	− 0.25
Social Affect	− 0.63	− 2.25*††	− 0.25
RRB	0.00	0.25	0.00
MSEL: Receptive Language	− 1.38†	1.13**††	0.50
Expressive Language	− 0.88	2.88*††	− 0.75††
Vineland-3: Communication	3.38*††	5.25	3.40†
Receptive Language	0.75†	1.38†	0.60
Expressive Language	0.75†	0.38	0.40
MCDI: Phrases Understood	4.00†	− 0.29	2.67††
Words Understood	13.0	44.57†	45.0††
Words Produced	11.57	21.14	16.33††

ADOS-2 Autism Diagnostic Observation Schedule-Second Edition; MSEL Mullen Scales of Early Learning; MCDI MacArthur-Bates Communicative Development Inventory

*p < 0.05; **p < 0.01; †Cohen’s d > 0.5; ††Cohen’s d > 0.8

Language Development

MSEL raw scores in Receptive and Expressive Language trended upwards, suggesting improvement in communication across the study period (Fig. 2A, B). One sample t-tests were conducted on change scores for each interval. For Receptive Language (Fig. 2A), there was no significant change from baseline to start of treatment (M difference = − 1.38, t(7) = − 1.67, p = 0.14, d = − 0.59), nor from end of treatment to follow-up (M difference = 0.5, t(3) = 1, p = 0.39, d = 0.5). However, there was a significant improvement across the treatment interval (M difference = 1.13, t(7) = 3.81, p = 0.007, d = 1.35). For Expressive Language (Fig. 2B), there was no significant change from baseline to start of treatment (M difference = -0.88, t(7) = − 1.31, p = 0.23, d = − 0.46) or from end of treatment to follow-up (M difference = − 0.75, t(3) = − 3, p = 0.058, d = − 1.5). There was, however, a significant improvement during the treatment interval (M difference = 2.88, t(7) = 2.63, p = 0.034, d = 0.93).

Fig. 2

MSEL Receptive (A) and Expressive (B) Language scores expressed as raw scores across the study period, where a higher score indicates increased communication. Receptive (A) and Expressive (B) Language improved across the study period, with statistically significant improvement within the treatment interval. MSEL Mullen Scales of Early Learning

Vineland-3 standard scores in the Communication domain trended upwards across the study period (Fig. 3). One-sample t-tests were conducted on change scores and showed a significant increase between baseline and start of treatment (M difference = 3.38, t(7) = 2.43, p = 0.045, d = 0.86), but improvement was not significant during the treatment interval (M difference = 5.25, t(7) = 1.41, p = 0.201, d = 0.50) or from end of treatment to follow up (M difference = 3.4, t(4) = 1.51, p = 0.205, d = 0.68). Change scores were calculated for subscales of Receptive and Expressive Language using Vineland V-scale scores (M = 15, SD = 3). The increases in Receptive and Expressive V-scale scores were not significant during any of the three intervals (all p values ≥ 0.08). There were, however, medium effect sizes (d > 0.5) between baseline and start of treatment for both Receptive and Expressive Language, and a medium effect size during the treatment interval for Receptive Language (d = 0.67).

Fig. 3

Vineland Communication standard scores (M = 100, SD = 15) trended upwards throughout the study period but did not show statistically significant improvements within any of the three intervals

MCDI scores for Phrases Understood, Words Understood, and Words Produced trended upwards across the study period (Fig. 4A, B), but there was no significant change on these subscales during the study period (all p values ≥ 0.101). Although change across the study period was not statistically significant, improvement in Words Understood (Fig. 4A) showed a fairly large effect size in the treatment interval (M difference = 44.57, t(6) = 1.93, p = 0.101, d = 0.73). In the period following end of treatment, the magnitude of gains yielded a large effect size (d > 0.8), despite not being significant, for Phrases Understood, Words Understood, and Words Produced.

Fig. 4

MCDI Words Understood (A) and Produced (B) across the study period. Words Understood (A) and Produced (B) increased throughout the study period but did not show statistically significant change in any of the three intervals. Measures of effect size demonstrated a medium effect during the treatment interval for Words Understood (A) and a large effect from end of treatment to follow-up for Words Understood (A) and Produced (B). MCDI MacArthur-Bates Communicative Development Inventory

Fidelity

Rater 1 completed fidelity checklists for 100% of sessions and Rater 2 for 50% of all sessions. Interventionists administered SIAT with an average fidelity of 96%. Fidelity ranged from 85 to 99% across participants. Interobserver agreement was calculated for 50% of sessions and ranged from 91 to 99%. Interobserver agreement (IOA) was calculated by calculating agreements across both raters and dividing by agreements plus disagreements and multiplying by 100 (Table 4). Results indicated Cohen’s k = 0.77 [Table 4].

Table 4

Interventionist fidelity during intervention sessions

Participant	Rater 1a (8 sessions) (%)	Rater 2a (4 sessions) (%)	Rater agreementb (%)
1	94.00	85.38	90.51
2	97.85	93.13	93.96
3	97.00	93.13	94.83
4	98.73	96.58	98.23
5	98.73	96.58	95.69
6	97.88	94.85	97.41
7	98.30	98.30	99.14
8	97.43	97.43	98.28
Total	97.49	94.42	96.01

aRater 1 and rater 2 columns indicate performance on a fidelity checklist (% success) with which the intervention was delivered

bRater agreement indicates agreement between raters 1 and 2

Discussion

This pilot study examined outcomes for toddlers and young children with ASD or ASD symptomatology following SIAT, a 4-week caregiver coaching program implemented by trained therapists. Results demonstrated feasibility of SIAT as a low-burden intervention that may ultimately complement existing treatments. Importantly, SIAT provides caregivers with therapeutic skills to integrate into everyday settings with their child with ASD.

This pilot study sought to examine change in caregiver knowledge. Scores on the Caregiver Knowledge Questionnaire significantly improved following the intervention, suggesting caregivers learned new information about the covered topics. In addition, while caregivers reported high levels of engagement prior to the intervention, average scores on the caregiver engagement questionnaire increased approximately 10%.

The decrease in ADOS-2 Total Scores suggests improvement in ASD symptomatology associated with treatment. The significant decrease within the treatment window, and not between baseline and start of treatment, suggests this improvement was likely due to SIAT, rather than the result of natural development over time or ongoing services. Notably, the ADOS-2 Social Affect score is a strong indicator of improvement in skills that were specifically targeted in this intervention (e.g., gestures, eye contact, shared enjoyment, joint attention, and social overtures) (Alvarez-Tabio, 2019). Similarly, results from the MSEL showed significant improvement in both expressive and receptive language skills only during the treatment interval, with scores stable across the baseline period. In contrast, scores on the Vineland and MCDI did not support significant changes in language ability isolated to the intervention period. However, medium effect sizes (d > 0.5) during the treatment interval for Vineland Receptive Language and MCDI Words Understood, and large effect sizes (d > 0.8) from end of treatment to follow-up on all three MCDI subscales (Phrases Understood, Words Understood, and Words Produced), suggest that a larger follow up study is warranted. Results also highlight how intervention-related change can be more subtle and difficult to capture statistically when they are added to continued, naturally occurring developmental gains in the background.

Notably, the intervention was administered with high levels of fidelity, suggesting feasibility of the intervention for interventionists with varied clinical backgrounds, including behavior analysts and speech-language pathologists. Additionally, SIAT requires a shorter in-person time commitment than currently available empirically supported interventions, and utilized play-based activities that are likely to be found in the child’s natural environment. This allows for repeated opportunities to incorporate key concepts and increase practice for children and caregivers in their everyday routines, all within a one-month duration.

This study is limited by the small sample size, the lack of a follow up visit for a subset of participants, and the lack of control group or examiners/caregiver raters blind to treatment. Future studies should also incorporate measures of caregiver stress and anxiety. While results cannot be generalized due to these limitations, improvement on standardized clinician-administered assessments specific to the treatment period suggests that the SIAT intervention warrants further study and may hold value as a novel, cost-effective, manualized treatment option for young children. An intervention of this nature has the potential to allow ongoing caregiver-directed coaching in a naturalistic setting, as well as increased accessibility due to the decreased time demand. Future studies should be conducted in larger samples, using blind examiners, and using a randomized wait-list or active control group design.