A Randomized, Community-Based Feasibility Trial of Modified ESDM for Toddlers with Suspected Autism.

A randomized feasibility trial of a parent coaching (PC) intervention was conducted across 16 community agencies in a Canadian province. Parents of toddlers with suspected autism were assigned to either a PC group (n = 24) or an enhanced community treatment (ECT) group (n = 25). PC participants received 24 weeks of coaching support from community service providers trained in the project. Children in both groups also received available community services and supplementary materials. PC children made significantly greater gains in word understanding and PC parents had significantly higher quality of life, satisfaction, and self-efficacy scores. Results are discussed in terms of the challenges of conducting feasibility studies in community settings and the lessons learned in the project.

Naturalistic developmental behavioral interventions (NDBIs) are derived from both the principles of behavioral learning and the developmental sciences and have been used primarily with young children on the autism spectrum1 (Schreibman et al., 2015). Research investigating effectiveness outcomes of the Early Start Denver Model (ESDM), one of the oldest and most well-established of the NDBIs (Rogers & Dawson, 2010), has been evaluated in two systematic reviews (Baril & Humphreys, 2017; Waddington et al., 2016) and a recent meta-analysis (Fuller et al., 2020). The authors of all three evaluations concluded that ESDM is a “promising” intervention for young autistic children, with positive outcomes in the cognitive and language domains in particular. A subset of studies included in the reviews and several subsequent studies focused on parent-implemented ESDM (i.e., P-ESDM), in which parents are coached to deliver the intervention. Several of these studies found that coaching was effective in improving parent use of ESDM techniques and that child outcomes also improved in one or more domains (e.g., Vismara et al., 2009, 2012, 2013; Waddington et al., 2021; Zhou et al., 2018). Other P-ESDM research found outcomes in favor of parent change but not child change (e.g., Rogers et al., 2019, 2020; Vismara et al., 2018). Finally, one P-ESDM study found no significant difference between coaching and control groups for either parent or child outcomes (Rogers et al., 2012a, b).

Most NDBI research to date has examined the efficacy of various intervention models under highly controlled conditions that include the use of experienced interventionists who implement a treatment with a high degree of fidelity, in contexts that are designed to eliminate or reduce confounding variables (Robey, 2004). Efficacy studies are essential in order to establish treatment effectiveness under optimal conditions. Thus, in many published ESDM studies, interventions were delivered in university-based clinics, even when parents were the primary implementers. Exceptions were those in which ESDM was delivered in community child care settings such as preschools or daycare centres (e.g., Eapen et al., 2013; Fulton et al., 2014; Tupou et al., 2020; Vivanti et al., 2014, 2019), family homes (e.g., Holtzinger et al., 2019; Rogers et al., 2019; Waddington et al., 2021), or via telehealth (Vismara et al., 2009, 2012, 2013, 2018). In all but one of the P-ESDM studies mentioned previously (Zhou et al., 2018), parent coaching (PC) was provided either by the model developers or by professionals (typically, with graduate degrees) who completed a multi-component training program and demonstrated a high level of implementation fidelity. While such rigorous oversight is certainly optimal, it is not generally available in community-based service programs or early childhood settings. Studies that examine intervention effectiveness—the outcomes that can be achieved in real-life settings, under less-than-optimal environmental and staffing conditions (Robey, 2004)—are also needed if P-ESDM and other parent-implemented NDBIs are to become widely available to children and families who are supported primarily by early childhood paraprofessionals in community settings.

Another limitation of the current body of ESDM research has to do with the inclusion/exclusion criteria applied to parents. Many studies excluded children whose primary caregiver met one of the following criteria: (a) self-reported substance abuse and/or significant mental illness (e.g., Rogers et al., 2012a, b, 2019); (b) lived farther than a specified distance from the University clinic that housed the study (e.g., Rogers et al., 2014, 2019); and/or (c) did not meet various criteria for spoken and/or written English language ability (e.g., Rogers et al., 2020; Vismara et al., 2018). While none of these criteria is unreasonable in the context of a research efficacy study, they become untenable when applied to publicly-funded community services where families rightfully expect equitable access regardless of their child’s diagnosis, their own language/literacy skills, and their ability to participate with regularity. In particular, the “specified distance from clinic” criterion limits the extent to which an intervention such as ESDM is available to families in rural communities who may not have reliable internet access and often receive home-based support from itinerant community service providers (Suppo & Floyd, 2012).

Given the promising results of ESDM efficacy studies (Fuller et al., 2020) since publication of the first randomized controlled trial (Dawson et al., 2010), further evaluation of the challenges and opportunities of adoption and scale-up of this NDBI model in widespread community settings seems timely. As noted by Dingfelder and Mandell (2011), “an innovation’s reception is dependent on social context…[and] social forces consistently trump unvarnished effectiveness” (p. 598). In community-based early intervention programs, such “social forces” include the availability of qualified and trained staff, the cost of offering an innovation within existing funding constraints, and the acceptability of the innovation to both staff and their constituents (i.e., families). In combination, these training, implementation, and cost factors must be addressed in a community effectiveness model.

One approach to evaluating the effectiveness of a community-based intervention is to conduct small-scale feasibility or pilot studies that take real world constraints into consideration from the outset. In their discussion of a conceptual framework for such studies, Eldridge et al. (2016) acknowledged the ambiguous distinction between a feasibility study and pilot study, noting that “all pilot studies are feasibility studies but not all feasibility studies are pilot studies” (p. 8). They suggested that either term can be applied to studies that ask “whether something can be done, should we proceed with it, and if so, how” (p. 1) by conducting small-scale experimental trials. Several such studies have been conducted to examine the feasibility of standard or modified versions of ESDM in community settings. For example, Vivanti et al. (2014) investigated the feasibility of a group-based version of ESDM (G-ESDM) that was provided in community childcare settings by preschool staff certified in ESDM. Holtzinger et al. (2019) examined the feasibility of low-intensity, standard ESDM in Austria; the intervention was delivered primarily in children’s home environments by local, certified ESDM therapists. Finally, Abouzeid et al. (2020) investigated the feasibility of low-intensity, standard P-ESDM for children on a waitlist for intensive autism services in Quebec, Canada. The intervention was delivered in community centres by parent coaches from a local community agency who were trained but not certified in ESDM. Results of all three studies confirmed the feasibility and acceptability of the intervention and also found improvements in parent (Abouzeid et al., 2020) or child outcomes (Holtzinger et al., 2019; Vivanti et al., 2014).

It is well established that early autism diagnosis and intervention are crucial and have the potential to yield better developmental outcomes (Zwaigenbaum et al., 2015). However, in Canada, families may wait up to 2 years after referral to obtain a formal autism diagnosis (Penner et al., 2018). The result is a delay in parents’ ability to access funding for specialized early intervention services that require an autism diagnosis. One way of addressing this problem is to offer a “preemptive intervention” (Insel, 2007, p. 5) to young children who are at increased likelihood of autism either because they have an older autistic sibling or because they show a pattern of atypical behavior suggestive of autism (Zwaigenbaum et al., 2009). Preemptive interventions, as the name implies, are designed to mitigate developmental risks and modify early behavioral trajectories, not eliminate a condition. Brian et al. (2017) examined the short-term efficacy of a parent-mediated intervention for toddlers aged 16–30 months with confirmed or suspected autism diagnoses and found child gains related to vocal initiations in the experimental group. Both Green et al. (2017) and Whitehouse et al. (2021) examined the longitudinal efficacy of preemptive interventions for even younger children (age 9–14 months) with suspected autism. Both studies found small effects for reduced autism symptom severity at age 3 for children in the experimental groups. Similar results were also reported in a quasi-experimental study of modified P-ESDM for infants with suspected autism (e.g., Rogers et al., 2014).

In line with these efforts, Rogers et al. (2020) conducted a randomized feasibility trial aimed at addressing the need for preemptive autism supports in low-resourced rural and urban communities. Early intervention providers in participating community agencies were randomized to receive streamlined training in either a modified version of ESDM (called Community ESDM, C-ESDM) or a comparison intervention. Parents of toddlers with suspected autism (age 12–30 months) received either C-ESDM coaching support or active support by early intervention providers who had access to a series of publicly available online modules about child development. Results indicated that C-ESDM parents demonstrated statistically significant skill gains relative to the comparison group, although there were no short-term differences in child change. Nonetheless, this study demonstrated the feasibility and potential utility of a version of ESDM that was specifically designed for toddlers with suspected autism in under-resourced communities.

British Columbia (BC) is a western Canadian province that spans 944,735 km (364,764 miles, between Alaska and Texas in size) and has an estimated population of 5.1 million people (Statistics Canada, 2020), approximately 40% of whom live outside of large urban population centres (Statistics Canada, 2016). In BC, children with suspected autism are referred to regional teams for multidisciplinary diagnostic assessments that include the Autism Diagnostic Observation Schedule and other measures. Because the demand for such assessments far exceeds the professional resources available, waitlists for assessment have been in excess of 70 weeks for several years (see http://www.phsa.ca/our-services/programs-services/bc-autism-assessment-network). The result is that access to early autism intervention is also delayed, as families are unable to access government funding for these specialized services without an autism diagnosis.

In order to address this concern, and in line with other efforts to examine the impact of a preemptive intervention prior to autism diagnosis, we conducted a community-based research project over a 3 year period. The overall project, named Parent and Child Early (PACE) Coaching, consisted of several components,2 one of which was to examine the feasibility of providing a modified, community-based version of P-ESDM to parents of toddlers with suspected autism. In this paper, we present the outcomes of a randomized feasibility trial that compared PACE Coaching outcomes to those of a comparison group. The trial was an extension of the C-ESDM model across a large geographic region in Canada, with considerably more community partner involvement in all aspects of recruitment, screening, and data collection. Our hypotheses were that, relative to those in the comparison group, (a) parents receiving PACE Coaching would demonstrate more skilled use of scaffolding and following-in on the child’s focus [referred to herein as “following-in”], have higher quality of life and lower parent distress scores, and have higher parent satisfaction and self-efficacy scores; (b) children receiving PACE Coaching would demonstrate greater gains in the number of words understood and words produced, parent–child interaction skills, and developmental index scores; and (c) change in one or more child variables during parent–child interactions would be predicted by parent change in scaffolding and following-in.

Method

All components of this project were approved by the Institutional Review Boards of participating universities and, when required, by the review boards of participating community agencies. All participants provided informed consent.

Background and Pilot

In consultation with ESDM developers, PACE Coaching was modified from standard P-ESDM in two primary ways. First, a simplified assessment and curriculum measure (Rogers et al., 2013) that was first used in the C-ESDM study mentioned previously (Rogers et al., 2020) was used to identify child skills that were the focus of PC. This consisted of target skills in multiple domains, including receptive and expressive communication, joint attention, imitation, social skills, cognition, and play skills. Second, parent coach fidelity was assessed via a modified version of the standard P-ESDM coach rating tool (Rogers et al., 2021). These modifications were made in order to focus on core social communication skills and to accommodate the personnel and resource constraints that were inherent in the community settings where the project was conducted.

We first undertook an informal pilot project with the assistance of three certified ESDM therapists who were later responsible for supporting the coaches involved in the randomized feasibility trial. The pilot was designed to examine the acceptability, implementation, and practicality of (a) the PACE Coaching modified curriculum; (b) the anticipated measurement plan; and (c) providing remote (i.e., online) coaching to parents. We enlisted the assistance of two community agencies to identify parents of children who were not eligible for the feasibility study because they were older than 36 months of age and/or were already diagnosed with autism. Three such families were identified, consented to participate, and completed baseline measures at the community agency with a member of the research team. One family was assigned to each therapist, who visited their assigned family at home to conduct an initial assessment of the child’s skills, identify parent priorities, and teach parents how to use the secure online platform we planned to use for remote coaching. Following this, the therapists planned to deliver one 75-min and one 60-min remote coaching session per week for 12 weeks. We also planned to re-administer the assessment battery with each family at the midpoint of the coaching period (~ 6 weeks) and at the end point.

We made three modifications to the original research plan based on feedback during the pilot. First, none of the three families—all of whom were highly motivated and eager to participate—was able to accommodate two coaching sessions per week. This became apparent in the first month of the pilot and required a revision to one 60-min coaching session and one brief telephone follow-up session per week for 24 weeks, in order to accommodate the original target of 24 coaching sessions. Second, the Directors of the two community recruitment agencies told us that they would be able to rearrange internal therapy schedules to accommodate room for two but not three project assessments. Parents also told us that three assessment sessions would be unacceptably burdensome for them. Based on this information, we eliminated the midpoint assessment and scaled back to pre- and post-intervention assessments only. Third, based on feedback from both parents and therapists about the challenges of delivering and receiving PC remotely, we decided to eliminate this option and offer in-person coaching only during the research study.

In the end, all three families endorsed their participation in the pilot and believed that their skills and those of their child had improved to various degrees. Post-test measures also indicated both parent and child change on some measures, with considerable variability. After adjustments were made, all parents, therapists, and agency leaders reported that the coaching and assessment schedules were realistic and manageable.

Recruitment

In many BC communities, families of children with developmental disabilities and/or suspected autism receive therapy and support services from Child Development Centres (CDCs) that are supported in part by government funding. While not a requirement, many CDCs belong to a provincial agency, the BC Association for Child Development and Intervention (BCACDI), whose mission is to advocate for quality services and act as a coordinating body for member agencies. The research team contacted the Executive Directors (EDs) of 20 of the existing 32 BCACDI member agencies across small, medium, and large population centres, to introduce the study and inquire about their willingness to meet and discuss participation. Following initial meetings, the EDs of 16 agencies collaborated with the research team to invite parents to have their child screened for autism. Approximately 1000 one-page introductions to the PACE Coaching project and over 4000 parent flyers with local CDC contact information were distributed to physicians, other health professionals (e.g., speech–language pathologists, public health nurses), and directors of community agencies that provided parent support in each community. Recipients were asked to post the flyers in waiting rooms and clinics, to inform parents about the availability of autism screening at the CDC. In small communities, recruitment efforts were also supplemented with parent-targeted postings via social media.

Autism Screening

Graduate-level clinical specialists in autism (e.g., speech–language pathologists, occupational therapists) who were nominated by the ED of each agency were trained by the project team in order to complete all tasks related to recruitment, screening, and assessment. In preparation for screening, agency specialists completed three online modules: (a) a Course in Research Ethics (CORE) that is mandated at all Canadian Universities (Canadian Institutes of Health Research, Natural Sciences and Engineering Research Council of Canada, and Social Sciences and Humanities Research Council, 2018); (b) an introduction to the project; and (c) a recruitment and screening module that provided step-by-step instructions for identifying eligible children, inviting their parents to participate in screening, obtaining informed consent, and conducting screening with the Modified Checklist for Autism in Toddlers-Revised with Follow-up (M-CHAT-R/F; Robins et al., 2009). The M-CHAT-R/F is two-stage instrument that is easy to administer and has been validated for use with toddlers (Robins et al., 2014). Agency specialists were required to achieve passing scores (> 90% correct) on online quizzes related to both the CORE and recruitment/screening modules prior to initiating the screening process. They then applied eligibility criteria to all children in the agency database and to the children of all parents who contacted the agency on the basis of a project flyer or other forms of advertisement (e.g., social media).

Screening Eligibility

Children were eligible for screening if: (a) they were not previously diagnosed with autism, (b) a parent or early intervention professional had concerns about their social-communication development, and (c) they were between 15 and 36 months of age (extended from 30 months). Exclusion criteria included a genetic or chromosomal disorder, neuromotor disorder (e.g., cerebral palsy) that significantly interfered with movement, severe hearing and/or vision impairment that was uncorrected with aids, serious medical condition that interfered with daily functioning, seizure disorder that was not well-controlled, and/or severe behavior problem that presented a danger to self or others or interfered with daily functioning (e.g., self-injurious behavior). Children would also have been excluded if their caregiving parent was unable to speak, understand, and read/write in English, unless a family member living in the same household was available and willing to act as a translator; however, this criterion was never applied. Parents of eligible children were invited for screening and those who consented were then assisted by the agency specialist to complete the M-CHAT-R/F according to the established protocol, including follow-up questions when indicated (see www.mchatscreen.com).

Randomized Controlled Trial

Trial Design

The trial was conducted in 16 community agencies. The same agency specialists who participated in screening completed an online invite-and-consent module and associated online quiz prior to engaging in the consent process with families. Other than a score indicating increased autism likelihood on the M-CHAT-R/F, no inclusion or exclusion criteria in addition to those required for screening were applied to children. However, parents were required to have at least one home address and access to a telephone so that agency staff could contact them as needed during the trial. This criterion was not required for screening, which involved only a single parent contact.

The trial employed a modified intent-to-treat design (Gupta, 2011), with data collected from each family prior to random assignment (Time 1, T1) and following completion of the study period (Time 2, T2). The design excluded data from parents who withdrew prior to the T2 assessment, but otherwise included data for all parents and children, even when study protocols were violated or data were incomplete (e.g., parents and children who could not be assessed in person at T2 because of COVID-19 restrictions). The advantage of this approach is that it maximizes the sample size, limits inferences based on arbitrary or ad hoc subgroups of participants in the trial, and allows for the greatest generalizability (Moher et al., 2001). However, the estimate of treatment effect is generally conservative because noncompliant and compliant participants are mixed together in the final analysis and data may be incomplete, resulting in potential dilution of effects (i.e., Type II error).

Randomization and Allocation Concealment Strategies

Parents who were invited and consented to the RCT completed T1 assessments (described in a subsequent section) prior to randomization. Because socioeconomic status varies widely from region to region across the province (Human Early Learning Partnership, 2009), we assigned parents at random to the PC and enhanced community treatment (ECT) groups within each CDC to control for extraneous variables (e.g., economic hardship, availability of public transportation).

T1 assessments were conducted by graduate-level agency specialists prior to randomization; thus, assessors were blind to group allocation at T1. The Principal Investigator, who had no direct involvement in any aspect of intervention, used an online randomization tool to create randomly ordered community lists and merged the list with the results of a second tool that created randomly permuted blocks of four, with two PC and two ECT slots in each block. After all T1 assessments were competed for a family, she assigned the family to the next available slot in the community block and notified them by phone of the assignment, with a follow-up email. Follow-up emails were also sent to the leadership team of the community agency. If the family was in the ECT group, the agency specialist involved in the project contacted the family with information about services available at the agency. Regardless of group assignment, all parents were also provided with information about how to obtain a referral for an autism diagnostic assessment. If the family was in the PC group, the local leadership team made a decision about the parent coach assignment and notified the research team about this decision.

Sample Size

In a meta-analysis of 12 ESDM efficacy studies, Fuller et al. (2020) calculated an aggregated effect size across primary measures as Hedge’s g = 0.357. We converted this effect size statistic to a Cohen's f = 0.18 for easier application to a repeated measures ANOVA framework. We conservatively estimated r = 0.50 as the T1–T2 correlation among the primary and secondary repeated measures. Using these parameters, an a priori power analysis via G*Power 3 (Faul et al., 2007) indicated that a total sample size of 64 participants would be sufficient to detect a small to medium effect size of f = 0.18 with 80% power. Sixty-four families consented to participate but, because of attrition between consent and T2 assessments, only 49 completed the study. A post hoc power analysis with the final sample size, using the same parameters, yielded an estimated power of 70%. Although slightly below the idealized threshold of 80% power, a sample size of 49 was deemed sufficient to detect small to medium effect sizes (Cohen’s d = 0.36 or greater) with reasonable power.

Parent Coaching (PC) Group

Thirty-two families were assigned to the PC group. During the intervention period, six parents withdrew from the study. Four withdrew because of multiple family challenges (e.g., domestic violence, low income, health crises); one withdrew because of pregnancy complications that required long-term bedrest; and one was unable to meet the time requirements of the study, despite multiple adaptations. Although some PC children were diagnosed with autism during the study, none of the withdrawals occurred post-diagnosis. In addition, two parents completed the entire PC intervention but did not complete T2 assessments; one of these parents declined to participate and the other moved unexpectedly out of the country. Finally, two parents completed the intervention shortly before the termination of services in our partner agencies due to COVID-19. We were able to obtain parent report measures from both families and parent–child interaction videos from one, so we included their data. In the end, T2 data were available, at least in part, for 24 PC families.

Enhanced Community Treatment (ECT) Group

Thirty families were assigned to the ECT group. One parent withdrew from the study at the outset because of dissatisfaction with the group assignment. In addition, four parents did not respond to requests to complete the T2 assessments; no information about the diagnostic status of these children was available. Finally, one parent finished the intervention period shortly before COVID-19 and completed parent report measures only. In the end, T2 data were available, at least in part, for 25 ECT families. Table 1 summarizes key demographic data for PC and ECT families who completed the study and for those who withdrew. There were no significant differences (p < 0.05) for families who did and did not complete the trial in the distributions of any child or parent demographic variable or for T1 child and parent scores in either treatment group.

Table 1

Child and family demographics for PC and ECT families at T1

Variable	Subcategory	PC group (n = 24)	ECT group (n = 25)
Mean child age at T1		27.1 months (SD 4.7)	24.7 months (SD 4.9)
Child sex assigned at birth	Male	66.7%	83.3%
M-CHAT-R/Fa at T1		9.4 (SD 3.6)	8.4 (SD 3.3)
Primary caregiver	Birth parent	95.8%	80.0%
	Grandmother; adoptive or foster mother	4.2%	20.0%
Primary caregiver M age at T1		32.4 years (SD 6.9)	36.2 years (SD 7.6)
Primary caregiver education	High schoolb	37.5%	16.0%
	Trade/technical/collegeb	50.0%	56.0%
	University undergraduateb	12.5%	16.0%
	University graduate	0	12.0%
Primary caregiver employment	Full time work	8.3%	8.0%
	Part time work	29.2%	24.0%
	Unemployed	62.6%	68.0%
Partner living with primary caregiver		83.3%	96.0%
Residence population type	Large urban (≥ 100,000)	45.8%	48.0%
	Medium (30,000–99,999)	29.2%	40.0%
	Small (1000–29,999)	25.0%	12.0%
Annual household income	< $30,000	12.5%	12.0%
	$30,001–$80,000	62.5%	48.0%
	$80,001–$100,000	20.8%	24.0%
	> $100,000	4.2%	16.0%
Primary languages spoken at home	English only	79.2%	64.0%
	English and another	20.8%	36.0%

aM-CHAT-R/F scores: 3–7 = medium likelihood of autism; 8–20 = high likelihood of autism

bAttended or completed

Measures and Data Collection

Parent and child measures were administered at two time points: prior to randomization (T1) and following completion of the intervention period (T2); assessors were blind to group assignment at T1 but not at T2. In most cases, the measures were administered by the same graduate-level agency specialists who participated in screening (none of whom were parent coaches); when this was not possible, a member of the research team travelled to the site and administered the assessments. Prior to initiation of data collection, all assessors completed three online modules that provided (a) a general overview of the assessment process, (b) instructions for administering the parent report forms and collecting parent–child interaction videos, and (c) instructions for administering the cognitive measure in a standardized fashion. The latter included video examples of all tasks on the cognitive measure, demonstrated by a member of the research team with neurotypical toddlers whose parents consented to video-recording and sharing of the videos within the project. Assessors were required to achieve scores of at least 90% correct on the modules prior to initiating T1 assessments; the modules remained available online throughout the project for review, as needed.

Primary Outcome Measures

Primary (i.e., proximal) measures examined child and parent outcomes that were directly related to the foci of PC. All measures were administered at T1 and T2 by graduate-level agency specialists or research team members who completed the online assessment modules developed for the project, as described previously. All measures were scored by research assistants who were trained to follow standard scoring protocols and were unaware of the group assignment.

MacArthur Bates Communicative Development Inventory (CDI)

Two versions of the CDI were used in this study. At both T1 and T2, all parents completed the Words and Gestures (WG) version (Fenson et al., 1993) a parent report measure that assesses children’s ability to (a) understand and/or produce words from a list of 396 words; and (b) produce various types of early and later communicative gestures. To avoid a ceiling effect, parents also completed the Words and Sentences (WS) version if (a) the child was able to say > 200 words on the WG and/or (b) the child had phrase speech (i.e., was regularly combining words). No child met either criterion at T1 but some did at T2. The CDI: WS provides 680 vocabulary words, including all of those on the WG form, and also includes items related to more advanced language skills. Because the CDIs are intended for children between 16 and 30 months of age and most children in this study were older than this at one or both time points, only raw scores (i.e., not percentile scores) were calculated for this measure. The CDIs were found to have acceptable predictive and concurrent validity when used with young autistic children (Luyster et al., 2007).

Communication Play Protocol (CPP) and Joint Engagement Rating Inventory (JERI)

The CPP (Adamson & Bakeman, 2016) was designed to elicit four different types of parent–child interactions: free play, social interaction/turn-taking, requesting, and commenting. It was selected for this project as a method for assessing both parent and child changes that were reflective of the focus of parent training but were also independent of the specific curriculum that was used. All CDCs were provided with the toys required for the CPP, which was conducted in a quiet room at the participating agency or in the home, based on family preference or agency practice (i.e., home-based services). The agency specialist provided specific sets of toys (with choices) and simple instruction cards for the four interaction scenes to the parent, who then attempted to engage in each type of interaction with the child for 3–4 min, in a standardized order. The specialist video-recorded the interactions and uploaded the videos to a secure cloud platform that met all Canadian and institutional privacy and security requirements.

JERI Coding

CPP videos were scored by the research team using an adapted version of the JERI (Adamson et al., 2019; Suma, Adamson, & Bakeman, 2017) that was developed in collaboration with researchers at Georgia State University, who also trained a research team member as master coder for this project. Adamson et al. (2012) reported high reliability (i.e., 85–88% agreement) when the JERI was used to rate CPP interactions between young children with autism or Down syndrome and a parent/caregiver. They also found moderate to high correlations (r = 0.79, range = 0.56–0.96) between rating items and a variety of corresponding measures for each item.

Each interaction scene was scored with seven different codes, resulting in 28 ratings per set (i.e., four scenes per dyad with seven codes each). Four codes were about the child—three that corresponded to a child’s engagement state (joint engagement; coordinated joint engagement; symbol infused-joint engagement) and a fourth that reflected the child’s expressive language level and use during the interaction. Two items were about the caregiver: scaffolding and following-in on the child’s focus. The final item reflected the shared topic and the dyad’s fluency and connectedness during the interaction.

Supplementary Table 1 summarizes the criteria for assignment of minimum (1), middle (4), and maximum (7) JERI scores that were applied to the CPP scenes. For a child to be considered in an episode of joint engagement (JE), they had to be attending to the same object or event as the parent/caregiver. Hence, JE required a shared topic during a play interaction involving toys. Coordinated joint engagement (CJE) and symbol infused-joint engagement (SJE) were both sub-categories of JE. For CJE, the child had to be actively and repeatedly acknowledging the caregiver’s participation, either through eye gaze or verbally. For SIJE, there had to be evidence that the child was actively attending to symbols (e.g., spoken words or manual signs) or producing symbols during the JE state. Expressive language was evaluated in terms of the number of single words or phrase the child produced during the CPP.

Three JERI codes were used to assess parent/caregiver contributions during parent–child play interactions. They included (a) scaffolding (Scaf; a parent’s ability to provide support of their child’s play around a shared topic), (b) following-in (Follin; a parent’s ability to join, sustain, and build on their child’s interest), and (c) fluency and connectedness (FC; the extent to which a parent–child interaction is fluid and balanced, without being dominated by one partner). Although FC was coded on the basis of the contributions of both members of the dyad to the interaction, it was considered relevant because one of those members was the parent/caregiver.

Research assistants were trained by the master coder and were required to demonstrate inter-rater agreement of at least 80% on practice videos from families who withdrew from the project after T1. Following training and prior to the initiation of study coding, all videos were de-identified and assigned to coders randomly by site, group, and time point; that is, coders were unaware of the partner agency of origin, a family’s group assignment (PC or ECT), and whether a video set was from T1 or T2. To ensure ongoing reliability across both scenes and video sets, the coders met weekly to discuss scores and watch videos from different dyads together. All discrepancies were discussed and final scores were determined by consensus. The JERI authors recommended that weighted kappa be used to assess inter-rater agreement, with a benchmark of 80% agreement within 1 point (Suma et al., 2017). Based on weekly meetings and independent scoring of 21% of videos, weighted kappa was 0.92. Point-by-point reliability allowing for disagreements within 1 point was 99% and allowing for no disagreements was 87%.

Secondary Outcome Measures

Secondary (i.e., distal) measures examined child and parent outcomes that were indirectly related to the foci of PC. Administration and scoring procedures were identical to those described previously for primary measures.

Merrill–Palmer-Revised Scales of Development (MPR)

The MPR (Roid & Sampers, 2004) provides a comprehensive developmental evaluation using toy-based activities that are designed to engage young children between ages 1 month and 6.5 years. The Cognitive Battery of the test is comprised of cognitive, fine motor, and receptive language subscales. Scores across the three subscales are combined to produce a summary score, the Developmental Index (DI; M = 100, SD = 15). In a sample of 180 preschool-aged autistic children, Dempsey et al. (2020) found evidence of good concurrent and predictive validity of the MPR with other well-validated measures of cognition and receptive language.

Brief Family Distress Scale

The Brief (Weiss & Lunsky, 2011) consists of a single item that measures a family’s level of distress along a continuum, from 1 = “Everything is fine, my family and I are not in crisis at all” (1) to 10 = “We are currently in crisis, and it could not get any worse.” The measure was tested with a sample of 164 Canadian parents of autistic children, and significant moderate-sized correlations were found between total crisis severity and other measures of family coping, quality of life, and parent mental health (Weiss & Lunsky, 2011). It was also found to have high convergent and content validity when assessed against the Caregiver Strain Questionnaire, a longer measure with excellent internal reliability scores (Benninger & Witwer, 2017).

CarerQoL

The CarerQoL (Brouwer et al., 2006) assesses perceived caregiver quality of life (QoL) across seven key dimensions: fulfillment, support, mental health, physical health, financial problems, relational problems, and problems with daily activities. Each item is scored on a 3-point scale; a total score was calculated across all seven dimensions, following the coding protocol (Brouwer et al., 2006). The CarerQoL also assesses “happiness” using a visual analogue scale that ranges from 1 (completely unhappy) to 10 (completely happy). It has been found to have high test–retest reliability after 2 weeks and the results of validity tests with 224 families of autistic children suggest that it can be useful to assess quality of life and the impact of caregiving burden (Hoefman et al., 2014).

Parent Feedback Questionnaire

This 25-item questionnaire was developed for the project and was administered at T2 only. Questions 1–7 asked the parent to rate changes in their child over the course of the project in seven domains that paralleled the C-ESDM curriculum, on a scale of 1 (no longer a problem) to 7 (much worse). Questions 8–19 asked for feedback about the parent’s satisfaction/experience as a study participant, on a scale of 1 (very negative) to 7 (very positive). Questions 20–24 used a similar scale to ask about self-efficacy changes for the parent. There was also an invitation to “Please share with us anything else that you think is important regarding changes in your child or your experience in this study.”

Supplemental Measures

In addition to the primary and secondary measures, all parents completed a Family Demographic Survey at T1 that included basic identifying information for all family members. Parents also completed a monthly, online Activity Log for the duration of the study that asked them about the services and supports their child had received in the previous month (e.g., occupational therapy, preschool) and the approximate number of hours of each service for the month. Finally, information about children’s diagnostic status was collected from parents and the M-CHAT was re-administered to all children at study completion.

Parent Coach Fidelity

A description of the coach training program and related outcomes is beyond the scope of this paper and is described in detail elsewhere (Mirenda et al., 2021; Smith et al., 2021). Because this was a community-based collaborative project, all trainees who completed the coach training program were eligible to coach a family in the study. Trainee/coach fidelity was assessed by three coach trainers, all of whom were ESDM certified therapists and ESDM parent coaches, at four time points during the project—three during the formal training period and one at completion of the RCT.

In the first phase of training, trainees were taught to deliver a simplified version of C-ESDM (i.e., PACE Coaching; see Intervention Procedures, below) to young children on the autism spectrum. Trainee fidelity of implementation was assessed both prior to and at completion of this phase using a 24-item Therapist Practice Checklist (TPC) that assessed skills for working with a child (referred to as the Parent Progress Rating System, https://helpisinyourhands.org/provider/resourcecenter). The mean baseline TPC score for trainees who participated in the RCT was 73.1% (SD = 19.28) and the mean post-test score was 87.0% (SD = 13.84). In the second training phase, trainees were taught to coach parents to implement PACE Coaching strategies with their child. Trainers assessed a 60-min video-recorded coaching session at completion of this training phase via a 40-item Coaching Skills Checklist (CSC; adapted from the Coaching Fidelity of Implementation Brief Checklist; Rogers et al., 2021). Because of technology challenges (i.e., difficulty recording and/or uploading the videos), videos were available for only 42% of trainees at training completion; the mean fidelity score for these trainees was 75.0% (SD = 14.86). The CSC was also used to assess fidelity at completion of the PC intervention in the RCT; videos were available for 74% of coaches at this time point because of technology challenges and/or onset of the COVID-19 pandemic. The mean coach fidelity score across families was 80.1% (SD = 17.80).

Intervention Procedures

Parent Coaching Intervention

Parents in the PC group were assigned to a coach at the local CDC who completed the training associated with the project (Mirenda et al., 2021). PC parents were to receive 1 h of PC for 24 out of 30 weeks, plus one brief (i.e., 5–15 min) telephone follow-up session weekly or as needed. The six extra weeks were allotted to accommodate holiday pauses (e.g., Christmas, summer) and potential interruptions because of family or coach circumstances (e.g., illness). Parents could also receive any services that were available to them and/or their child through the CDC except for speech–language therapy, which was likely to overlap with the PC intervention. If a child in the PC group was diagnosed with autism while enrolled in the project, the caregiver was provided with a choice: (a) they could continue to participate in PACE Coaching and defer access to government funding for autism services without penalty until T2 assessments were completed or (b) they could withdraw from PACE Coaching and access autism funding immediately. All PC families whose children were diagnosed during the study chose option (a). All coaching was delivered either in the family home or at the CDC, depending on parental preference. Parents were reimbursed by the project for transportation costs if they traveled to the CDC and for child care costs (for dependent siblings) if coaching occurred in the home. During the intervention period, coaches met remotely either weekly or bi-weekly with the trainer with whom they worked during coach training to discuss each family’s progress and challenges.

In addition to weekly PC sessions, PC parents were also provided with (a) a book entitled An Early Start for Your Child with Autism: Using Everyday Activities to Help Kids Connect, Communicate, and Learn (Rogers et al., 2012a) and (b) access to Help is in Your Hands, a series of online modules (organized into 16 lessons, 4 per module) that were designed for parents of young children (Rogers & Stahmer, 2021; https://helpisinyourhands.org). Parents were encouraged by their coach to read specific book chapters and/or watch specific module lessons prior to each coaching session but they were not required to do so. Coaching was delivered in accordance with a collaborative model in which coaching is viewed as a “mutual conversation between two individuals who each have information to share and skills to gain” (Hanft et al., 2004, p. 31). Coaches used the framework developed by Rogers et al. (2021) to support parents as they learned a variety of strategies in the context of joint activity routines that could be embedded in ongoing daily activities in home (e.g., meal and snack times, bath time, dressing routines, etc.) and community settings (e.g., going to the playground, store, etc.). The strategies included those related to: (a) increasing the child’s attention and motivation; (b) using sensory social routines; (c) promoting dyadic engagement and turn-taking; (d) enhancing nonverbal communication; (e) building imitation skills; (f) facilitating joint attention; (g) promoting speech development; (h) identifying antecedent–behavior–consequence relationships (the “ABCs of learning”); (i) using prompting, shaping, and fading techniques to teach new skills; and (j) conducting simple functional assessment of behavior to develop new interventions.

Typically, parents engaged in two or more different activities during each coaching session (e.g., singing, reading a book, toy play, social play) to practice both new and cumulative skills. Guided by the coach, who provided feedback and suggestions, parents learned to self-assess each practice activity and reflect on how to make improvements, as needed. At the end of each coaching session, parents were provided with a written session summary, suggestions for practice activities, “Refrigerator Lists” that accompanied the Help is in Your Hands lessons and served as reminders of skills to practice between coaching sessions, and additional resources related to the session topic.

Enhanced Community Treatment Intervention

Parents in the ECT group received any services that were available to them and/or their child through the local CDC, including speech–language therapy, for 24 out of 30 weeks; as with the PC group, extra weeks were allotted to accommodate holiday time, illness, etc. They were also provided with the same Early Start for Your Child with Autism book and Help is in Your Hands modules as the PC group. If a child in the ECT group was diagnosed with autism while enrolled in the project, the caregiver could also access autism funding and arrange for additional supports. ECT families participated in the same T1 and T2 assessments as PC families and were also asked to complete monthly Activity Logs, as described previously.

Data Analytic Plan

Descriptive statistics of study participant characteristics at T1 included frequency (%) and means and standard deviations (SD) for categorical and continuous variables, respectively. Key demographic variables and assessment scores for children and parents at T1 were compared between groups, using independent samples t-tests and, for categorical variables, Fisher’s exact tests. Independent samples t-tests were used to compare the mean hours of coaching received by PC and ECT families, by type of service. For primary and secondary outcomes that were assessed at T1 and T2, Shapiro–Wilks tests were used to assess normality prior to all analyses. In the few cases where results were significant (p < 0.05), Mann–Whitney U tests were used to examine change between T1 and T2 scores and Wilcoxon signed-ranks test were used to calculate within-group effect sizes (i.e., Z scores). When normality was established, we used a series of repeated measures analyses of variance (ANOVAs) to examine the main effect of Time (T1 vs. T2), Group (PC vs. ECT), and the Time × Group interaction for each primary and secondary continuous outcome measure. Independent samples t-tests were used to compare results of the cumulative Activity Logs and the Parent Feedback Questionnaire (PFQ) across the two groups. To examine predictors, we first constructed a correlation matrix to examine study variables that we hypothesized would be related to child change. We then entered the correlated variables into a linear regression.

Results

Participant Flow

Figure 1 depicts participant flow, including both screening and the RCT. As displayed in Fig. 1, 445 children were assessed for eligibility across all sites; of these, 285 (64%) were eligible for screening and 212 caregivers (47.6%) were invited. Of those invited, 146 (68.9%) consented, 142 completed the M-CHAT-R/F, and 92 children from these families (64.8%) had scores indicative of possible autism (i.e., scores ≥ 3 on the M-CHAT-R/F or ≥ 2 after the Follow-up Interview). Of the 92 parents (i.e., primary caregivers), 77 (83.7%) were invited to participate in the RCT. An additional 10 families (10.9%) participated in a related but separate study that was designed specifically for Indigenous families and is not reported here. Five families were not invited because the child was older than 36 months when the RCT commenced. Of the 77 parents invited to the RCT, 64 (83.1%) consented and 62 (80.5%) completed T1 assessments and were allocated to either the PC or ECT group. The two families that were lost to follow-up at this point did not respond to multiple invitations to complete the T1 assessment.

Fig. 1

Participant flow from assessment for eligibility to study completion

Initial Equivalency of Groups

Key demographic variables and assessment scores for children and parents at T1 were compared between groups. All results were non-significant (p > 0.05), indicating that the randomization procedure resulted in groups that were not statistically different.

T1–T2 Assessment Interval

All but six T2 assessments in the PC group and all but two T2 assessments in the ECT group were administered within 4 weeks of intervention completion. For the exceptions, this interval was longer because of holiday time (e.g., Christmas, summer vacation), illness, or other factors that affected parent or child availability. The mean T1–T2 interval for PC families was 33.0 weeks and the mean for ECT families was 27.4 weeks, a significant difference according to an independent samples t-test (p < 0.01).

Activity Logs

The Activity Log response rate for PC parents was 79.2%, with only three families submitting Logs for fewer than 3 months and 70.8% submitting them every month during the coaching period. The response rate for ECT parents was 80.7%, with no families submitting fewer than three Logs and 68% submitting them every month. On average, PC parents reported reading 6.8 chapters and ECT parents reported reading 5.4 chapters of the Early Start… book, with no significant difference between groups. In addition, parents in both groups viewed an average of 4–5 Help is in Your Hands lessons each, again with no significant between-group difference.

Because not all parents submitted Logs monthly, we calculated the average number of one-to-one and group hours of service received per month for each family, based on the number of Log submissions. As noted previously (see Measures), the coaching period for PC families also varied; thus, we calculated the average hours per month of coaching plus follow-up hours, based on the total number of months. Although parents in the PC group reported significantly fewer mean hours of one-to-one services per month compared to parents in the ECT group (p = 0.03), there was no significant difference between the two groups for total hours of service received (Table 2).

Table 2

Mean hours per month of service reported by families, by group and service type

Service type	PC group		ECT group		t	p
	M h/month (range)	SD	M h/month (range)	SD
One-to-one servicesa	1.43 (0–6.25)	1.58	2.84 (0.33–11.30)	2.46	2.92	.03*
Group servicesb	2.08 (0–10.36)	2.57	3.63 (2.66–15.0)	3.38	1.69	.10
Parent coaching + follow-up	4.13 (2.00–9.87)	1.70	n/a	n/a	n/a	n/a
Total services	7.35 (2.66–15.00)	3.38	6.47 (0.50–20.80)	5.13	.71	.48

*Significant

aIncludes, for example, occupational/physio therapy, infant development consultation, and (for the ECT group only) speech–language therapy

bIncludes, for example, Parent–Child Mother Goose, group parent workshops/training, and preschool

Effect of Group Assignment on Parent/Caregiver Outcomes

This section presents the results related to the first hypothesis in this study—namely that, relative to the ECT group, PC parents would demonstrate more skilled use of scaffolding and following-in on the JERI, have higher quality of life on the CarerQoL, have lower parent distress on the Brief, and have higher parent satisfaction and self-efficacy scores on the Parent Satisfaction Questionnaire.

JERI

Repeated measures ANOVAs for the individual and combined JERI code scores for scaffolding and following-in indicated significant main effects for Time for both groups (p < 0.001), with medium to large effect sizes. Table 3 summarizes the means, standard deviations, and effect sizes for Time in both groups; effect size trends in the PC group were consistently larger than in the ECT group.

Table 3

Means/medians and standard deviations/inter-quartile ranges for Time 1 and Time 2 child and parent scores, with effect sizes, by group

Measure	PC group		PC group		PC group	ECT group		ECT group		ECT group
	Time 1		Time 2			Time 1		Time 2
	Mean	SD	Mean	SD	Cohen’s d	Mean	SD	Mean	SD	Cohen’s d
JERI JE	16.83	4.08	21.38	5.63	.92	16.33	4.32	20.29	5.45	.80
JERI CJE	6.29	2.44	8.50	3.67	.68	6.54	2.77	8.54	3.82	.62
JERI SIJE	7.21	4.86	12.08	6.16	.94	6.54	3.06	11.50	6.04	.96
JERI ExLang	7.54	4.40	11.92	6.00	.86	7.33	4.00	10.79	5.70	.68
JERI Scaf	15.67	2.84	17.92	3.09	.71	15.88	3.33	17.83	3.36	.62
JERI Follin	16.38	3.37	20.17	3.00	1.24	16.01	2.72	19.46	3.12	1.09
JERI FC	15.29	3.24	17.83	3.76	.77	15.17	2.37	17.21	3.62	.62
MPR DI	55.23	23.37	60.00	27.59	.20	52.25	19.56	60.58	22.81	.36
Brief Family Distress	3.50	1.41	3.30	0.98	.12	3.28	1.77	3.48	1.64	.14
CarerQoL total	9.00	2.52	9.61	2.25	.24	9.88	2.60	9.36	2.77	.20
CarerQoL happiness	7.00	1.68	7.35	1.19	.24	7.00	1.56	7.16	1.38	.11
PFQ satisfactiona	n/a	n/a	75.32	4.62	n/a	n/a	n/a	64.54	9.59	n/a
PFQ self-efficacya	n/a	n/a	24.73	2.31	n/a	n/a	n/a	21.78	4.20	n/a
	Median	IQR	Median	IQR	Z score	Median	IQR	Median	IQR	Z score
CDI words producedb	16	5, 95	145	34, 245	3.86	19	5, 37	92	18, 202	3.92
CDI words understoodb	137	77, 235	280	212, 388	4.20	145	82, 232	220	157, 342	4.17

PC parent coaching, ECT community treatment, SD standard deviation, JERI Joint Engagement Rating Inventory, JE joint engagement, CJE coordinated joint engagement, SIJE symbol-infused joint engagement, ExLang expressive language, Scaf scaffolding, Follin following-in, FC fluency and connectedness, MPR DI Merrill–Palmer-Revised Developmental Index, PFQ Parent Feedback Questionnaire, n/a not applicable, CDI Communicative Development Inventory, IQR inter-quartile range (25th and 75th percentiles)

aCompleted at T2 only

bNon-parametric tests were used for analysis, so medians, inter-quartile ranges, and Z scores are reported instead of means, SDs, and Cohen’s d

CarerQoL and Brief

Repeated measures ANOVAs for CarerQoL happiness scores were not significant for either Time or Group, with no interaction (p > 0.05). However, there was a significant Time × Group interaction for the CarerQoL total score in favor of the PC group, F(1,46) = 4.95, p = 0.031, with a small effect size. Repeated measures ANOVAs for the Brief were also not significant (p > 0.05). Table 3 summarizes these results.

Parent Satisfaction

We conducted a series of independent samples t-tests to examine three types of scores on the PFQ, which was completed by parents in both groups at T2 only. The first set of seven items asked parents to rate their child’s behavior change in key domains corresponding to the curriculum (e.g., communicating with you with gestures, sounds, words, etc.). On average, parents in both groups scored all items as “significantly improved” and no significant between-group difference was found for any domain (p > 0.05). The second set of 12 items asked parents to rate their satisfaction with various aspects of the study. There was a significant difference in favor of the PC group, F(1,44) = 22.89, p = 0.0001, with a large effect size. The third set of four items asked parents to self-assess their changes in attitude, knowledge, skills, and ability to access resources. Again, we found a significant difference in favor of the PC group, F(1,43) = 8.38, p = 0.006, with a large effect size.

In addition to items that were rated, the PFQ asked parents to “share anything else that you think is important regarding changes in your child or your experience in this study.” Comments were generally reflective of satisfaction, with more positive comments from PC parents overall. Representative examples from PC parents include:

I feel like he’s a different kid. Part of it is age/maturing/going to preschool, but a large part is the communication development due to PACE (i.e., I can hold his hand when we’re out, not hang on to his wrist for dear life).

Our child is a completely different person! Our child communicates, asks for help, plays with others, does everything we hoped he would do and were so worried that he would never do.

The changes my child has experienced through PACE are mind blowing. He has grown in every single aspect that we worked on and more. Plus, I now have the tools and confidence to help him and his younger brother. We owe a huge debt of gratitude to PACE and our coach.

Having weekly visits, phone calls and handouts helped keep us accountable and connected on our current goals… My child has grown exponentially throughout this study and I’m so sad it’s over.

A few ECT parents commented on their group assignment; for example, one noted, “Super bummed that I didn’t get into the Parent Coaching group.” However, most ECT parents commented that the supports they received from the local CDC because of their involvement in the project were important for them and for their child:

Being a part of his development, 100% getting to his level, really helped with us in communicating with him.

I learned that my child responds…to songs very well. He gets excited, starts clapping and smiles when I sing to him, and gives me a good eye contact.

Nothing but positive changes. Nothing drastic but still improvement.

…her attending [the preschool program] really helped with her speech…

Others noted that they initiated various strategies on their own after learning that their child might be autistic; as one parent noted, “He has gone from almost no or very little communicative abilities to communicating all the time. I do not believe this is because of the study but instead our use of sign language and phone apps that allow him to communicate in other ways…”.

Effect of Group Assignment on Child Outcomes

This section presents results related to the second hypothesis—namely that, relative to those in the ECT group, children in the PC group would demonstrate larger increases in the number of words understood and words produced on the Communicative Development Inventories (CDI), higher scores during parent–child interactions as measured by JERI codes, and larger gains in DI scores on the Merrill–Palmer-Revised Scales. Table 3 summarizes the means, standard deviations, and Time effect sizes for both groups when results were significant.

Communicative Development Inventories

At T1, none of the children spoke > 200 words or had phrase speech, so all parents completed the CDI: WG form only. At T2, 8 parents (5 in the PC group and 3 in the ECT group) completed both the WG and the WS because their children met these criteria. For these children, at T2, we combined the number of WG words understood and produced with the number of unique words produced from the WS, for a total words produced score. We also calculated a total words understood score by combining WG words understood, WG words understood and produced, and WS words produced. Mann–Whitney U tests were used to compared group scores for CDI words produced and words understood because tests of normality were significant. As is the convention for non-normally distributed data, results are reported in terms of medians rather than means, inter-quartile ranges (IQRs) rather than standard deviations, and Z-scores rather than Cohen’s d effect sizes (Habibzadeh, 2017). Table 3 provides a detailed summary of the results.

For words produced, there was no significant difference between the two groups (p = 0.41). For the PC group, the median number of words produced was 16 at T1 and 145 at T2, a median gain of 129 words. For the ECT group, the median number of words produced was 19 at T1 and 92 at T2, a median gain of 83 words. For words understood, there was a significant difference in favor of the PC group (p = 0.043) with a medium effect size (Z = 2.02). For the PC group, the median number of words understood was 137 at T1 and 280 at T2, a median gain of 143 words. For the ECT group, the median number of words understood was 145 at T1 and 220 at T2, a median gain of 75 words. Because the CDI is a parent report measure, we also conducted a Spearman rank order correlation to examine the association between changes in CDI words understood (i.e., T2 − T1 differences) and changes in MPR receptive language raw scores (a measure of language comprehension). The result was significant, r = 0.629, p = 0.002. This provides supportive evidence for the accuracy of parental responses on the CDI.

There were five JERI variables that assessed children’s skills during play interactions: (a) joint engagement, JE; (b) coordinated joint engagement, CJE; (c) symbol-infused joint engagement, SIJE; (d) expressive language, ExLang; and (e) fluency and connectedness, FC, which also reflected parent skills, as noted previously. Across all five measures, repeated measures ANOVAs indicated significant main effects for Time only for both groups (p < 0.001) with medium to large effect sizes. Effect size trends in the PC group were consistently larger than in the ECT group for all but one measure (SIJE).

Merrill–Palmer-Revised Scales

A repeated measures ANOVA for MPR DI standard scores was not significant for either Time or Group and there was no significant interaction (p > 0.05).

Predictors of Child Change

This section provides results for the third and final hypothesis—namely that change in one or more child variables during parent–child interactions would be predicted by parent change in scaffolding and following-in (Scaf/Follin) that was reflective of parent/caregiver interaction skills. JERI codes were used to examine this hypothesis.

As noted previously (see Measures), four JERI codes (JE, CJE, SIJE, and ExLang) were reflective of children’s ability during parent–child interactions and a fifth code (FC) was reflective of the overall parent–child interaction. Because there were no between group differences, we combined the PC + ECT groups and constructed a correlation matrix to evaluate associations between changes in the five child variables from T1 − T2 (i.e., difference scores) and change in the combined Scaf/Follin variable from T1 − T2. We also examined additional study variables that we hypothesized might be related to child change, including (a) child MPR DI standard score at T1; (b) child CDI words produced at T1; (c) and (d) the mean hours per month of support received by parents, as reported on the monthly Activity Logs (see Table 2). In addition, for the PC group, we examined relations between both parent and child JERI variables and final coach fidelity scores.

Correlation results indicated that parent Scaf/Follin was significantly associated with all five JERI variables (r = 0.43–0.90). MPR DI at T1 was correlated with all variables (r = 0.37–0.50) except for CJE, and mean hours per month of service was associated with both JE (r = 0.33) and FC (r = 0.31). CDI words produced at T1 was not correlated with any of the JERI child variables, nor was the final coach fidelity score for parents in the PC group.

Based on these results, we then conducted a series of linear regressions toward the goal of identifying a combination of variables that predicted changes in JERI scores over time. We did this for all JERI variables that were significantly correlated except CJE, which was correlated with only one variable of interest. Visual inspection of the residuals for each model confirmed that all assumptions were met. Results indicated that parent change on the combined Scaf/Follin difference score from T1 − T2 accounted for 76.3% of the variance in child change for JE, 25.8% of the variance for SIJE, and 65.6% of the variance for FC. In addition, MPR DI at T1 predicted one child variable on the JERI, ExLang, accounting for 16.7% of the variance. No significant associations were found for any JERI difference score and the mean hours of service/month. Table 4 summarizes the results, which confirm the hypothesis related to a strong association between parent change and child change.

Table 4

Regression results for study variables hypothesized to predict child change in JERI scores

Predictor	JERI score variable
	Joint engagement (JE)				Symbol-infused joint engagement (SIJE)				Expressive language use (ExLang)				Fluency and connectedness (FC)
	β	SE	t	p	β	SE	t	p	β	SE	t	p	β	SE	t	p
Scaf/Follin difference	.670	.064	10.55	< .001*	.366	.102	3.59	.001*	.116	.083	1.39	.172	.410	.051	8.07	.001*
Mean hours service/month	.097	.091	1.07	.291	− .022	.145	− .152	.880	.059	.119	.499	.620	.049	.072	.680	.500
MPR DI T1	.025	.019	1.30	.201	.033	.031	1.06	.294	.054	.026	2.13	.039*	.021	.016	1.32	.193

*Significant

Discussion

The PACE Coaching project was a randomized feasibility trial of a modified version of P-ESDM in multiple community sites across a large geographic area in Canada. It builds on previous preemptive intervention trials designed for children with suspected autism and previous feasibility studies that examined various ESDM-based interventions in community settings. First, we hypothesized that compared to those in the ECT group, children in the PC group would demonstrate greater gains in cognitive ability (MPR DI scores), both words understood and words produced (CDIs), and parent–child interaction scores (JERI). This hypothesis was confirmed for words understood only. Second, we hypothesized that parents in the PC group would demonstrate more skilled use of JERI scaffolding and following-in, compared to parents in the ECT group; however, parents in both groups showed significant change in these two variables, with no group differences. Third, we expected that parents in the PC group would have higher quality of life (CarerQoL), lower parent distress (Brief), and higher satisfaction and self-efficacy scores (PSQ) compared to those in the ECT group. This hypothesis was confirmed for all measures except parent distress. Finally, we expected that parent change in JERI scaffolding and following-in would predict change in one or more child variables during parent–child interactions. We found this to be the case for three of the five JERI variables: joint engagement, symbol-infused joint engagement, and fluency and connectedness.

This study was unique in several ways. First, it was conducted in collaboration with 16 community agencies, not in a university-affiliated clinical or research setting. This required the research team to establish and maintain productive working relationships with staff in the partner agencies for a 3-year period, even as we asked them to make significant operational changes to accommodate the needs of the project (see Smith et al., 2021). Second, the study targeted parents of very young children (aged 15–36 months) who were suspected of having autism, not children who were already diagnosed. This meant that we had to train staff in the community agencies (via online modules produced by the project) to identify, invite, consent, and recruit parents for autism screening and to administer the M-CHAT-R/F, a measure that few of them had used before. Third, in the first year of the project, the project trained parent coaches who were primarily community college graduates, not graduate-level ESDM therapists and parent trainers, and then supported them to provide coaching to families in the PC group (Mirenda et al., 2021). Fourth, we relied on professionals in community agencies—not experienced assessors in a research setting—to invite, consent, and conduct assessments at two time points with parents and children. This meant that we had to produce a series of online modules that were sufficiently informative for teaching them to conduct the assessments uniformly across sites, and they had to devote considerable time viewing the modules and practicing test administration procedures. Fifth, having trained community professionals—not full-time, dedicated research staff—in the skills required for the study, we then relied on them to conduct assessments, deliver coaching to families with remote support from three training facilitators, and provide considerable data to the research team that was above and beyond their usual experience. Finally, we accepted into the study any parent who was able to speak and write basic English (with the aid of a family translator, if necessary) and had at least one address and a telephone number. In the end, this meant that attrition was around 36% and that parents in the RCT were (compared to BC population statistics) less educated, had lower annual household incomes, and were less likely to live in urban population centres where supports were more widely available. However, it is important to note that many parents with high levels of family burden (e.g., unemployment, mental health challenges, caregiver disability) were able to complete the project, despite the challenges they experienced.

The logic of any parent-mediated intervention is that child change is dependent on parents’ ability to deliver the intervention, and parent delivery is dependent on the skills of providers who coach the parents. In this study, we were unable to establish a consistent chain of effect from coach fidelity to parent change to child change. Coaches were trained to deliver a modified version of ESDM and achieved varying levels of fidelity, and fidelity scores were not correlated with either parent or child change on the JERI. Change in parents’ ability to support their child by scaffolding and following-in also varied widely; some parents in both groups were quite skilled in both domains from the outset and thus showed little change, others started from very low levels of ability and showed considerable improvement, and still others were stable across both assessment time points. However, importantly, overall parent change on JERI scores for scaffolding and following-in was related to child change in joint engagement and improvements in parent–child fluency and connectedness. With regard to child change, there was no significant difference between children in the PC and ECT groups for either cognitive ability or words produced from T1–T2. However, compared to ECT children, PC children showed greater gains in words understood (i.e., vocabulary comprehension) on the CDI, a result that was strongly correlated with changes in receptive language raw scores on the MPR. This is important because early receptive vocabulary has been found to be one of the best predictors of later receptive (Wetherby et al., 2007) and expressive vocabulary in young autistic children (Yoder et al., 2015). Thus, children who understand more words when they are very young are more likely to continue to do so and to develop speech (i.e., expressive) communication as they get older. Speech communication by age 5, in turn, is the best predictor of later outcomes in terms of adaptive skills, school achievement, and independence in adulthood (Lord et al., 2008).

Parents’ quality of life, experience/satisfaction scores, and self-efficacy scores were also significantly higher for the PC group. Parent satisfaction in particular is often under-valued as an outcome measure in RCTs under the assumption that parents in the intervention group will always provide higher satisfaction ratings than those in the control group. This is often the case when the control group receives no additional support at all (Howard, 2019) but it is not necessarily so when the control group receives some type of benefit. In this study, parents in the ECT group had the benefits of early autism screening, support to initiate a referral for additional diagnostic assessment, access to enhanced services at the CDC, and the Early Start book and Help is in Your Hands modules. Thus, higher parent satisfaction and self-efficacy scores coupled with a modest improvement in quality of life in the PC group should not be discounted.

Unexpectedly, we found significant changes over time in both child and parent JERI scores for both the PC and ECT groups. There are a few plausible explanations for this finding, all of which are related to the enhanced services that were available to ECT families secondary to study enrolment. First, they were provided with the Early Start… book (Rogers et al., 2012a) and access to Help is in Your Hands modules at study enrolment. These materials provided ECT parents with information beyond the “treatment as usual” that is the standard for a comparison group in an RCT. Second, monthly Activity Log data (Table 2) showed that, compared to PC families, those in the ECT group received significantly more hours of one-to-one services other than PC from the CDC during the study period. One-to-one services consisted of support from an infant development consultant and/or individual speech–language, occupational, and/or physio therapy. It is likely that some of these services were aimed at helping ECT parents to use techniques for interacting with their child and/or supporting social communication development that were similar to those used in PC. In addition, in some CDCs, ECT children had access to group services (e.g., preschool, parent–child Mother Goose) that were unavailable to them prior to study enrollment and may have indirectly resulted in enhanced parent–child interactions. Finally, we specifically asked EDs not to assign ECT families to the caseloads of coaches involved in the project and we asked coaches to avoid applying PC to these families if such an assignment was unavoidable. However, despite these precautions and given the high caseload pressures that existed in all of the CDCs, parent coaches were sometimes assigned to ECT families and might have inadvertently provided some form of PC to them. In combination, the enhanced service side effects of autism screening and subsequent study enrolment are similar to that found in some previous ESDM studies (e.g., Rogers et al., 2020, 2012b). As evidenced in the quotes from ECT parents, it is heartening to note the potentially positive impact of early autism screening and study enrolment alone on children’s development, a phenomenon that Micheletti et al. (2020) referred to as the “surveillance effect” (p. 13).

Limitations

There are a number of limitations to this study that are important to acknowledge. First, in a meta-analysis of M-CHAT accuracy with toddlers, Yuen et al. (2018) found a “lack of evidence on its performance with low-risk children or at age 18 months” (p. 1093). This review evaluated the original M-CHAT, not the M-CHAT-R/F that was used in this study; nonetheless, our use of this measure as a screening tool might have limited the children who were eligible to participate. Second, Indigenous families are under-represented in the present study because we offered them the option of (a) participating in the standard RCT (in which they had a 50/50 chance of receiving PC) or (b) participating in a subproject that was designed to accommodate Indigenous cultural values and did not involve randomization (in which all families received PC). All Indigenous families chose option (b). Although technically a limitation, we believe that offering a choice between the RCT and the subproject was more respectful of Indigenous values than automatic random assignment would have been. Third, because of attrition, the final sample size did not allow us to accurately or reliably detect group differences with effect sizes less than d = 0.36. Related to this, we applied a significance level of p = 0.05 to all comparisons, rather than correcting for multiple tests. Fourth, because of the scope of the study, we were unable to measure intervention fidelity for PC parents; thus, we could not directly evaluate relations between what parents were coached to do, what they actually did, and how this affected child change. Fifth, we did not ask parents to track either the activities in which they practiced the strategies used during coaching sessions or the time spent on these activities. A few previous ESDM studies have included such data (e.g., Dawson et al., 2010) but most others have not, presumably because of the burden that this places on parents. Sixth, we relied on graduate-level clinicians in the CDCs to administer all assessment measures following successful completion of online modules that provided detailed information about test protocols (especially for the CPP videos and the MPR). However, it was impossible to assess fidelity of test administration directly by viewing the assessors as they conducted the tests, and they were aware of families’ group assignment at T2. We partially compensated for these concerns by having trained research assistants check and score all assessment forms and videos for protocol compliance (e.g., for adherence to MPR basal and ceiling rules) and, when rare violations were found, they consulted with the assessors to correct them. Nonetheless, administrator bias might have occurred at T2 and test administration errors that affected final scores might have gone undetected. Seventh, the mean T1–T2 assessment interval for the PC group was significantly longer than for the ECT group, which might have affected the outcomes. Eighth, to accommodate parent preferences, coaching was provided in either the family home or at the CDC, but we were not able to examine the extent to which the setting might have affected the outcome. Finally, the onset of the COVID-19 pandemic interfered with our ability to collect coach fidelity data from some coaches and parent and/or child data from a few families.

Moving Forward

In this report, we described one component of a multi-element project that was conducted to examine the feasibility and outcomes of a modified version of P-ESDM in community agencies across a large Canadian province. The results add to the growing body of work in the field of implementation science that confirms the well-known axiom, “Implementation is a process, not an event” (Fixsen et al., 2005, p. 15). In that vein, our results support the feasibility of offering preemptive community-based interventions to parents of very young children with suspected autism, especially in situations where the availability of early autism intervention is contingent on diagnosis and diagnostic waitlists are long. Our results and those of others (e.g., Rogers et al., 2020) also suggest that more work is required toward the goal of disseminating high-quality PC in community settings. More work is also needed to refine strategies for PC that are flexible and adaptable for families across the socio-economic, ethnic, cultural, and language spectrum. Finally, our results suggest that broad-based child change, the ultimate goal, is not likely to be achieved without first addressing these challenges.

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 14 kb)