Feasibility and Initial Efficacy of an Adapted Telepractice Listening Comprehension Intervention for School-Aged Children with Autism.

This study evaluates the feasibility and initial efficacy of an 11-week listening comprehension intervention, Building Vocabulary and Early Reading Strategies (BVERS) that was delivered remotely to 14 elementary-aged children with autism spectrum disorder. Children were randomly assigned to one of two groups: BVERS only, or BVERS with a parent instructional component (BVERS + PC). Results indicate that the intervention was feasible to implement. All parents were satisfied with intervention implementation, and 8 of 10 stated that they were satisfied with their child's outcomes following the intervention. Results of a Wilcoxon signed-rank test showed growth in listening comprehension following the intervention, but no growth in narrative retell or vocabulary. There were no group differences in change scores from pre- to post-test.

Introduction

Telepractice, which is the remote delivery of assessment or intervention services over an electronic platform, has been growing in popularity over the past several years (Hodge et al., 2019). The COVID-19 pandemic, beginning in the winter months of 2020, dramatically increased the need for telepractice services to a wide range of individuals (Krach et al., 2020). Among those are children with autism spectrum disorder (autism) and their families, who are often more vulnerable to disruptions in daily life and typical school instruction than families without a child with autism (Osborne & Reed, 2009; White et al., 2021b). In their national survey of families of children with autism conducted at the start of the COVID-19 pandemic (n = 3502), White et al. (2021b) found that caregivers indicated high rates of special education (80%) and speech therapy (88%) service disruption due to the onset of the pandemic. A follow-up survey two weeks later found that many families had already transitioned to online or remote-delivered special education (47%) or speech and language therapy (50%) services for their children with autism (White et al., 2021a).

Given the dramatic increase in remote-delivered instruction and special education services, further guidance on these services for children with autism is needed. Approximately one decade prior to the COVID-19 pandemic, Boisvert et al. (2010) conducted a systematic review focused on telepractice with students with autism. The eight studies that met criteria for the review included service delivery (e.g., diagnostic assessment, functional analysis, and behavioral intervention) and development and implementation of an individualized education plan. The majority of these studies included fewer than five participants and all studies concentrated on social and behavioral outcomes. An updated systematic review by Ellison et al. (2021) found 47 studies examining interventions delivered via telepractice with children with autism. Again, all but one of these interventions focused on non-academic skills such as behavior management, functional communication, or social skills. Additionally, only 4 studies targeted children with autism directly, as most were focused on parent-implemented interventions providing training and coaching online. The authors conclude that telepractice services are equivalent to and may even be better than in-person services for children with autism and should be explored further.

Beach et al. (2021) recently adapted a summer reading program for virtual instruction for typically developing elementary students struggling with reading development. The program of use, Sound Partners (Vadasy et al., 2005) was delivered by trained tutors to rising second and third grade students who were reading below grade level. The researchers found that the students showed some improvement on reading fluency and accuracy. While this study was not limited to participants with autism, the results suggest that virtual learning can provide the opportunity for individualized instruction in reading and can be delivered effectively.

Many children with autism require extra support to successfully develop reading skills, particularly reading comprehension. The extant literature on reading development in children with autism suggests that this group of students struggles with reading comprehension at higher rates than their typically developing peers (Estes et al., 2011; Huemer & Mann, 2010; Jones et al., 2009; McIntyre et al., 2017; Nation et al., 2006) and that these difficulties become more pronounced as students enter secondary school (Solari et al., 2019). The COVID-19 pandemic exposed a need for remote-delivered interventions that support language and reading comprehension development in children with autism.

One way to intervene in reading comprehension is through listening comprehension, which is a precursor for reading comprehension (Kendeou et al., 2005; Nation & Snowling, 2004). The development of listening comprehension is dependent on both structural language skills and higher-order skills such as inferencing or metacognition and includes verbal ability and vocabulary (Kirby & Savage, 2008). When reading or listening to a story, a child must have enough language awareness to know how to make predictions about an author’s intentions or what the characters in a story are thinking or feeling. To make these types of inferences, children create mental schemas using their prior knowledge and the available information from the text to make an inference about what is being communicated (Kintsch, 1988; Kirby & Savage, 2008).

Previous studies of in-person listening comprehension instruction for school-aged children with autism have focused on shared book reading interventions. A shared reading approach involves a teacher or caregiver reading a storybook aloud while promoting child engagement through interactive questions before, during, and after reading (Hudson & Test, 2011). For example, a teacher might ask students to make predictions about a story based on the title or cover, or emphasize certain parts of the text during reading to promote comprehension. Shared reading interventions have been associated with improvements in oral language and comprehension in children with typical development (National Early Literacy Panel [NELP], 2008; Walsh & Blewitt, 2006; Zevenbergen et al., 2003) and school-aged children with autism (Boyle et al., 2019; Henry & Solari, 2020; Kim et al., 2018; Solari et al., 2020; Whalon & Hart, 2011). In a single-case design study of 3 elementary-aged students with autism, Kim et al. (2018) examined the effects of a shared reading intervention on narrative story comprehension and task engagement. The researchers found that all students improved their story comprehension during the intervention phase and maintained improvement at follow-up.

Solari et al. (2020) explored preliminary effects of a vocabulary and listening comprehension intervention (Building Vocabulary and Early Reading Strategies; Eager BVERS) on the narrative retell, vocabulary, and listening comprehension skills of 12 students with autism. The intervention utilized children’s books to target the development of language-related skills and social cognition, as overlap between cognitive and language demands required for reading comprehension may be particularly difficult for students with autism who struggle with social-communication. After 17 weeks, the six students who received the intervention outperformed the six who did not on a measure of narrative competence. A follow-up study (Henry & Solari, 2020) using the same intervention with more participants with autism (n = 43) found that students in the intervention group outperformed students in the control group on standardized measures of listening comprehension, expressive vocabulary, and narrative retell. This intervention approach was implemented in-person across both studies, and the feasibility of its application via telepractice has not yet been examined.

There are several important considerations for moving these types of reading interventions to a remote format. While children with autism may have an affinity for media viewed via a technology platform such as Zoom (Shane & Albert, 2008), there is more opportunity for distraction and disengagement when instruction is taking place directly inside a child’s home (Garbe et al., 2020). Conversely, parent involvement between sessions can potentially improve in-session engagement and overall learning outcomes (Karver et al., 2006; Kazantzis et al., 2010). Caregivers can successfully learn how to implement these types of parent-mediated intervention components through online training. For example, Pierson et al. (2021) found that parents of children with autism were able to implement a dialogic reading intervention with high fidelity when provided with coaching and training via telepractice.

Current Study

The aim of this study was to investigate the feasibility and acceptability as well as consider the initial efficacy of a telepractice oral language and listening comprehension intervention, BVERS, which has been established to have efficacy for in-person delivery with students with autism (Henry & Solari, 2020; Solari et al., 2020). To our knowledge, there has never been a telepractice study for children with autism that tests the feasibility of an intervention to support reading-related skill development. Therefore, this study addresses a timely and pertinent issue regarding educational support needs for children with autism. A secondary aim of this research was to determine the feasibility of a supplemental, parent-mediated component of the intervention, and whether this component had an impact on participant outcomes. These aims were guided by the following research questions (RQs):

RQ 1: Is the BVERS intervention, when delivered remotely through a videoconferencing service, feasible and acceptable for use with school-aged children with autism?

RQ 2: Does the remote-delivered BVERS intervention show initial efficacy for children with autism, as measured by standardized oral language and listening comprehension assessments?

RQ 3: Compared with the BVERS-only treatment condition, does the addition of a supplemental parent component (BVERS + PC) impact the feasibility and acceptability of the intervention?

Methods

Participants and Setting

The regional Institutional Review Board approved recruitment and data collection procedures. Written informed parent consent and child assent were obtained from participants’ guardians prior to testing. Fourteen school-aged children with autism participated in the virtual listening comprehension intervention. All children in this study received special education services under the classification of “Autism” or had a diagnosis of Autism Spectrum Disorder from a licensed psychologist or psychiatrist; confirmation of diagnosis was provided by the parent. Students in first through third grades were eligible to participate in the study. Exclusionary criteria included children who were nonverbal or used an adaptive communication device and children who had a Vineland Adaptive Behavior Scales, Third Edition (Vineland-3; Sparrow et al., 2016) Adaptive Behavior Composite (ABC) score lower than 71 or a Wechsler Abbreviated Scale of Intelligence-Second Edition (WASI-II; Wechsler, 2011) Verbal Comprehension Index (VCI) lower than 71.

All assessments and intervention sessions were conducted remotely through Zoom, a videoconferencing service. Prior to intervention, internet speeds for participants and instructors were tested to ensure adequate connectivity. Personal computers and tablets served as the principal technology used for remote service delivery. Participants completed assessments and intervention sessions in their homes in the location of their choosing (e.g., bedroom, kitchen, office, living room, etc.). All assessments were administered by postdoctoral fellows who had extensive experience conducting in-person assessments and who prepared assessment materials using emerging best practices for online assessment delivery. Assessors were not blind to treatment condition. The time between pretest and posttest assessments was over 17 weeks due to school breaks and a two-week delay between conducting pretest assessments and starting intervention sessions.

Intervention Preparation

Group Assignment

Before the intervention began, 17 participants were assessed to determine eligibility for participation in the study; three participants were not included due to a pretest Vineland-3 ABC score below 70 or because they could not reliably respond to the assessment stimuli. Using matched pairs randomized block design, the remaining participants (n = 14) were assigned to seven matched pairs based on two factors: a) verbal comprehension ability (as measured via the WASI-II) and b) structural language skills (as measured by the Word Structure subtest of the Clinical Evaluation of Language Fundamentals, Fifth Edition [CELF-5, Wiig et al., 2013]). Participants in each dyad were assigned to one of two treatments, yielding seven participants in each condition. There were no significant differences between groups on pretest measures of verbal reasoning, adaptive behavior, oral language or listening comprehension. Demographic and descriptive data for these groups are presented in Table 1.

Table 1

Age and descriptive measures

Measure	BVERS (N = 7)		BVERS + PC (N = 7)
	Mean (SD)	Range	Mean (SD)	Range
Age (years:months)	7:8	6:8–8:8	8:3	7:2–9:8
	(0:8)		(1:0)
WASI-II VCI	95.71	78–116	86.14	73–103
	(16.29)		(11.92)
Vineland-3 ABC	79.75	73–91	77.83	71–92
	(8.30)		(7.83)
CELF-5 Word Structure	8.25	3–15	8.50	5–15
	(4.99)		(3.51)

WASI-II VCI Wechsler Abbreviated Scale of Intelligence-Second Edition Verbal Comprehension Index, Vineland-3 ABC Vineland Adaptive Behavior Scales, Third Edition Adaptive Behavior Composite, CELF-5 Clinical Evaluation of Language Fundamentals, 5th Edition

Instructor Training and Coaching

Instructors were undergraduate research assistants who completed a 6-hour training prior to the start of the study. Training included 4 hours of didactic instruction over Zoom and 2 hours observing video recordings of exemplars modeling the intervention procedures as well as engaging in individual and group practice. Prior to implementing the intervention, all research assistants demonstrated at least 90% fidelity on intervention procedures. Throughout the course of the intervention, instructors received coaching from postdoctoral fellows. Group sessions were held once every two weeks to review child progress and discuss instructional adjustments that would be appropriate for children who were struggling or not progressing appropriately. Coaches also ensured that all instructors’ behavioral supports or instructional adaptations were not jeopardizing the integrity of the intervention.

Intervention Session Procedures

The intervention used in this study was adapted from a listening comprehension and vocabulary intervention that has previously shown efficacy with children with autism (Henry & Solari, 2020; Solari et al., 2020). The intervention was designed to teach four comprehension skills: direct recall of textual information, making connections to background knowledge, narrative retell and narrative story elements, and making inferences. These skills were taught through a combination of explicit instruction and guided practice centered around authentic children’s storybooks. The sequence of books is organized by levels that differ based on story length and complexity: Level 1 storybooks are the simplest, Level 2 storybooks are slightly more challenging, and Level 3 storybooks include more complex language and content. In this study, instructional levels were assigned to each child based on their expressive language and verbal comprehension skills (i.e., children with lower language skills were assigned to Level 1).

The intervention was delivered 2 days per week, for 11 weeks. Each 30-minute session included the following components: (1) explicit instruction in or review of the target comprehension skill, (2) a read-aloud of a target children’s book, (3) instruction in story vocabulary, and (4) journal activities to support comprehension. Individual books were read across two lessons, with a full read-through of the book during each lesson. These repeated readings allowed participants to become more familiar with the plot and the characters of the story to support deeper comprehension (Henry & Solari, 2021). Given the telepractice format, the research team scanned and converted all books to PDFs, and instructors projected each page of the book over Zoom so that the pictures and text could be seen by the child.

Each read-aloud began with a guiding question that provided a purpose for that day’s reading, such as, “Who is the main character in this story and what are they trying to do?”. During the read-aloud, the instructor stopped at set points in the story to ask the child questions or engage in think-aloud modeling. The read-aloud portion of the lesson concluded with a review of the guiding question. If the child could not answer the guiding question, the instructor provided a binary choice (“Was the main character Tom or Jerry?”) before proceeding to a more intense scaffold (e.g., “Say, ‘Tom was the main character of the story’”) if the child continued to respond incorrectly.

Increased visual supports were added to the read-aloud portion of the intervention to make it more appropriate for remote settings and to increase engagement. Similarly to Henry and Solari (2020) and Solari et al. (2020), instructors used hand signals to accompany each comprehension skill. In this study, these hand signals were accompanied by physical props that were mailed to all participants and instructors. Props showed a picture of the intervention’s character mascot (a cartoon beaver) performing the hand signal. Other props included the “thumbs up” and “thumbs down” symbols, as well as emoticons indicating feelings like “happy,” “sad,” “angry,” and “scared.” These props allowed participants to provide a nonverbal response to instructor questions or to indicate excitement about the story. Additionally, further visual supports were included to support inference-making. When asking participants to make an inference about the story, instructors would first share screenshots of text or images from the book to indicate the “clues” needed to make the inference. Then, images of key “background knowledge” were shown. Finally, all images were presented on the screen as participants were encouraged to use these story clues and background knowledge to make the target inference.

Instructors also provided explicit instruction in story vocabulary. Two to three novel vocabulary words were taught for each book. During the first read-aloud of the target text, all words were introduced with a brief, child-appropriate definition. Following the read-aloud, the instructor re-defined each of the words and engaged the child in a brief vocabulary activity to promote retention. These vocabulary activities were interactive, asking children to provide examples and non-examples of a word, or to act out a word (e.g., physically pulling an object across a table for the word drag). Following the second read-aloud of that text, a review activity for each vocabulary word encouraged the child to practice the word in new contexts and identify words based off their definitions.

To encourage discussion and facilitate a greater understanding of the story, each intervention session concluded with a journal activity. Journals were mailed to participants’ homes prior to the start of the intervention, and children were able to write or draw their responses using this physical journal or a whiteboard that had also been provided by the researchers. Participants were always given the option to write or to draw their response. Journal activities were related to the target comprehension strategy. For example, activities for the Direct Recall unit included prompts to write or draw details from the story, whereas the Making Inferences unit asked children to fill out character maps to promote perspective-taking and connections between characters (Henry & Solari, 2021).

Intervention Procedures for Treatment Group Receiving Parent Component (BVERS + PC)

All participants received the twice-weekly, researcher-delivered intervention sessions, regardless of intervention condition. Children who had been randomly assigned to the second treatment condition also received supplemental, parent-mediated instruction. Parents of children in this treatment group were asked to conduct two additional read-alouds of the target text each week. Prior to beginning the intervention, parents were asked to review a brief training handout and video of how to conduct these read-alouds. Physical copies of all books were mailed to parents’ homes, including color-coded stickers indicating stopping points in the story and questions to be asked during the read-aloud. Parents read the first half of the book during the first supplemental read-aloud and finished the second half of the book during the second read-aloud. Parents logged information about these read-alouds on a weekly distributed survey.

Measures

Descriptive measures (including measures of adaptive behavior, verbal reasoning, and word structure) were assessed only at pretest to determine eligibility and characterize the study sample. Outcome measures included pre- and posttest assessments of expressive vocabulary, listening comprehension, and narrative retell. Subtests from the CELF-5 and Expressive Vocabulary Test, Third Edition (EVT-3; Williams, 2018) were administered via Q-Global, an online administration platform created by Pearson Education, Inc., for electronic delivery of these assessments. All other outcome measures did not include visual components and could therefore be delivered remotely without adaptations. Feasibility was measured via intervention markers such as child attendance and instructors’ procedural integrity, whereas parent satisfaction and intervention acceptability were measured via parent surveys.

Descriptive Measures

Adaptive Behavior

The Vineland-3 Domain-Level Parent/Caregiver Form (Sparrow et al., 2016) provided an age-normed standardized measure (M = 100; SD = 15) of home and family-life behavior (i.e., ABC). Adaptive behavior encompasses an individual’s ability to socialize and communicate effectively and to complete personal-care tasks. The parent/caregiver form has strong internal consistency for the age groups included in this study (α = 0.97).

Verbal Reasoning

The VCI was calculated using the WASI-II. The VCI is comprised of two subtests, Similarities and Vocabulary, and provides age-normed standard scores (M = 100; SD = 15). These subtests assess an individual’s ability to access and apply word knowledge, and have good reliability according to the manual (average r = 0.93).

Structural Language

The Word Structure subtest of the CELF-5 (Wiig et al., 2013) was used to evaluate acquisition of morphological knowledge and the ability to apply word structure rules and use pronouns to refer to people and possessive relationships. Examinees are asked to complete sentences by using the word form that matches a presented picture. Age-based scaled scores (M = 10; SD = 3) are provided. This subtest has strong internal consistency (α = 0.94) for children with autism based manual-provided special group study findings (Wiig et al., 2013).

Outcome Measures

Listening Comprehension

The Understanding Spoken Paragraphs subtest of the CELF-5 provided an age-normed standardized (M = 10; SD = 3) assessment of the ability to listen to and understand short stories and then respond to detail-oriented and inferential questions about the stories. This subtest has strong internal consistency (α = 0.91) for children with autism based manual-provided special group study findings (Wiig et al., 2013).

Expressive Vocabulary

Expressive vocabulary was assessed with the EVT-3 (Williams, 2018). In this assessment, individuals are asked to verbally identify a picture or provide an alternative word describing the picture. Age-based standard scores (M = 100; SD = 15) were used in all analyses of expressive vocabulary. Coefficient alphas for the ages included in the study ranged from 0.94 to 0.97 (Williams, 2018).

Narrative Retell

Narrative memory and retell was assessed with the Narrative Memory subtest of the Developmental NEuroPSYchological Assessment, Second Edition (NEPSY-II; Korkman et al., 2007). The NEPSY-II is a comprehensive instrument assesses the psychological development of preschool and school-age children. The Narrative Memory subtest evaluates the examinee’s ability to recall details from stories. Examinees listen to an orally read story and are asked to remember details under free recall, cued recall, and recognition conditions. Age-based scaled scores (M = 10, SD = 3) of free and cued retell were used in analyses of narrative memory for this study. The reliability coefficients for this construct ranged from 0.72 to 0.76 for the age ranges included in this study (Korkman et al., 2007).

Feasibility and Acceptability Measures

Feasibility

Trial feasibility was assessed using data related to recruitment (the number of caregivers who expressed interest and number consented), retention (number of participants who withdrew), and attendance (number of intervention sessions attended and number of no-shows).

Procedural Integrity

To measure treatment integrity and ensure that the intervention sessions were implemented consistently, fidelity checklists were used across the course of the study. Fidelity checklists included sections for the three core components of the intervention (read-aloud, vocabulary, and journal activity) as well as a section indicating the overall quality of engagement, instructional pacing, and preparation and organization of materials. Research members (undergraduate research assistants) used video recordings of intervention sessions to conduct these fidelity observations; fidelity was assessed in 3 randomly selected sessions for each instructor. Observations were conducted by research staff who were not involved in intervention delivery. Research staff established interobserver agreement (≥ 90%) with a postdoctoral fellow using the fidelity checklists prior to conducting independent observations.

Instructors also completed daily logs outlining the treatment components implemented during that day’s session (i.e., read-aloud, vocabulary, and/or journal activity). Instructors noted when an activity was skipped due to time constraints. Daily logs also contained information about child behavior, with instructors rating the child’s behavior on a 5-point rating scale ranging from 1 to 5 (1 = “Very Poor”, 2 = “Poor”, 3 = “Okay”, 4 = “Good”, 5 = “Excellent”). A rating of “Very Poor” would indicate that the child was extremely resistant and/or was completely unable to focus on the lesson, whereas a “Poor” rating would suggest that there were some interruptions or resistance, but the child was still able to complete some instructional activities. An “Okay” rating would indicate a more neutral session where the child may not have demonstrated high levels of involvement but was also not displaying negative behaviors. A “Good” rating would specify a session where the child was generally focused and involved in the activities, and an “Excellent” rating would indicate that the child was highly involved and excited about participating in the activities. Instructors also noted whether a parent or caregiver was present during the intervention session. Parents participating in the BVERS + PC treatment group filled out weekly logs indicating how many read-alouds were implemented that week and rated their child’s behavior on a scale similar to the daily logs.

Parent Satisfaction

Following the intervention, parents anonymously responded to an online survey via Qualtrics XM. The first eight items of the survey were adapted from Weiner et al. (2017) and asked parents to rate the acceptability and appropriateness of the intervention using a 5-point Likert scale ranging from 1 (“Completely Disagree”) to 5 (“Completely Agree”). This measure has shown good reliability (αs = 0.85–0.91). Remaining survey items were developed by the researchers for this study. Items 9 and 10 assessed parent satisfaction with implementation of the intervention and their child’s progress throughout the intervention using a 5-point Likert scale ranging from 1 (“Extremely Dissatisfied”) to 5 (“Extremely Satisfied”). Parents were also asked whether they believed the intervention was sufficient to meet their child’s needs and whether their child progressed in certain academic skills (e.g., listening comprehension) after participating in the intervention. Parents who participated in the BVERS + PC group answered further survey questions on the feasibility of the parent component and whether they felt that the parent component was a necessary additional factor in their child’s progress over and above the twice-weekly BVERS sessions.

Statistical Analyses

Analyses were conducted in SPSS 27. Due to the small sample size and non-normal distribution of the data, non-parametric tests were used in all analyses. Wilcoxon signed-ranks tests (non-parametric tests used for paired data) were used to detect significant differences between pre- and posttest scores on all outcome measures. As this was a pilot study, no adjustments were made for multiple comparisons (Althouse, 2016; Feise, 2002).

Pre-post change scores were used to analyze potential differences in intervention outcomes between treatment conditions. A Mann–Whitney U test was used to determine whether the change in the intervention-only condition (BVERS) was significantly different from the change in the parent component condition (BVERS + PC). Effect sizes (r) were calculated by dividing the critical-z value by the square root of the number of observations; (Pallant, 2007). This effect size may be interpreted within the guidelines of Cohen’s d (e.g., 0.20 = small effect, 0.50 = medium effect, and 0.80 = large effect; Cohen, 1988).

Results

Recruitment, Retention, and Attendance

The recruitment period for this study was one month. Twenty-three caregivers expressed interest in the study and asked for additional information. Seventeen (74%) consented and were assessed, and 14 (82% of those assessed) were determined to be eligible for the intervention. No participants withdrew from the study. Children attended 23 sessions on average (M = 22.71, SD = 2.40). Out of 331 total scheduled sessions across all participants, only 13 (3.92%) were no-shows. Six out of the 14 children had a documented no-show, and 3 children had more than one.

Procedural Integrity

Instructors’ adherence to activities, as measured by the fidelity checklist, was generally high. On average, instructors were able to adhere to 89.6% of the intended intervention components (SD = 14.3%, range 54–100%). Adherence was especially high for the read-aloud component for the intervention (97.9%, SD = 6.8%), but lower for the vocabulary (76.6%, SD = 34.6%) and journal activity components (82.9%, SD = 36%). Lower scores on the fidelity checklist were generally associated with running out of time to complete all intervention components, resulting in a score of zero. On the daily logs, instructors indicated that they were able to complete all intervention components (read-aloud, vocabulary, and journal activity) in the majority of sessions. Instructors indicated that they missed some intervention components in 39 sessions (12.3% of all sessions conducted). Child behavior rankings generally were ranked as “Good” or “Excellent” (64.4%), though participants in some sessions had behaviors that were rated as “Okay” (18.8%), “Poor” (15.7%) or “Very Poor” (1.1%). Parents or caregivers were present in 55.1% of all sessions.

On their weekly logs, 88.0% of caregivers participating in the BVERS + PC group indicated that they were able to complete the goal of two read-alouds per week. Parents’ ratings of their children’s behaviors during these read-alouds were generally “Good” (66.0%) with equal endorsement of “Okay” (16.0%) and “Excellent” (18.0%). No parents rated their children’s behavior as “Poor” or “Very Poor.”

Parent Satisfaction

A total of 10 parents (71.4%) completed the parent satisfaction survey. On average, parents rated the four items on acceptability highly (M = 4.70, SD = 0.48), which suggests that parents generally found this intervention acceptable for use with their child. Parents rated the test items on appropriateness as slightly lower but still within the “Agree” range on average (M = 4.22, SD = 1.18), which suggests that while generally, parents believe that this intervention is appropriate for use with children with autism, some may be unsure. The distribution of these scores can be found in Table 2. All parents reported being “Extremely Satisfied” or “Satisfied” with the implementation of the intervention, and 80% reported being “Extremely Satisfied” or “Satisfied” with their child’s progress. One parent (10%) reported that they were “Neither Satisfied nor Dissatisfied” with their child’s progress, and one stated that they were “Dissatisfied.” Seventy percent of parents indicated that the level of intervention was “sufficient” or “Somewhat Sufficient” to meet their child’s needs, while 20% reported that they were unsure. One parent stated that the level of intervention was insufficient to meet the needs of their child. Sixty percent of parents surveyed indicated that their child improved in the ability to attend to online instruction, and 50% saw growth in language and communication, 50% in listening comprehension, and 50% in reading comprehension.

Table 2

Distribution of Responses to Acceptability and Appropriateness Survey Questions

Item	Completely disagree	Disagree	Neither agree nor disagree	Agree	Completely agree	M	SD
	n (%)	n (%)	n (%)	n (%)	n (%)
The intervention meets my approval	–	–	–	3 (30.0%)	7 (70.0%)	4.70	0.48
The intervention is appealing	–	–	–	3 (30.0%)	7 (70.0%)	4.70	0.48
I like the intervention	–	–	–	3 (30.0%)	7 (70.0%)	4.70	0.48
I welcome use of the intervention	–	–	–	3 (30.0%)	7 (70.0%)	4.70	0.48
The intervention seems fitting for my child	–	2 (20.0%)	–	2 (20.0%)	6 (60.0%)	4.20	1.23
The intervention seems suitable for my child	–	2 (20.0%)	–	2 (20.0%)	6 (60.0%)	4.20	1.23
The intervention seems applicable for my child	–	1 (10.0%)	–	3 (30.0%)	6 (60.0%)	4.40	0.97
The intervention seems like a good match for my child	–	2 (20.0%)	1 (10.0%)	1 (10.0%)	6 (60.0%)	4.10	1.29

Of the seven parents who participated in the BVERS + PC group, a total of five parents (71.4%) responded to the survey. All parents indicated that the weekly read-alouds were “Feasible” or “Somewhat Feasible” to complete on a weekly basis and that the instructions for the read-alouds were easy to understand and implement. While four parents stated that the parent component was necessary for their child to see progress over and above the twice-weekly sessions, one parent indicated that they were unsure.

Initial Efficacy

Wilcoxon signed-rank tests were conducted on standardized measures of expressive vocabulary, narrative retell, and listening comprehension. Overall, participants did not show significant growth in EVT-3 (z =  − 0.87, p = 0.39, r = -0.23) or NEPSY-II Narrative Memory scores (z =  − 1.15, p = 0.25, r =  − 0.31). However, they demonstrated statistically significant improvement on CELF-5 Understanding Spoken Paragraphs, z =  − 2.06, p = 0.04, r = − 0.55.

Mann–Whitney U tests were conducted on change scores (the difference between pretest and posttest scores) for all standardized assessments, revealing no statistically significant differences between treatment conditions (BVERS and BVERS + PC) on the EVT-3 (U = 17.50, z =  − 0.90, p = 0.37, r = − 0.24), NEPSY-II Narrative Memory (U = 17.50, z =  − 0.91, p = 0.36, r = − 0.24), or CELF-5 Understanding Spoken Paragraphs (U = 22.50, z =  − 0.26, p = 0.79, r = − 0.07). See Table 3 for a summary of these results.

Table 3

Wilcoxon signed rank (all groups) and Mann–Whitney U (BVERS and BVERS + PC) tests

Variable	Wilcoxon Signed Rank				Mann–Whitney U
					BVERS		BVERS + PC
	Pretest Mean (Range)	Posttest Mean (Range)	Z	r	Pretest Mean (Range)	Posttest Mean (Range)	Pretest Mean (Range)	Posttest Mean (Range)	Z	r
CELF-5 USP	5.57 (2–9)	6.50 (1–12)	− 2.06*	− 0.55	7.00 (2–9)	8.00 (2–11)	4.14 (2–9)	5.00 (1–12)	− 0.26	− .07
EVT-3	92.00 (69–128)	94.00 (72–128)	− 0.87	− 0.23	100.14 (83–128)	103.57 (82–128)	83.87 (69–101)	84.43 (72–99)	− 0.90	− .24
NEPSY-II NM	3.43 (1–7)	4.00 (1–10)	− 1.15	− 0.31	4.00 (1–7)	5.43 (1–10)	2.86 (1–6)	2.57 (1–7)	− 0.91	− .24

CELF-5 USP Clinical Evaluation of Language Fundamentals, 5th Edition, Understanding Spoken Paragraphs subtest (scaled scores); EVT-3 Expressive Vocabulary Test, 3rd Edition (standard scores); NEPSY-II Narrative Memory Developmental NEuroPSYchological Assessment, 2nd Edition, Narrative Memory subtest (scaled scores)

*p < .05

Discussion

With the rise of the COVID-19 pandemic, many students have been receiving distance education support, including children with autism. However, there is currently no evidence base for telepractice supporting reading development for students from this population, though there is a clear need for efficacious intervention programs impacting reading-related skills such as listening comprehension. This study begins to fill in this gap by providing preliminary evidence for the feasibility and potential efficacy of a telepractice listening comprehension intervention for elementary-aged children with autism.

Intervention Feasibility

Overall, the results of this study support the feasibility of the intervention. Attendance data were strong; participants only missed 3.92% of planned sessions. Additionally, there was no attrition, as all participants who were consented and met eligibility criteria completed the 11-week intervention. These attendance data suggest that participating families were able to feasibly incorporate the intervention sessions into their weekly routines.

Data on treatment integrity and fidelity also supported the feasibility of the intervention. Overall fidelity data hovered just under 90%. Importantly, fidelity scores were excellent for the read-aloud portion of the intervention session, which included most of the core instruction and support in target reading comprehension strategies. Scores were lower for vocabulary and journal activity components, typically because all or some of the activities were not completed. On daily logs, tutors often indicated that intervention components were skipped if the parent signed onto the session late or if child behavior made it difficult to complete the full session. Child behavior rankings were generally good, but there was still a sizeable portion of intervention sessions (34.5%) where instructors indicated that the child had exhibited “Poor” or “Very Poor” behavior. Future research studies would benefit from additional strategies to support engagement and prevent distractions, especially in remote settings. Additionally, scheduled reminders to parents may address issues of tardiness.

Results of the parent satisfaction survey suggest that parents found the intervention acceptable, and all parents reported that they liked the intervention and were satisfied with the implementation of the intervention. However, parents provided a mixed response on whether the intervention was appropriate for their child. In a comment on the satisfaction survey, one parent noted that her child had more significant language needs and required more scaffolds and visual supports to access the intervention. While several visual communication tools were added to this iteration of the intervention to support engagement, future studies should consider further adaptations for learners with attention difficulties or low cognitive and language abilities.

The findings from the study suggest that the intervention may have had a positive effect on listening comprehension growth for the participants according to results from the CELF-5 Understanding Spoken Paragraphs subtest. While there were no discernable effects of the intervention on EVT-3 or NEPSY-II Narrative Memory score, growth on the measure of listening comprehension, with a medium effect size, suggests that participating children were learning and retaining key comprehension strategies. Listening comprehension skills are related to and predictive of later reading comprehension outcomes (Kendeou et al., 2005; Nation & Snowling, 2004), therefore it is important to build a strong foundation for these skills before moving into the higher grades.

While these data are preliminary, it should also be noted that no statistically significant differences between the BVERS and BVERS + PC groups were found on any of the language or listening comprehension measures. This suggests that students receiving the intervention only twice a week from an instructor responded to the intervention just as well as students receiving the supplemental parent component. Results support the feasibility and acceptability of the parent component, as parents indicated that they were able to implement the twice-weekly supplemental component on a regular basis and the majority reported that they believed the supplemental read-alouds made a difference in their child’s comprehension growth over and above the regular BVERS sessions. Therefore, while these data do not provide enough information to determine whether two supplemental sessions per week are enough to promote additional growth in language abilities, parents feel that it is a feasible and worthwhile component to add to the existing intervention.

Implications for Practice and Research

Overall, this intervention could be reliably implemented remotely in short sessions and ranked high in terms of acceptability and parent satisfaction. The scripted nature of the curriculum reduced initial instructor training requirements, yet procedural integrity remained high. This suggests that this intervention is practical for use in a variety of settings and by many users, including classroom teachers or parents of children with autism. It may not be feasible for families who live in rural or remote areas to drive to a physical location to receive reading services. This intervention would allow for a remote option to receive these services in or outside of school. Additionally, this intervention targets multiple areas of concern for children with autism including oral language, vocabulary, and listening comprehension.

Limitations and Future Directions

This study should be considered in light of several limitations. First, the small sample size and lack of a true control group make it difficult to assess whether the significant change in participants’ listening comprehension scores was due to the intervention. For example, growth in listening comprehension may be due to external factors such as reading services received in school. A number of parents attended the intervention sessions alongside their children; they may have picked up the comprehension strategies taught in the BVERS intervention and used these in their independent read-alouds with their children. Future research studies should also consider adjusting assessment procedures, as the assessors in this study were not blind to condition. It would also be useful to include curriculum-based or proximal measures to examine intervention efficacy, as several of the standardized measures used in this study (i.e., CELF-5 and NEPSY-II) did not have alternate forms available and children may have remembered details from prior readings of the stories. Additionally, the small sample does not allow for additional analyses that might determine potential mediators or moderators of participant outcomes, such as curriculum level, fidelity, or instructor characteristics. More research is needed to determine whether a longer or more intensive intervention would have effects on other outcomes of interest, including vocabulary and narrative retell. Further examination of the parent component is also needed to determine its value in improving child outcomes. It may be that increasing the scope or intensity of this component may allow parents to effect more change for their children with autism.