Parent Perceptions of Caregiver-Mediated Telemedicine Tools for Assessing Autism Risk in Toddlers.

Telemedicine tools have potential for increasing access to diagnostic services for children with autism spectrum disorder (ASD). Past work has utilized tele-assessment procedures in which remote psychologists observe administration of interactive screening instruments by trained, on-site providers. Although promising, this approach relies on two clinicians, limiting its efficiency and scalability. The present study examined the use, acceptability, and parents' perceptions of two caregiver-mediated tools for assessing ASD risk in toddlers, in which remote clinicians guided parents to complete interactive screening activities with their children. Most parents found tele-assessment to be comfortable, and many reported liking the parent-led nature of these tools. Parents also offered constructive feedback, which was used to modify the tele-assessment process for future study.

As the prevalence of autism spectrum disorder (ASD) has increased, so too have needs for screening, diagnosis, and intervention access for young children. Current prevalence estimates indicate that one in 54 children has ASD (Maenner 2020). To maximize early identification and intervention, American Academy of Pediatrics guidelines recommend combining routine developmental surveillance at all pediatric visits with standardized autism-specific screening for all children at 18 and 24 months of age, as well as at any point that caregivers report concerns (Hyman et al. 2020). However, universal screening remains aspirational (Guthrie et al. 2019; Zwaigenbaum and Maguire 2019), and many children are not diagnosed with ASD until after four years of age even when developmental concerns are present during their toddler years (Maenner et al. 2020). Although reasons for these diagnostic delays are likely multifaceted (Bishop-Fitzpatrick and Kind 2017; Mazurek et al. 2014), a lack of qualified diagnostic professionals (Hyman and Johnson 2012; Kalkbrenner et al. 2011) as well as socioeconomic, geographic, and linguistic barriers (Antezana et al. 2017; Durkin et al. 2010; Khowaja et al. 2015) make it difficult for families to access care. Telemedicine tools may reduce this care disparity by increasing access to diagnostic and treatment services for individuals with ASD, particularly in rural and underserved communities where access to qualified professionals is limited.

Emerging research has examined the feasibility and acceptability of telemedicine procedures in the assessment and diagnosis of ASD, using video technologies to allow clinicians to remotely observe child behavior and converse with caregivers (Juárez et al. 2018; Kanne et al. 2018; Reese et al. 2015; Stainbrook et al. 2019). For example, Juárez et al. (2018) compared a tele-assessment procedure in which a remote psychologist observed administration of the Screening Tool for Autism in Toddlers and Young Children (STAT; Stone et al. 2000) by a trained, on-site early intervention provider and found that remote psychologists correctly identified approximately 80% of children later diagnosed with ASD via comprehensive evaluation. In the second phase of this work, Juárez et al. (2018) investigated feasibility and parent perceptions of the telemedicine assessment procedures. They found that parents reported high levels of satisfaction with the tele-assessment process, indicating that the process was convenient, private, and saved them an average of four hours of travel time from rural regions. Most parents (90%) indicated that, if they had not received the telemedicine evaluation, they would have waited for an appointment at the more distant diagnostic center, resulting in later diagnosis and potentially delayed access to early intervention services.

Although a promising method for increasing access to diagnostic care, this work did not address a key barrier to care regarding the human resource requirements of assessment. This approach to tele-assessment of ASD relies on the availability and dedicated time of two expert providers (i.e., STAT administrator and observing clinician), which ultimately limits its efficiency and scalability. Another novel approach to tele-assessment has been the involvement of parents or caregivers in completing structured activities with their children, under the verbal guidance of a clinician observing remotely. This type of in-the-moment remote coaching has long been used in parent training interventions, such as parent–child interaction therapy (Funderberk and Eyberg 2011), in which a therapist coaches parents from another room using a wireless microphone and earpiece. The use of telemedicine for remote coaching is increasingly common in parent-mediated interventions for children with ASD (Lindgren et al. 2016; Pickard et al., 2016; Vismara et al. 2012), but this modality has been less often studied for the purpose of ASD assessment.

In one initial parent-guided approach to ASD assessment (Reese et al. 2015), a remote clinician guided parents through a modified administration of the autism diagnostic observation schedule, second edition (ADOS-2; Lord et al., 2012). The assessment procedure included observation of the child’s free play, interview questions with parents, and a modified ADOS-2 administration. Parents were guided through evaluation procedures by a psychologist, who was either in the room with the family or interacting via video conference technology. Parent guidance was completed through a description and orientation to ADOS-2 activities, as well as viewing a 16-min video modeling the activities parents would complete. Additional research clinicians observed the visits, both in-person and via video technology, to provide data on inter-rater agreement. Children later received interdisciplinary comprehensive assessments within 60 days of participating in the tele-assessment visit. When comparing diagnostic impressions based on the parent-guided procedure to diagnoses assigned following comprehensive evaluation, remote clinicians achieved diagnostic accuracy for 86% of participants. In an earlier study using this procedure, parents reported high levels of satisfaction with the process and noted that they felt comfortable during the assessment (Reese et al. 2013).

There are several benefits to, as well as precedence for, the active involvement of parents during ASD assessments (Head and Abbeduto 2007; Huerta and Lord 2012; Luyster et al. 2009). Many modules of the ADOS-2 involve the presence and scripted involvement of parents or caregivers (Lord et al. 2012). For example, caregivers may be asked to call their child’s name, demonstrate what makes their child smile, or participate in a play routine with their child. Clinicians also observe how a child initiates and responds to interactions with his or her parent or caregiver. This type of observation is valuable, as it is expected that young children behave differently with familiar caregivers than with unfamiliar adults. Furthermore, observing a child interact with a close, familiar caregiver may provide insight into that child’s best and/or most typical social interactions.

Though parents and caregivers play an essential role in ASD assessment, both in providing information about their children and through interacting with their children, little is known about parents’ perspectives on their involvement in the assessment process. Parents often report dissatisfaction with the process of receiving an ASD diagnosis and describe it as stressful (Crane et al. 2016; Howlin and Moore 1997). Parent satisfaction with the diagnostic process is impacted by factors including the wait time for receiving a diagnosis, the manner in which providers share an ASD diagnosis, and the amount of help and support parents receive during and following the diagnosis (Crane et al. 2016; Howlin and Moore 1997; Osborne and Reed 2008).

Absent from the literature on parent perceptions of the diagnostic process is information regarding parents’ role and active involvement during the diagnostic evaluation. In contrast, parent involvement in ASD intervention approaches has been well studied (Meadan et al. 2009; Wetherby et al. 2014), with parents often reporting satisfaction with and benefit from parent-implemented interventions (McConachie and Diggle 2007; Stahmer et al. 2017). There is also some indication that interventions requiring a high level of parent involvement are associated with high levels of parent stress (Kasari et al. 2015). Similarly, it is possible that parents may benefit from and appreciate being involved directly in ASD assessment procedures. However, it is also possible that parents may experience direct involvement in assessment activities as stressful.

From a telemedicine perspective, involving parents in the administration of assessment activities may reduce or eliminate the need for a second trained examiner in addition to a remote clinician. However, no formal tools or procedures have been thoroughly evaluated for use in caregiver-mediated ASD assessment. As Reese et al. (2015) described, initial attempts to coach parents to correctly administer ADOS-2 activities involved detailed descriptions of activities, explanations of multiple social bids and presses, and viewing of video models. Further, it may be challenging to score and interpret the ADOS-2 using this modified approach.

A goal of the present study was to determine whether assessment tools specifically created or adapted for tele-assessment of ASD could be used by parents in a comfortable, efficient way, while allowing clinicians to remotely observe child behavior and possible symptoms of ASD. As part of a larger project evaluating the diagnostic accuracy and clinical utility of two novel, parent-guided screening tools either developed (TELE-ASD-PEDS) or adapted for (TELE-STAT) the remote assessment of ASD in young children, we conducted preliminary evaluations of parent perceptions and feedback regarding these telemedicine tools. This critical parent feedback on the telediagnostic process was collected to understand parents’ experience and perceptions, as well as to provide data to further adapt the TELE-ASD-PEDS and TELE-STAT for use in this manner. We hypothesized that parents would generally provide data regarding the acceptability of the tele-screening process, but would also offer key qualitative feedback for enhanced use over time.

Methods

The data included in the present study are part of the initial feasibility and optimization component of a larger, ongoing clinical trial (clinicaltrials.gov, 2019, NCT03847337) investigating the accuracy and limits of telediagnostic assessment in children with concerns for ASD.

Participants

Participants were recruited from two sources: from an existing large clinical research database consisting of children with ASD and developmental delay (DD) (n = 35) and from a clinical waitlist for evaluation of developmental concerns related to ASD (n = 16). Eligibility criteria included children being 18–36 months of age and having a primary caregiver that spoke English. Participants were excluded if children had significant sensorimotor differences (e.g., visual impairment, motor impairment) that would have impacted ability to complete assessment activities. Of 97 eligible families contacted about study participation, 51 consented to enroll.

Participants included 51 children (36 boys, 15 girls) between 1.63 and 3.01 years of age (m = 2.49 years, SD = 0.35 years) and their caregivers (see Table 1). Of these, 35 children were diagnosed with ASD, 10 had developmental delays, and six were typically developing. Caregivers included 45 mothers and eight fathers.

Table 1

Participant demographics

	ASD (n = 35)n (%)	DD (n = 10)n (%)	TD (n = 6)n (%)
Toddlers
Age (m[SD])	2.51 (0.32)	2.43 (0.46)	2.45 (0.37)
Male	26 (74%)	7 (70%)	3 (50%)
Female	9 (26%)	3 (30%)	3 (50%)
Toddler race
White	22 (63%)	6 (60%)	4 (67%)
Black or African American	6 (17%)	3 (30%)	1 (17%)
More than one race	2 (6%)	–	1 (17%)
Opted not to answer	5 (14%)	1 (10%)	−
Toddler ethnicity
Hispanic or Latino	3 (9%)	–	–
Not Hispanic or Latino	21 (60%)	8 (80%)	4 (67%)
Opted not to answer	11 (31%)	2 (20%)	2 (33%)
Household income ($)
< 10,000	1 (3%)	1 (10%)	1 (17%)
10,000–20,000	2 (6%)	1 (10%)	–
20,000–30,000	–	3 (30%)	1 (17%)
30,000–40,000	4 (11%)	1 (10%)	–
40,000–50,000	1 (3%)	–	–
50,000–60,000	5 (14%)	–	–
60,000–70,000	1 (3%)	–	1 (17%)
70,000–80,000	6 (17%)	2 (20%)	–
80,000–90,000	3 (9%)	1 (10%)	–
90,000–100,000	3 (9%)	–	–
> 100,000	7 (20%)	–	2 (33%)
Opted not to answer	2 (6%)	1 (10%)	–

ASD Autism spectrum disorder, DD developmental delay, TD typically developing

Of the participants recruited from a clinical research database, 22 children had a prior diagnosis of ASD; eight children had a prior diagnosis of developmental delay; and five children were typically developing. Prior diagnostic evaluations for children with ASD or DD were completed by clinical psychologists (n = 23), pediatricians (n = 3), or developmental pediatricians (n = 3) between 26 and 391 days (m = 178.65, SD = 119.23) prior to completing tele-screening. Given (a) the study’s preliminary emphasis on understanding parent perceptions of the telediagnostic process rather than validation of tools themselves and (b) that children recruited from this registry were evaluated by a range of providers at various intervals of time prior to participation in the study, data regarding current cognitive, adaptive, and ADOS-2 scores are not available for these participants.

Families recruited from the clinical waitlist completed telehealth screening on the same day that they received a full diagnostic evaluation by a clinical psychologist or licensed senior psychological examiner. For these participants, tele-screening was completed immediately prior to full diagnostic evaluations, and clinicians who completed diagnostic evaluations were not aware of remote assessors’ clinical impressions. Of these children, 13 received diagnoses of ASD, two received diagnoses of developmental delay, and one was typically developing. Cognitive, adaptive, and ADOS-2 scores for this subset of participants are presented in Table 2.

Table 2

Cognitive, adaptive, and ADOS-2 scores by diagnostic status for participants receiving diagnostic evaluations on the same day as tele-screening

	ASD (n = 13)		DD (n = 2)		TD (n = 1)
	m	SD	m	SD	Score
ADOS-2 Total Score	21.00	4.88	6.00	2.82	2.00
MSEL-ELC	54.85	12.52	68.50	7.78	114.00
VABS-2-ABC	59.38	9.10	64.50	2.12	91.00

MSEL-ELC Mullen Scales of Early Learning, Early Learning Composite Standard Score; VABS-2-ABC Vineland Adaptive Behavior Scales-Second Edition, Adaptive Behavior Composite

Remote assessors (n = 7) were licensed clinical psychologists and licensed senior psychological examiners with expertise in diagnosing ASD in young children. All remote assessors were research reliable on the ADOS-2 and engaged in active clinical practice focused on ASD diagnosis. Remote assessors were blind to children’s clinical diagnoses.

Measures

TELE-STAT

The TELE-STAT was adapted from the STAT for the purpose of tele-screening. The STAT is a play-based Level 2 ASD screening tool designed and validated for use with children between 14 and 47 months of age (Stone et al. 2004, 2008). It includes 12 clinician-led activities intended to elicit child behavior in four different domains: play, requesting, directing attention, and imitation. Specific activities include encouraging the child to roll a ball or car back and forth with the examiner, presenting a snack and waiting for the child to make a request, and presenting motor activities for the child to imitate. Each activity is scored as a “pass” (score of 0), indicating that the behavior or skill was observed, or a “fail” (score or 0.25, 0.5, or 1 depending on domain) indicating that the child did not demonstrate the behavior in question. Scores across all activities are summed, resulting in a total score between 0 and 4, with scores greater than 2 indicating autism risk.

In the TELE-STAT, adaptations included a brief introduction to communicate to parents that the remote assessor would ask them to provide specific prompts and complete specific activities with their child. Remote assessors also asked parents for clarification where necessary regarding parent impression of child eye contact. Otherwise, remote assessors used the STAT standardized script to coach parents through implementation. Remote assessors completed the STAT standardized scoring following each activity. STAT scores are presented in Table 3.

Table 3

TELE-STAT and TELE-ASD-PEDS scores [m(SD)] by clinical diagnosis-telemedicine diagnosis

	ASD-ASDn = 33	No ASD-ASDn = 5	ASD-no ASDn = 2	No ASD-No ASD (n = 11)
TELE-STAT
N	16	2	1	5
Total TELE-STAT score	2.75 (0.96)	2.38 (1.23)	0.5	0.55 (0.33)
TELE-ASD-PEDSa
N	17	3	1	6
ASD-PEDS Total Scoreb	15.53 (1.77)	16.00 (2.65)	11.00	8.83 (1.72)
Socially directed speech and sounds	2.18 (0.53)	2.33 (0.58)	1.00	1.17 (0.41)
Frequent and flexible eye contact	2.24 (0.44)	2.67 (0.58)	2.00	1.33 (0.52)
Unusual vocalizations	2.47 (0.44)	2.33 (0.58)	1.00	1.17 (0.41)
Unusual or repetitive play	2.12 (0.49)	2.33 (0.58)	2.00	1.33 (0.52)
Unusual or repetitive body movements	2.29 (0.77)	2.67 (0.58)	1.00	1.50 (0.55)
Combines gestures, eye contact, and speech/vocalization	2.35 (0.61)	2.67 (0.58)	2.00	1.17 (0.41)
Unusual sensory exploration or reaction	1.88 (0.86)	1.00 (0.00)	2.00	1.17 (0.41)

aMean Likert-style ratings for individual TELE-ASD-PEDS items are presented. 1 = symptom not present; 2 = symptom present but at subclinical levels; 3 = symptom obviously consistent with ASD

bTotal ASD-PEDS scores were created by summing Likert-style ratings for all items, resulting in a lowest possible score of 7.00 and a highest possible score of 21.00

TELE-ASD-PEDS

The TELE-ASD-PEDS is a novel tool developed specifically for remote assessment of young children. It is derived from the application of machine learning technology to a database containing gold standard diagnostic tool data from several hundred individuals with and without ASD (Adiani et al. 2019). The most predictive items for young children were derived and then applied to interactive, parent-led tasks with guidance from expert clinicians (i.e. 2 ADOS-2 trainers and 5 ADOS-2 research reliable psychologists). The TELE-ASD-PEDS includes 12 discrete social activities or social bids, including opportunities for free play, opportunities for physical play routines (e.g., peekaboo, tickling), and activities (e.g., bubbles, snack) that may prompt a child to request. Following administration, clinicians rate child behavior on seven items (e.g., socially directed speech and sounds, use of eye contact, atypical vocalizations, unusual or repetitive play), allowing for dichotomous (yes/no) as well as Likert (3 = behaviors characteristic of ASD clearly present; 2 = possible atypical behavior; 1 = behaviors characteristic of ASD not present) scoring regarding presence and severity of predictive symptoms. Specific behaviors coded and associated Likert-style ASD-PEDS scores are presented in Table 3.

Parent Questionnaire

Immediately after the evaluation, parents answered seven questions related to pacing of the visit, content of the procedures, and usability of the materials, as well as their comfort with the process. Each question was scored along a Likert-type scale (1 = not true, 2 = somewhat true, 3 = very true). Parents were also asked to provide open-ended feedback, including suggestions for changes to the process and whether there were aspects of tele-screening they liked more or less compared to their child’s full diagnostic evaluation.

Procedure

Participants were randomized to receive one of two tele-screening procedures: the TELE-STAT (n = 24) or TELE-ASD-PEDS (n = 27). Randomization occurred prior to consent and assent. Group assignments were issued after consent and assent were obtained. All procedures were approved by the Institutional Review Board. No adverse events occurred.

Participants were escorted to the tele-screening room by a research assistant who oriented them to the technology and the space, including the test materials (e.g., toys, bubbles, snacks) on wall-mounted shelves. The research assistant established a connection with the remote assessor using the video conference technology and ensured that both the clinician and family could see and hear each other. The tele-screening visit began after the research assistant left the room. During the tele-screening process, parents interacted with a remote assessor via a wall-mounted monitor, speakers, and a HIPAA-compliant video platform (Cisco Systems 2015). This configuration allowed for two-way audiovisual communication and camera control by the remote assessor.

Remote assessors followed scripts and guided parents through structured interactions with their children, following the specific procedures described above for either the TELE-STAT or TELE-ASD-PEDS. The tele-screening procedure took an average of 23 min (SD = 5.00), with no significant difference between the two screening tools. Each measure was coded according to its instructions regarding behaviors that the clinician observed. Remote assessors then documented their diagnostic impressions (ASD vs. no ASD) immediately following screening. Diagnostic impressions were not shared with participants but were used to provide preliminary information about remote assessors’ agreement with clinical diagnoses. Immediately following the tele-screening, parents completed the parent questionnaire to provide feedback on the process. Of participants who completed tele-screening immediately prior to full diagnostic evaluations (n = 16), parents provided their feedback on tele-screening after the tele-visit but prior to the diagnostic evaluation. They completed a second brief questionnaire comparing the two visits immediately after the diagnostic evaluation.

Results

Parent Perception of Tele-Assessment

Survey responses from parents indicated that many found tele-assessment procedures to be acceptable and comfortable (see Table 4). Most parents reported that they felt comfortable playing with their children during the tele-screening, found the spoken instructions easy to follow, and believed that the screening lasted about the right amount of time. A minority of parents (12%) indicated that screening activities did not elicit the child behaviors about which they were most concerned.

Table 4

Quantitative parent feedback by diagnostic status

	Total	ASD (n = 35)	DD (n = 10)	TD (n = 6)
It felt comfortable for me to play with my child as part of this screening	2.94 (0.31)	2.94 (0.34)	2.90 (0.32)	3.00 (0)
The spoken instructions were easy to follow	2.94 (0.24)	2.91 (0.28)	3.00 (0)	3.00 (0)
Before I started, I understood what I would be doing	2.88 (0.33)	2.86 (0.36)	2.90 (0.32)	3.00 (0)
I feel like our doctor could use this activity with my child and me	2.86 (0.41)	2.85 (0.44)	2.90 (0.32)	2.83 (0.41)
The screening activity lasted about the right amount of time	2.80 (0.49)	2.83 (0.45)	2.90 (0.32)	2.50 (0.84)
After this type of screening activity, would you feel comfortable with someone talking to you about an autism diagnosis for your child?	2.75 (0.48)	2.71 (0.52)	2.90 (0.32)	2.67 (0.52)
The activities got my child to show the behaviors I was concerned about	2.25 (0.66)	2.34 (0.59)	2.10 (0.74)	2.00 (0.89)

Participants responded to each item using a Likert scale, with 1 representing “Not True,” 2 representing “Somewhat True,” and 3 representing “Very True.” Participant ratings on these items did not differ across any comparisons run (p’s greater than .05)

Nonparametric analysis using Mann–Whitney U tests yielded no significant differences (p values > 0.05) on these items between parents who completed the TELE-STAT and those who completed the TELE-ASD-PEDS. Similarly, Mann–Whitney U tests indicated no differences (p values > 0.05) in parent ratings of these items as a function of whether children received full evaluation on the same day as tele-screening or on a previous date. Finally, Mann–Whitney U tests indicated no significant differences (p values > 0.05) in parent ratings as a function of remote assessors’ diagnostic agreement (i.e., whether remote assessors’ impressions were consistent or inconsistent with children’s clinical diagnoses).

Overall, when asked whether they would prefer to play with their child as part of screening or observe a psychologist play with their child, 19% of parents indicated that they would prefer to play with their child by themselves, 4% would prefer to observe a psychologist interact with their child, and 77% would prefer to both play and observe.

Parent Suggestions for Optimized Telemedicine Assessment Process

Examination of parents’ qualitative responses provided additional information about aspects of tele-assessment that parents enjoyed, as well as constructive feedback on suggested changes to tele-assessment procedures. In total, 45 parents provided 116 written comments (see Table 5), including comments about technology (n = 25), the parent-led nature of the screening process (n = 21), comparisons between full evaluation and tele-screening (n = 16), and the convenience of the process (n = 8).

Table 5

Themes in qualitative parent feedback

Theme	Number (percent) of written commentsN (%)	Examples
Technology	25 (22%)	“The video kept cutting in and out making it hard to hear and understand what was being said.”
Parent-led nature of screening	21 (18%)	“My child interacted with me rather than people he didn't know” “I enjoyed being a part of the process but worried my interaction was too much or too little”
Comparing diagnostic evaluations to tele-screening	16 (14%)	“It feels like if the screening tool was just that- a screening, it would be good. But maybe a full evaluation is still needed? For whatever reason, [my child] often performs well at these type of things- like today, he didn't get frustrated or have a meltdown. I wonder if he would have been overlooked if he had only received this screening instead of the full evaluation he got.” “I like the telemedicine as it included me as a parent but I also like the full evaluation as it allows me to see what happens when someone else is working with my child.”
Time	11 (9%)	“I feel like a screening should last a little longer. Maybe 30–45 min.” “It was fast and was able to hold his attention with all the different task.”
Comfort/convenience	8 (7%)	“Really easy screening. Very comfortable for both baby and I.”
Screening space and activities	7 (6%)	“I would change to have toys out of eyesight for child. Maybe in a box next to parent.”
Clinician and staff communication	9 (8%)	“I appreciated the questions about eye contact and the doctor's patience.”
Other	19 (16%)	“It was about the same play wise” “Never had a visit like this for her.”

When asked about changes they would make to the screening visit, 19 parents provided comments. Most comments (n = 13) focused on technology, including concerns related to audio quality, the small size of the screen, and their children’s interest in accessing or touching the screen and camera. Parents also commented on the screening activities (n = 4), including suggestions for adding more or different activities. Two parents suggested extending the amount of time for screening. As noted above, screening visits lasted 23 min, on average. One parent wrote that a screening visit of 30–45 min would be preferable.

When asked what they liked more about the tele-screening, compared to their child’s full diagnostic evaluation, 32 parents provided a total of 35 comments. The majority of these comments (n = 14) emphasized the parent-led nature of the screening process. Parents commented that they liked being involved and that their children were more comfortable or better able to pay attention when interacting with a parent rather than an unfamiliar clinician. Other comments focused on the shorter time of the screening as compared to a full diagnostic evaluation (n = 6), the convenience of the screening process (n = 4), and that the screening purpose and instructions were easy to understand (n = 3).

When asked what they liked less about the tele-screening process, 15 parents provided a total of 16 comments. Three parents wrote that there was nothing they liked less. Four parents commented on technology issues. Four parents indicated that the full evaluation had gone into more detail, or that the behaviors they were most concerned about were not apparent during the tele-screening. Three parents noted that the tele-screening process was less personal than the full evaluation, and one parent voiced concern that they were not administering the activities correctly.

Finally, when asked for other comparisons between the screening and full evaluation or other general comments, 36 parents provided a total of 46 comments. These comments addressed many of the topics described above, including technology, parent and child comfort, and parents’ role in the process. Some parents (n = 8) commented that they thought that tele-screening may be useful as an initial step in the evaluation process, but also wanted face-to-face interaction with a provider or an opportunity for a provider to interact with their child as well (e.g., “I feel it was a comfortable tool for parent child but because it’s not in person, it might be easier to miss something, for that reason maybe should have more than one screening”).

Diagnostic Agreement

Preliminary diagnostic agreement information is presented in Table 3. Remote clinicians accurately identified 33 children with clinical ASD diagnoses and accurately identified 11 children who did not have ASD. Five children with developmental delays were inaccurately identified as having ASD, and two children with ASD were not identified as such by remote assessors. Although a relatively small proportion of the sample, none of the typically developing children were incorrectly identified as having ASD. Overall diagnostic agreement was 86%. Analysis of diagnostic agreement and characteristics of children who were accurately vs. inaccurately identified is ongoing.

Discussion

This work evaluated parent perceptions of two promising tools either adapted for or designed intentionally for the use of telemedicine to improve access to services for children with ASD and their families. Results provide initial information on parent perceptions of these novel telemedicine tools used to guide parents through administration of ASD-screening activities with their toddlers. Overall, parents rated tele-screening to be acceptable and comfortable. Parents’ qualitative feedback offered important insight into potential modifications to tele-screening procedures. Some parents also described limitations of tele-screening, highlighting the importance of creating procedures that capture parent concerns and child behavior as comprehensively as possible. Together, these parent comments provide important guidance regarding the development and modification of tele-assessment processes.

A preliminary question in considering tele-assessment of children with concerns for ASD has been whether parents find this approach to ASD identification to be acceptable, comfortable, and useful. Results of the present study indicate that the majority of parents found the clinician’s instructions to be easy to follow, understood what they were asked to do, and felt comfortable playing with their child as part of the screening. Qualitatively, parents provided positive feedback on many aspects of the tele-assessment process. Several parents appreciated the parent-led nature of the tele-assessment, noting that their children may feel more comfortable playing with a parent than an unfamiliar clinician. Some parents reported satisfaction with clinician communication during the assessment, particularly when clinicians asked questions to better understand the child’s behavior or to clarify how the child was interacting with his or her parent. This positive feedback from parents is in keeping with past work on parents’ responses to being involved in providing intervention to their children (McConachie and Diggle 2007; Stahmer et al. 2017). This is also important information for clinicians interacting with families during both in-person and telemedicine-based assessments, as there has been limited research on parents’ perceptions of their own involvement in the diagnostic evaluation process (Head and Abbeduto 2007).

Parents also provided feedback highlighting areas in which tele-assessment procedures and technology can be improved. Many parent comments addressed difficulty hearing or seeing the clinician, concern about what the clinician could and could not see, and problems with internet connections. These comments highlight the challenges inherent in telemedicine. Existing research and guidelines for telepractice stress the barriers associated with technological challenges and emphasize the importance of technical support (Boisvert et al. 2010; Joint Task Force for the Development of Telepsychology Guidelines for Psychologists 2013). Future work should continue to gather iterative feedback from stakeholder parents that will inform the development of tools and resources delivered via telehealth.

Within this work, feedback allowed us to take practical steps to address several parent concerns in the next phase of work. These include changes to the technological set-up, such as using a larger video screen, providing a portable speaker that parents can hold next to them, and using a wired internet connection rather than relying on a wireless network. It also included several adjustments to the tele-evaluation rooms and materials. Specifically, child-friendly art and noise-canceling panels were added to walls, a larger wall-mounted monitor was utilized, audio issues were resolved, and materials used in assessment activities were placed into labeled boxes for ease of access.

In addition to feedback on technology challenges, some parents commented on potential limitations of the tele-screening process. A minority of parents reported concern that the tele-screening tools did not allow clinicians to observe the full range of children’s behavior, including some behaviors about which parents were most concerned. Others expressed concern that the brief nature of the screening may result in their child’s vulnerabilities being overlooked. Finally, some parents provided comments referencing the value of face-to-face evaluation with an experienced clinician. Some noted that they found it valuable to watch someone else work with their child. Others indicated that perhaps tele-screening may be a first step in a process that ultimately includes in-person diagnostic evaluation. Further, when all parent participants were asked whether they preferred to play with their child, observe a clinician playing with their child, or both, most parents indicated that they preferred to both play and observe. This feedback suggests that while parents appreciated the opportunity to be directly involved in their child’s assessment, some families may continue to prefer traditional, in-person visits with clinicians, either in addition to or instead of tele-screening.

Parents’ concerns highlight a key limitation of any brief screening or assessment procedure, which provide clinicians only a glimpse of a child’s behavior and may or may not be representative of that child’s behavior in other settings. For this reason, best clinical practice in ASD evaluation has long stressed the importance of caregiver interviewing combined with observation of child behavior (Huerta and Lord 2012). In subsequent phases of this ongoing clinical trial, remote assessors supplement use of the TELE-STAT or TELE-ASD-PEDS with a clinical interview to obtain developmental history, parents’ current concerns, and presence or absence of ASD-related symptoms in other settings. Given the stress frequently associated with the diagnostic process for parents (Crane et al. 2016), any novel assessment modality must be responsive to parents’ concerns. For tele-assessment of ASD to be an acceptable and scalable clinical model, it is essential that parents and clinicians view the process as comprehensive and capable of capturing the child behaviors about which parents are specifically concerned.

Though tele-assessment presents a promising avenue for delivering services to children and families in a convenient and efficient manner, not all families will prefer or be appropriate for tele-assessment. A tele-assessment model of ASD identification is meant to serve families whose needs can be met with a distance consultation and assessment approach. Stainbrook et al. (2019) described a set of decision rules used to determine which families would be most successfully served by a telemedicine assessment. Children considered most appropriate for this model were those between the ages of 18 and 33 months without significant medical complexity, who were not yet speaking in sentences, and for whom specific concern for ASD had been raised by parents, caregivers, or medical or early intervention providers. Children with higher levels of diagnostic complexity, medical comorbidities, or more advanced language, as well as families who preferred to travel for in-person assessment, were referred for comprehensive evaluations (Stainbrook et al. 2019). This type of triage approach may help to decrease wait times both for families who can access telemedicine assessment as well as those who prefer or likely require an in-person or lengthier evaluation. This approach also allows families an opportunity to choose which type of care they receive.

This study has several key limitations that warrant further investigation. First, the present study has a relatively small sample size and an unequal distribution of children across diagnostic status. Participants in this study were from families who spoke English as a primary language, limiting our ability to understand how tele-screening may operate for non-English speaking families. Our sample is also made up of families who elected to participate in research on telehealth. As such, it is possible that these families already have positive views of telemedicine as a healthcare modality. As telehealth becomes increasingly common throughout medical and psychological fields, it will be important for future research to understand how patients who opt for telemedicine visits are different from those who do not.

An additional limitation is that psychometric properties are not yet available on the TELE-STAT and TELE-ASD-PEDS, limiting our ability to comment on their ultimate utility. Ultimately, understanding diagnostic agreement in relation to characteristics of family and child is of critical importance, but beyond scope of the present study. The first phase of this clinical trial was focused on acceptability of these tools and sought parent feedback for the purpose of optimizing tele-screening procedures, laying the groundwork for further rigorous evaluation of these procedures. It is promising to note that initial diagnostic agreement in this study was 86%, similar to diagnostic agreement reported in other research on tele-assessment of ASD (Juárez et al. 2018; Reese et al. 2015). Further analysis of the diagnostic accuracy of tele-assessment procedures that combine parent-led activities with clinical interviewing is ongoing and will be submitted for publication separately. Though acceptability and parent feedback are important first steps in developing tele-screening procedures, it will be important to understand how and for whom these tools can be most useful.

The current study presents several fruitful directions for ongoing research. Within our ongoing clinical trial, immediate future directions include investigation of the psychometric properties of the TELE-STAT and TELE-ASD-PEDS. We intend to examine child and family factors impacting diagnostic accuracy and clinician diagnostic certainty. We also intend to investigate provider and caregiver perceptions of tele-assessment for ASD within a community sample, particularly in the context of COVID-19 as many providers explore telemedicine-based models of care. As telemedicine services expand, it will be important to continue seeking and responding to stakeholder feedback in collaborative, family-centered ways. In addition, it will be essential to understand for whom and under what circumstances telemedicine is most appropriate and effective. Continuing to assess factors impacting family preference for telemedicine versus in-person visits, as well as understanding barriers to telemedicine, will allow for the creation of models that best meet patients’ and families’ needs.

In conclusion, parent-led, clinician-guided ASD screening represents a promising approach to the use of telemedicine in early ASD assessment. While prior approaches to tele-assessment have relied on multiple expert providers or detailed coaching sessions with parents, the TELE-STAT and TELE-ASD-PEDS allow a remote clinician with expertise in ASD to guide caregivers through a set of activities with their toddlers in real time, creating opportunities to observe toddlers’ independent play, response to social bids, and interaction with caregivers. The feedback provided by parents in the current study both highlights the preliminary acceptability of these procedures, as well as presents key opportunities for further optimizing these procedures to best meet families’ needs. Ultimately, leveraging telemedicine for the purpose of ASD assessment may address barriers families face in accessing early assessment and diagnosis of ASD, including geographic barriers, long wait times, and limited availability of expert providers to complete diagnostic evaluations. In turn, this increased access can facilitate early access to interventions, resources, and family support, promoting long-term developmental outcomes and family functioning.