Characterizing Available Tools for Synchronous Virtual Assessment of Toddlers with Suspected Autism Spectrum Disorder: A Brief Report.

The COVID-19 pandemic, and associated social distancing mandates, has placed significant limitations on in-person health services, requiring creative solutions for supporting clinicians engaged in the diagnosis of autism spectrum disorder (ASD). This report describes the five virtual instruments available at the time of manuscript development for use by experienced clinicians making diagnostic determinations of ASD for toddlers across the 12- to 36-months age range. We focus on synchronous virtual assessments in which clinicians guide the child's caregiver through a range of assessment activities and observe spontaneous and elicited behaviors. Assessments are compared on dimensions of targeted behavioral domains, specific activities and presses employed, scoring approaches, and other key logistical considerations to guide instrument selection for use in varied clinical and research contexts.

For many children with autism spectrum disorder (ASD), access to ASD-specialized early intervention services is predicated on a formal diagnosis of ASD. Best practices in ASD diagnosis involve integration of data from caregiver report of developmental history and current symptoms, standardized assessments of cognitive, language, and adaptive functioning, and behavioral observations of ASD symptomology (Broder Fingert et al. 2019; Huerta and Lord 2012; Zwaigenbaum et al. 2015). While standardized diagnostic instruments, such as the Autism Diagnostic Observation Schedule, Second Edition (ADOS-2; Lord et al. 2012) are available to support elicitation and observation of ASD symptoms, they were designed for use in face-to-face, distraction-free settings with standardized objects/toys. These assessments take place within an interpersonal context, throughout which the examiner sets up situations for eliciting and observing reciprocal social interaction, communication, play, and restricted and repetitive behaviors.

The coronavirus (COVID-19) global pandemic and social distancing measures imposed to curtail its spread have created unprecedented challenges to conducting ASD diagnostic evaluations, and many clinics have paused or reduced in-person services indefinitely. Given that COVID-19 has disproportionately affected minority groups (Tai et al. 2020), the pandemic has exacerbated the pre-existing inequities in health-care service access that exist in our field (Bishop-Fitzpatrick and Kind 2017; Iacono et al. 2016). Moreover, the current safety regulations for using personal protective equipment (PPE), especially the use of face masks, represent not only a departure from standardized procedures, but also a barrier to establishing the interpersonal context that is necessary for evaluating certain ASD symptoms. As such, COVID-19 has served as a massive disrupter to traditional service delivery, necessitating immediate large-scale adoption of innovative approaches to care delivery, including virtual ASD diagnostic services. Given that the gold standard approach for diagnosing ASD emphasizes a best-estimate clinical judgment approach, rather than reliance on a single or established group of measures (Broder Fingert et al. 2019; Huerta and Lord 2012; Zwaigenbaum et al. 2015), it is feasible to consider tools that support observation of ASD symptomatology using virtual assessment.

Beyond offering an alternative for assessment during the COVID-19 pandemic, virtual assessment has the potential to address the limited availability of and access to professionals who provide diagnostic and clinical services for children with ASD living in rural areas (Carbone et al. 2010; Drahota et al. 2020). Compared to those in urban or suburban areas, children living in rural and underserved areas are diagnosed at a later age (Lauritsen et al. 2014) and have decreased access to health care services (Mandell et al. 2005). Further, children from households with lower socioeconomic status and those from disadvantaged neighborhoods are also less likely to receive a timely ASD diagnosis (Durkin et al. 2010; Mazumdar et al. 2013). Creative solutions for conducting remote diagnosis will not only ensure continued access to care for children and families during COVID-19, but also have the potential to fundamentally shift the service delivery landscape by addressing the significant disparities in access to early diagnostic and intervention services in the United States as well as globally.

In recent years, efforts to increase access to ASD evaluation services for families living in rural or underserved communities (i.e., those facing the greatest challenges obtaining an in-person ASD diagnostic assessment; Drahota et al. 2020) have led to a gradual increase in the development of protocols for remote diagnosis. However, prior to the COVID-19 pandemic, the development of remote diagnostic tools had been slow-going, with limited empirical validation of these measures. Indeed, a recent systematic review found that only seven studies have examined the use of video- and web-based applications for supporting ASD assessment (Dahiya et al. 2020). In addition, most studies used asynchronous, or “store-and-forward,” methods of virtual assessment, that require the caregiver to record video of the child for later review by a trained clinician. This approach has several advantages, such as enabling sampling of child behavior across multiple days, eliminating the need to coordinate schedules with a clinician, the ease of uploading videos over time rather than relying on sufficient bandwidth for live video conferencing, and the potential for automated scoring via artificial intelligence. However, the utility of asynchronous virtual assessment during COVID-19 is somewhat limited. Asynchronous virtual assessment has high technology demands (e.g., the need to record, upload and store videos securely), with some methods requiring specialized software (e.g., Cognoa tool, Abbas et al. 2020) that is not widely available. Further, asynchronous methods are not as readily implementable due to the need for protocol and platform specific training and comparatively high start-up/maintenance costs (e.g., pay per use requirement of NODA, Smith et al. 2017). Importantly, asynchronous virtual assessment also does not allow for adjustment of the number or types of presses administered based on clinical judgment of responses and behaviors of the individual child. As a result, key behaviors (or lack thereof) may not be observed.

In addition to virtual diagnostic protocols, research has also begun to examine the utility of virtual screening protocols to assess children for potential ASD (e.g., Telehealth Evaluation of Development for Infants (TEDI; Talbott et al. 2020)), and several groups have adapted observational screening measures for telehealth during COVID-19 (e.g., Systematic Observation of Red Flags (SORF; Dow et al. 2020), Autism Detection in Early Childhood (ADEC; Nah et al. 2014), and Screening Tool for Autism in Toddlers (STAT; Corona et al. 2020b)). However, these tools are designed to be Level 2 screening measures and thus their virtual adaptation does not fully address the current need for methods of diagnostic assessment that are immediately implementable in community and research settings.

Diagnostic assessment using synchronous virtual methods is one way to directly address the immediate, practical needs of those working with young children with ASD. Synchronous virtual diagnostic assessment, in which a clinician and caregiver/child dyad interact face-to-face via videoconferencing software, utilizes basic technology that has become familiar for many families and healthcare providers due to work and school accommodations secondary to COVID-19. Additionally, no assessment-specific training is required for those who are trained to conduct diagnostic evaluations in person, as synchronous virtual assessment is viewed as a direct alternative to in-person service delivery. Fortunately, many of the developers of synchronous virtual assessments during COVID-19 have distributed tools to support dissemination of the instruments and their application. While additional research establishing the validity of these tools is imperative, there is an immediate need to identify practical options for remote clinical and research assessment. Yet, it has been difficult for many users to systematically evaluate which virtual assessment tool best fits their priorities and contexts because there is a lack of overarching information to aid in understanding and comparing the similarities and differences between these instruments.

This brief report provides an overview of the synchronous virtual diagnostic tools available to support experienced clinicians in conducting in-the-moment behavioral assessments for toddlers referred for ASD evaluation when the use of well-established in-person diagnostic tools and processes is not possible or feasible. We focus on tools designed for use with toddlers (12- to 36-month age range) to support a clinical diagnosis of ASD. This age range was prioritized given the well-documented benefits of early identification for long term child outcomes. All tools included in this report are conducted via observation of the child during interactions with their caregiver and involve synchronous communication with a trained clinician. In all cases, the clinician directs and/or coaches caregivers to conduct specific activities that press for social communication behaviors that correspond to DSM-5 diagnostic criteria. Asynchronous methods of virtual evaluation are excluded from this review given the aforementioned limitations to their immediate use for the majority of families, clinicians, and researchers.

It is important to note that at the time of writing, none of the tools meeting the above criteria, and subsequently discussed below, have undergone rigorous psychometric evaluation, although all include items or activities that are similar to those in diagnostic tools with established psychometric properties. Our aim, therefore, is to provide comparative information on the dimensions of target behavioral domains, specific activities and presses used, scoring approaches, and other key logistical considerations for assisting clinicians and researchers in selecting the most appropriate tool for their goals, settings, and populations, while the evidence base for these tools is being examined.

Overview of Measures

The five remote assessment tools included in this report were identified via solicitations from professional organization listservs (e.g., American Psychological Association Division 33 Intellectual & Developmental Disabilities/Autism Spectrum Disorder), communication from professional organizations, webinar announcements, personal communications, and a review of the literature. We recognize that this review may not be exhaustive of all synchronous virtual assessment tools that have been developed during COVID-19, but every effort was made to be as comprehensive as possible. Tools were selected for inclusion if they were appropriate for use with children across the 12- to 36-months age range (as indicated by the test developers, or as indicated by the target language level specified for the assessment) and involved direct observation of child behavior and synchronous communication between a caregiver and expert clinician. Importantly, all tools are designed to be used as one part of a comprehensive diagnostic assessment for ASD—none are intended to be the sole source of information for rendering a diagnosis. The developers of these tools were extremely generous in sharing their most recent materials with us to ensure the accuracy of the information at the time of writing. However, it is important to acknowledge that these assessments are still early in the validation process, and are thus subject to change. See Tables 1 and 2 for detailed information about each tool, which are presented in alphabetical order.

Table 1

Observed behaviors, DSM-5 diagnostic criteria and corresponding activity

Observed child behavior	DSM-5 criteria	ASD-DIAL for young children (pre-verbal/early language learner)	A-VABO	BOSA-MV	OOPS:HE	TELE-ASD-PEDS
Verbal and nonverbal communication
Type of expressive language (e.g., makes sounds, uses single words, phrase speech)	N/A	–	If no words observed, parent prompted to ask simple questions	–	–	–
Uses a variety of conventional gestures (e.g., pointing, waving, nodding head)	A2	–	Parent sets up interaction to encourage gesture use (e.g., says “bye bye”)	–	–	–
Social attention and interaction
Social attention
Initiates interactions (e.g., invites parent to join back and forth play, shows items)	A1	Parent safely ignores child during free play with toys	–	–	–	Parent safely ignores child during free play with toys
Requests object or action from an adult	A1	Parent shows snack or other desirable item	–	Parent demonstrates bubbles	Parent demonstrates bubbles OR shows snack	Parent shows snack or other desirable item
Social response
Responds to adult’s social bids during familiar routines (e.g., peeks back during peek-a-boo)	A1	Parent says familiar words, sings song or makes loud noise; parent initiates peek-a-boo or tickling	Parent initiates peek-a-boo	–	Parent initiates peek-a-boo	Parent initiates peek-a-boo and then tickling (if no response to peek-a-boo)
Responds to adult’s attempts to direct attention to an object or event (e.g., turns head to follow a point)	A1	Parent points to something behind the child	Parent turns on noise-making toy; Parent directs child’s attention with verbalization (e.g., “oh look!”)	–	Parent talks about, looks at and then points to something out of reach	Parent points to something across the room
Responds to name being called	A1	Parent calls child's name	Parent calls child's name	–	Parent calls child’s name and if no response, then touches child’s shoulder before calling name again	Parent calls child's name
Responds to adult's requests (e.g., gives requested object)	A1	–	Parent gives gestural command (extends arms as if to pick the child up); parent gives verbal command (e.g., blow kiss, clap)	–	–	–
Social turn-taking/back and forth
Engages in reciprocal interaction during unstructured play	A1	Parent attempts to join child’s play*	–	Parent encourages the child to play with assessment toys*	Parent encourages the child to play with assessment toys*	Parent attempts to join child’s play*
Engages in reciprocal interaction during familiar routines with objects	A1	Parent initiates familiar and then less familiar social game (e.g., ready-set-go routine, rolling/tossing ball back and forth)	–	–	Parent initiates rolling/tossing ball back and forth; parent points at interesting pictures in a book	Parent initiates rolling/tossing a toy (e.g., ball, car) back and forth
Affective expression
Looks at adult to share change in emotional experiences (e.g., sharing pleasure)	A1	Parent elicits child's attention and smiles; parent sings favorite song and smiles	Parent elicits child's attention and smiles	–	–	–
Play and imitation
Use of toys
Demonstrates flexible play with toys during activities commensurate with skill level (e.g., exploratory, functional, or symbolic)	A3	Free play: Parent is instructed to respond naturally*	–	–	Free play: Parent is instructed to respond naturally and encourage child to play with toys (excluding favorite toys)*	Free play: Parent is instructed to respond naturally and given explicit instruction not to direct the child*
Imitates gestures, actions, or functional play with objects	N/A	Beginning level: Parent models gesture (e.g., waving), then models actions with toy (e.g., rolling car) Advanced level (> 18 month skill level): Parent models actions with household objects (e.g., brush hair with comb) and verbal instruction, then models new actions with household objects (e.g., the comb is a train) and verbal instruction	Parent gives child toy phone; parent models motor movement (e.g., drumming on table) and provides verbal instruction	–	Parent models gestures (e.g., shaking head, waving, high 5′s) while encouraging imitation, and if no response parent initiates a familiar imitation game	–
Engages in spontaneous or imitated symbolic play during structured activity	A3	Parent initiates pretend play routine with baby doll/animals/action figures, then encourages child to play, then introduces a new play idea	Parent models symbolic play (e.g., block as a phone) then offers to child	–	Parent initiates pretend play routine with item of interest by showing; if no response, parent models play action and provides verbal instruction	–
Restricted/repetitive behaviors or interests
Shows repetitive/stereotyped play with objects (e.g., spinning wheels, lining up toys, turning over objects)	B1, B3	–	Parent lines up blocks and then messes them up	–	Parent encourages child to play with favorite toys*	–
Shows behavioral rigidities/insistence on sameness (e.g., difficulties with transitions)	B2	–	–	Parent cleans up toys before presenting bubble activity, and then cleans up bubbles before presenting next set of toys*	–	–
Shows unusual or intense sensory responses/sensitivities and/or interests (e.g., unusual visual examination of objects, licking objects)	B4	Parent brings out items child is unusually drawn to and items child is unusually afraid of (clinician ensures this is okay and will not be too disruptive)*	–	–	–	–

*No specific press, but the parent establishes a specific context for observing the target behavior

– No specific press, but behaviors may be observed throughout assessment

Table 2

Administration and scoring logistics

	ASD-DIAL	A-VABO	BOSA-MV	OOPS: HE	TELE-ASD-PEDS
Provides written instructions to prepare families	No	Yes (reading level: grade 8.1)	Yes (reading level: grade 3.3)	Yes (reading level: grade 6.3)	Yes (reading level: grade 8.5)
Considered one component of a comprehensive ASD assessment (Y/N)	Yes	Yes	Yes	Yes	Yes
Includes specific scripts for administration (Y/N)	Yes	Yes	Yes	Yes	Yes
Estimated administration duration*	30–60 min	Unspecified	12 min	25 min	10–20 min
Can use family’s toys/objects	Yes	Yes	No	Yes	Yes
Number of toys	~ 15 toys and snack container	~ 7 toys	Two sets of provided toys and bubbles	~ 15 toys and snack container	3–5 toys and clear container (for snack or another desirable object)
Additional costs	N/A	N/A	Scoring forms: ADOS-2 Protocols Materials kit: ADOS-2 Toys	N/A	N/A
Activity order	Flexible	Unspecified	Prescribed	Prescribed	Prescribed
Provides specific presses (Y/N)	Yes	Yes	No	Yes	Yes
Duration of activities	Flexible	Prescribed	Prescribed	Prescribed	Mixed, prescribed and flexible
Number of separate activities^	10	15	3	10	8
Item-level scoring type	No	Yes, quantitative	Yes, dichotomous	Yes, dichotomous	Yes, dichotomous or quantitative
Scoring system	DSM-5 Checklist	Independent	ADOS-2 Protocols and then DSM-5 Checklist	Independent	Independent
Score interpretation type	Symptom level absence vs. presence	Unspecified	Symptom level absence vs. presence	Continuous	Continuous with cut-point
Published data on psychometrics	None	None	None	None	None

*Estimated administration duration does not include set-up and family orientation time

^If an activity (e.g., bubbles) is repeated at different times throughout the assessment, it is counted as a single activity

ASD-DIAL: Autism Spectrum Disorder—Diagnostic Interview and Activities—Lifespan, Version 2

Description

The ASD-DIAL (Miller 2020) was developed at the Center for Autism Research at the Children’s Hospital of Philadelphia in response to COVID-19 and social distancing measures. This measure includes activities designed by ASD experts that have clinical utility for eliciting behaviors that map onto relevant ASD constructs. The Activities for Young Children module of the ASD-DIAL was designed for young children regardless of verbal ability.

Activities

There are 10 possible presses for young children; these presses assess social responsiveness, turn-taking, requesting, imitation, and play. The clinician selects the specific presses, number of presses, and order of presses to administer based on the child’s evaluation needs. Most administrations last between 30 and 60 min. The manual includes scripted instructions for the clinician to help guide caregivers through each activity, as well as notes and suggestions for administration. The ASD-DIAL uses materials that are likely already available in many homes, such as blocks, cars, and bubbles.

Scoring

Observations from the ASD-DIAL are used to complete a DSM-5 checklist regarding the absence or presence of symptoms; no independent item-level scoring system is provided. The level of clinician diagnostic certainty is also noted. There is a strong recommendation to use the ASD-DIAL in conjunction with other sources of information (e.g., caregiver interview, review of records, Childhood Autism Rating Scale, Second Edition (CARS-2; Schopler et al. 2010)) to inform a clinical diagnosis of ASD.

A-VABO: Adapted Virtual Autism Behavior Observation

The A-VABO (Kryszak and Albright 2020) was developed at Nationwide Children’s Hospital and is designed to be used as one element of a multi-phase interdisciplinary virtual assessment protocol for ASD.

The A-VABO toddler form consists of 15 presses, which include responding to social bids, play and imitation, communication acts, and directly eliciting repetitive/stereotyped behavior. Each press has a permitted duration and number of trials as well as specific instructions for the materials needed, procedures to follow, and a script to guide caregivers through the activities. Some of the activities require specific toys and objects. However, during a pre-visit phase of the evaluation, the caregiver is asked to gather the materials from around the home (reading level of written instructions: grade 8.1), and if the caregiver is missing any of the materials, there is a list of possible item alternatives.

The A-VABO uses item-level scoring in which the child’s response to each press is scored as either 0, 1, or 2; each score is operationally defined, with a score of 2 representing the most appropriate response. It is recommended that a consensus diagnostic decision be based on the integration of results from the A-VABO with other sources of information (e.g., speech assessment, clinical and medical interviews, the Developmental Profile-3 interview (Alpern et al. 2007), CARS-2 (Schopler et al. 2010)).

BOSA: Brief Observation of Symptoms of Autism (BOSA)

The BOSA (Lord et al. 2020) was developed by Dr. Cathy Lord and her team at the UCLA CART Lab, in partnership with the UCLA Semel Institute. The BOSA was developed for administration either via telehealth or in person (with the clinician masked and maintaining social distance, but caregiver and child unmasked). This tool is an adaptation of the ADOS-2 and the Brief Observation of Social Communication Change (BOSCC; Grzadzinski et al. 2016) and is copyrighted by Western Psychological Services (WPS); permission from UCLA is required before the BOSA may be used. The BOSA-Minimally Verbal (BOSA-MV), described here, can be used for ambulatory children of any age. The BOSA can be administered and scored with children and caregivers who do not speak English, provided that the clinician is proficient in the target language and there is a valid translation of the ADOS-2 protocol in that language. While independent translation of the BOSA is not permitted, the BOSA team is currently working with WPS to create a Spanish translation.

The BOSA-MV takes 12 min and involves a standard sequence of activities: free-play and clean up with an initial toy set; blowing bubbles; free-play and clean up with a second toy set; blowing bubbles. The activities are not accompanied by explicit prompts or presses; rather, the clinician encourages the caregiver to engage their child with at least two toys from each toy set. In addition, caregivers receive an instruction sheet (reading level: grade 3.3) that provides the time limits, order, and descriptions of the activities, including suggestions for how to play with the toys included in each of the two toy sets. The toys required for the BOSA-MV are gathered from an ADOS-2 kit.

Clinicians who use the BOSA should have experience administering the ADOS-2 in either clinical or research settings, as the scoring codes from ADOS-2 modules are used. The BOSA score is derived by rating as many codes as possible for select items across different ADOS-2 module scoring protocols; as such, the ADOS-2 training, kits, and scoring protocols may incur additional costs for clinicians. ADOS-2 item scores are converted to binary BOSA scores (present or absent) on the BOSA-MV Toddler Module DSM-5 Checklist. Clinicians can make scoring decisions based on other incidental observations and caregiver report. It is recommended that the BOSA be used in conjunction with additional measures such as the Autism Diagnostic Interview-Revised (Lord et al. 1994), CARS-2 (Schopler et al. 2010), adaptive functioning measures, and cognitive testing to inform diagnostic impressions. When the BOSA is used for research purposes, the authors encourage groups to establish reliability among the research team.

OOPS: HE: Observation of Play Screener: Home Edition

The OOPS:HE (Nickel 2020) was developed by Dr. Robert Nickel at Oregon Health & Science University. The OOPS: HE was adapted from the original Observation of Play Screener (OOPS, Nickel 1997, revised 2001) which was created as a training tool for pediatricians on the characteristic behaviors of young children with ASD.

The OOPS: HE takes approximately 30 min and consists of 12 presses across 10 activities designed to observe the child’s social responses, turn-taking, requesting, imitation, and play. The clinician uses scripted instructions to guide the caregiver through each press. Some presses are flexible in duration, while others have a specific time limit and/or number of prompts. Caregivers receive an instruction sheet (reading level: grade 6.3) that outlines the recommended toys, suggested preparations (e.g., tips for camera placement), and overview of the OOPS: HE activities. Before the visit, the caregiver is asked to gather the recommended toys from around the home.

The OOPS: HE uses item-level scoring in which operationally defined target behaviors are scored as present or absent. A total score is generated by summing the number of items mapping onto social communication and social emotional reciprocity that were absent and the number of items mapping onto restricted and repetitive behaviors that were present. The clinician also records general qualitative observations of the child’s behavior. The OOPS: HE is meant to be used in conjunction with a caregiver interview and uploaded videos of household routines (e.g., snack, getting dressed) to supplement observation during virtual evaluations.

TELE-ASD-PEDS

The TELE-ASD-PEDS (Corona et al. 2020a) was developed by the TRIAD Team at Vanderbilt University Medical Center. This tool was developed prior to the COVID-19 pandemic to support caregiver-mediated remote assessment of ASD in children up to 36 months who do not yet have phrase speech. Items were selected by applying machine learning technology on a database of several hundred individuals with and without ASD. The items with the most predictive utility for young children were selected and transformed into interactive, caregiver-led tasks that comprise the TELE-ASD-PEDS. While there are no published data on the psychometric properties of this assessment, a clinical trial is currently underway to investigate its accuracy and limits. As part of this larger clinical trial, initial studies have found the TELE-ASD-PEDS to be acceptable, feasible, and comfortable for caregivers and providers, and preliminary estimates of diagnostic accuracy are promising (Corona et al. 2020b; Wagner et al. 2020).

The TELE-ASD-PEDS takes about 20 min to administer and consists of 10 presses across eight activities, which include those related to social responses, turn-taking, requesting, and play. The clinician uses a script to scaffold a caregiver through the tasks with their child. Caregivers receive an instruction sheet (reading level: grade 8.5) providing an overview of the assessment and instructions for gathering materials for the assessment from their home.

The TELE-ASD-PEDS rating form is used to code the observation of specific behaviors; symptoms are scored either dichotomously (e.g., present or absent), or using a three-point Likert scale (e.g., symptom obviously consistent with ASD, present at subclinical levels, or not present). A total score is computed by summing across all items, and raters are asked to indicate their level of diagnostic certainty. The TELE-ASD-PEDS is intended to be used as one component of a comprehensive ASD virtual assessment.

Discussion

The COVID-19 pandemic, and associated social distancing mandates, have placed significant limitations on our ability to conduct standardized in-person ASD assessments, and have generated the need to re-evaluate our current strategies for diagnosing ASD. To date we have identified five measures designed for synchronous virtual interactions between a clinician and a caregiver of a toddler at risk for ASD, all of which involve the clinician guiding the caregiver, via telehealth, to engage in a series of activities in the home setting. It is critical to acknowledge that this is a rapidly evolving field, and it is very likely that the measures described will continue to be modified as empirical data become available. While no published psychometric data were available for any of the measures at the time of writing, we hope that the summaries offered in this report will support clinical decision-making related to the selection and use of a virtual observational diagnostic assessment for individual service settings.

The five measures share important commonalities in terms of the behavioral domains assessed, the activities and materials used, and the supports offered to guide clinicians and families through the remote administration process. Yet, there are also differences across measures that may influence adoption decisions. For instance, administration times can range from 12 to 60 min, the number of toys required can range from 5 to over 15, and the extent to which families are likely to have and/or can use everyday items in their homes varies. In addition, there are significant differences in cost (e.g., the BOSA-MV requires toys and protocols from an ADOS-2 kit). The assessment approaches also differ in terms of the extent to which target behaviors are elicited directly or observed passively. For example, the A-VABO includes a prescribed press for repetitive/stereotyped behaviors, the OOPS: HE encourages the caregiver to establish a context for eliciting repetitive/stereotyped behaviors, and the other three measures simply encourage the clinician to observe and note the occurrence of such behaviors as they arise naturally across activities. Additional differences in materials, presses, administration, and scoring conventions and interpretation should be considered when deciding when, how, and for what purpose to deploy any of these measures. Overall, our hope is that this review provides relevant information to enable clinicians and researchers to make informed decisions about the optimal assessment for their unique situations as long as COVID-19 precautions and/or other factors prevent the use of traditional face-to-face ASD assessments.

While virtual assessment of ASD will ideally serve to increase access to diagnostic services for traditionally underserved populations, there remains a subset of individuals for whom telehealth will not be feasible. Telehealth requires a reliable internet connection that can support video conferencing, and also requires that families have appropriate equipment for engaging in the virtual visit. Therefore, families living in geographic areas without broadband connection, or who lack the requisite hardware and software, will be unable to participate in a virtual assessment unless these technologies are provided. Further, four of the five assessments we reviewed require that the family use their own toys for the assessment, which cannot be assumed to be universally available in all homes, and thus may provide another obstacle. Consequently, families who live in rural areas or are economically disadvantaged (i.e., families who already have disproportionately limited access to health services), are poised to have their access to ASD evaluations even more restricted if telehealth is the sole option. Another consideration is that there is the potential for under-resourced families to receive less valid assessments if eventual psychometric data is based primarily on families who have the requisite toys and technologies. Thus, while remote assessment will broaden access to services for many, there is a subset of families who will need the health service sector to provide the needed materials (e.g., Wi-Fi hotspot, equipment, toys) and/or conduct in-person diagnostic evaluations. Every effort should be made to meet this need as safely as possible throughout the COVID-19 pandemic, and we hope that as virtual assessments become more advanced and refined, these factors will be considered.

Ultimately, the pandemic has propelled the ASD clinical and research communities from slow, incremental change with respect to virtual assessments to a resounding and urgent need for radical innovation. While validation studies on the available measures need to move forward, it is critical to acknowledge that we are merely in the early stages of the innovation “lifecycle” (Brown and Katz 2009; Tschimmel 2012) for the design, development, deployment, and sustained adoption of virtual assessments for ASD. Progression through the lifecycle should be guided and informed by tracking clinicians’ and families’ experiences in navigating the logistics of the virtual assessment and diagnostic decision-making process, and characterizing the profiles of families for whom the virtual assessments prove to be a "good fit" (e.g., acceptable, feasible, desirable). As a community, we must scope the needs, challenges, and pain points for families and clinicians and then use these data to modify and/or create tools and approaches that ensure more individualized, equitable, and sustainable individualized service delivery. This “re-set” in assessment development provides the unique opportunity to embed culturally sensitive items and algorithmic parity for diverse and underserved groups into the early validation process, rather than after measures have been widely adopted. In addition, the field should be poised to capitalize on rapid improvements in software (e.g., Zoom) and hardware (e.g., cameras) to address needs such as encryption, ease of use, and performance optimization for weak Wi-Fi.

Even when the widespread, frequent use of virtual assessments for ASD subsides as the effects of the pandemic are mitigated, the current moment can be considered a critical period of rapid innovation for virtual assessment tools. This is the case not only for methods of synchronous virtual diagnostic assessment such as those reviewed herein, but also for asynchronous virtual assessment (e.g., Cognoa tool (Abbas et al. 2020), NODA (Smith et al. 2017)) and virtual ASD screeners (e.g., SORF (Dow et al. 2020), TEDI (Talbott et al. 2020)). With collective action at the clinical, research, policy, and stakeholder levels, virtual assessments can outlast the COVID-19 pandemic and be integrated into diagnostic processes to support triage, enhance efficiency, ensure ecological validity, and address current health disparities by improving access to critical ASD diagnostic assessments for families in remote and underserved areas.