Reliability and validity of a novel caregiver-assessed skills system based on the ALSO conception in children with autism spectrum disorders.

BACKGROUND: In responding to the potential challenges in the transition from childhood to adulthood for individuals with autism spectrum disorders (ASD) in China, a novel conception of "ALSO" was proposed to bridge the transitional needs and early intervention. To facilitate the application of ALSO in early intervention, ALSOLIFE skills assessment system (ALSOLIFE Assessment) was developed to enable caregivers to evaluate their children's skills guided by the ALSO conception. Given that the critical shortage of qualified professionals in China, many caregivers of children with ASD must function as home therapists. To address the practical needs of Chinese families of children with ASD, ALSOLIFE Assessment is designed as a free, online, technology-assisted, self-operated and behavioral intervention approaches supported system. The assessment report then further served as the basis for caregivers to deliver the tailored educational intervention to their children. Although ALSOLIFE Assessment provides caregivers a home-based intervention program, it is still unclear whether its evaluation is reliable and accurate. Therefore, we conduct this study to investigate the reliability and validity of the ALSOLIFE Assessment.
METHODS: A total of 1,050 children with ASD (1 to 10 years old) were recruited from 31 provinces of mainland China. Their caregivers participated simultaneously as ALSOLIFE Assessment evaluators. The testing results of Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP) and Psychoeducational Profile-3 (PEP-3) were also collected for a portion of the child participants (VB-MAPP, N=34; PEP-3, N=31) to check criterion-related validity.
RESULTS: The reliability and validity of the ALSOLIFE Assessment satisfied psychometric requirements after the reduction from 511 to 464 items. Exploratory factor analysis (EFA) of ALSOLIFE Assessment scoring data yielded six factors, and confirmatory factor analysis (CFA) verified the best fitting construct model is a bifactorial model with one general factor and six group factors. Compared to the VB-MAPP and PEP-3, the ALSOLIFE Assessment exhibited good criterion-related validity across three levels: total scores, 6 skill domains, and 22 skill subdomains. The reliability tests indicated its strong internal consistency, test-retest reliability, and inter-rater reliability.
CONCLUSIONS: The reliability and validity of the revised ALSOLIFE Assessment (464-items) satisfied psychometric requirements. It is essential to develop validated and comprehensive evaluation tools, embedded on the needs of Chinese families of children with ASD. 2021 Translational Pediatrics. All rights reserved.

Introduction

On World Autism Awareness Day in 2020, the United Nations released the theme “The Transition to Adulthood”, and advocated the public attention to challenges or obstacles during the transition to adulthood for individuals with autism spectrum disorder (ASD) to achieve the goal of “nurturing children with ASD to be independent social citizens” (1). However, the transition to adulthood for many with ASD is not a quick and easy process (2,3). Much unsatisfactory outcomes of the transition to adulthood for individuals with ASD has been reported (4,5). In China, previous literature found that less than 24% of adults with ASD in Shanghai had proper occupational skills, engaged in recreational activities, took transportation, or went shopping independently (6). In responding to the potential challenges of ASD trajectory and facilitating their transition to adulthood, Doctor Yanqing Guo from the Peking University Sixth Hospital in China proposed an innovative intervention conception, “ALSO”, to bridge early intervention and transition service to adulthood for individuals with ASD (7).

The core of ALSO emphasizes that “for children with ASD, current interventions should be guided by future needs in adulthood, and future goals in adulthood must be practiced in current interventions.” Several studies have indicated that merely addressing transitional goals at adolescence yielded less productive results than starting this process at the early childhood stage (8-10). Considering the much-needed time and resources, the ALSO conception aims to pursue transitional outcomes by enhancing early intervention. That is, ALSO sets “Occupational and independent-living skills” (O) in young adulthood as the ultimate objective of education intervention for individuals with ASD, and proposes this future objective could be achieved via comprehensive assessments and tailored education intervention on “Academic and cognitive skills” (A), “Living and life skills” (L), and “Social interaction and social regulation skills” (S) for individuals with ASD, which stands for the conception of “ALSO” (7).

To facilitate the application of ALSO in early interventions for children with ASD, a team of behavior analysts, special educators, technicians, and other experts collaboratively developed ALSOLIFE platform, which includes two layers of free services: the ALSOLIFE Skills Assessment System (ALSOLIFE Assessment) intended to provide a remote assessment tool for caregivers of children with ASD to evaluate their children’s skills based on the ALSO conception; and an aligned ALSOLIFE Individualized Education Plan system (ALSOLIFE IEP), designed for caregivers to deliver tailored education to children with ASD based on the ALSOLIFE Assessment report (11).

It should be noted that many commonly used behavior and skill assessments for children with ASD in China were imported. For example, the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP), the Early Start Denver Model (ESDM), and the Psychoeducational Profile-3 (PEP-3) are all originated in the United States. They are generally adapted for professionals who have had years of discipline specific training (12). Given that the critical shortage of qualified professionals in China, many parents must function as both caregivers and home therapists (13). The actual situation is that the abovementioned assessments are usually too difficult for most Chinese caregivers of children with ASD to acquire professional qualifications on applying these assessments accurately and skillfully (14,15).

To address the practical needs of families of children with ASD under the status quo of ASD intervention in China, ALSOLIFE platform is envisioned to help families by offering the free, online, self-operated, technology-assisted, and behavioral intervention approaches supported systems guided by the ALSO conception. Quite different from other assessment systems for children with ASD, the operations of the ALSOLIFE Assessment are kept as user-friendly and practical as possible, free of jargons. Through the access of internet, families can conduct skill assessment at their convenience and obtain online training manual according to their child’s performance profile. Home-based therapy is made possible through this comprehensive and accessible system.

Although embedded on the needs of Chinese families of children with ASD, the validation and effectiveness of the ALSOLIFE tools is still unknown. Assessment is the basis for intervention, the psychometric evaluation of the ALSOLIFE Assessment would be an essential step to validate the ALSOLIFE systems. Therefore, this study aimed to comprehensively examining the reliability and validity of the ALSOLIFE Assessment. The findings of the present study will guide further scientific improvement of the ALSOLIFE Assessment and consequently benefit the families of children with ASD in China by providing a practical, validated, cultural fit and scientifically tested assessment tool based on the ALSO conception. We present the following article in accordance with the SURGE reporting checklist (available at http://dx.doi.org/10.21037/tp-20-319).

Methods

Ethical statement

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013) and approved by the ethics board of the Institute of Psychology of the Chinese Academy of Sciences (No. H19022). The number of participants was based on power analysis (16), which yielded an estimated sample size of 220 participants or more as being likely to provide sufficient power (16).

Participants

To maintain the ecological validity of the study, we directly recruited participants through the ALSOLIFE platform upon receiving the informed consent from the legal guardians. There were two types of recruited participants: client participants (children with ASD) and assessor participants (primary caregivers of the enrolled children with ASD). As client participants, the enrolled children aging from 1–10 years need be diagnosed with ASD by providing a diagnosis proof signed by a qualified pediatric psychiatrist. Children who had any additional diagnosis were also asked to document the comorbidity. As assessor participants, caregivers were required to be the primary caregivers who are familiar with the children’s daily behavior and have plenty of opportunities to interact with the children. The enrolled caregivers were also required a minimal education level of middle school and having access to the ALSOLIFE. As an incentive, we provided each assessor who completed the ALSOLIFE Assessment with a toy gift worthy ¥50 yuan.

Based on these criteria, we enrolled a total of 1,050 participating children with ASD from 31 provinces or municipalities in China. The children’s age ranged from 1.52 to 10.43 years (mean: 4.46±1.52), and the sex ratio was 5.40:1 (886 boys and 164 girls). Thirty-four children also reported comorbidity, including one with depression (0.1%), 14 with epilepsy (1.3%), and 19 with attention deficit hyperactivity disorder (ADHD) (1.8%). Amongst the 1,137 participating caregivers, on the other hand, 955 were mothers, 80 were fathers, and 15 were grandparents. There were 155 people with a middle school education (14.8%), 156 with a high school degree (14.9%), 614 with a bachelor’s degree (58.5%), and 125 with a master’s degree (11.9%). The more detailed demographics information could be seen in .

Table 1

Demographics of the 1,050 children with ASD in the study

Characteristics	Number (%)
Ages group (years old)
1–2	15 (1.4)
2–3	167 (15.9)
3–4	245 (23.3)
4–5	288 (27.4)
5–6	170 (16.2)
6–7	105 (10.0)
7–8	42 (4.0)
≥8	18 (1.7)
Sex
Male	886 (84.4)
Female	164 (15.6)
Comorbidity besides ASD
None	988 (94.1)
Child depression	1 (0.1)
Epilepsy	14 (1.3)
ADHD	19 (1.8)
Developmental delay	26 (2.4)
Other assessment situation
Accepted VB-MAPP assessment in one month	34 (3.2)
Accepted PEP-3 assessment in one month	31 (3.0)

‘Comorbidity besides ASD’ is a multiple-choice item, so the sum of the percentages beyond 100. ASD, Autism spectrum disorders; ADHD, Attention deficit hyperactivity disorder; VB-MAPP, Verbal Behavior Milestones Assessment and Placement Program; PEP-3, Psychoeducational Profile-3.

Instruments

ALSOLIFE Skills Assessment System (ALSOLIFE Assessment) Scope

ALSOLIFE Assessment is an individually administered measure of skills and behaviors of children with ASD functioning between the ages of 0 to 6 years old. It comprises six skill domains, including academic, cognitive, life, living, social interaction, and social regulation skills, each with a series of subdomains (see ). Targeting the basic learning abilities, the academic skill domain includes motor imitation, tact, mathematical concepts, and speaking and writing skills. The cognitive skill domain assesses children’s cognitive functioning, including receptive labeling, sensory perception, and sample matching. The life skill domain evaluates basic life-care abilities, such as dressing, self-feeding, daily chores, and personal hygiene. The living skill domain refers to leisure related skills, such as independent game playing, painting and handcrafting skills, and exercise capacity. While the social regulation skill domain examines rule-governed behaviors, such as classroom behaviors, emotional management skills, and community life skills, the social interaction skills emphasizes the verbal communication, including interactive language, understanding social games, demonstrating prosocial behaviors, following directions, and mand. With a total of 511 assessment items across 22 subdomains, ALSOLIFE Assessment offered five difficulty levels, 2 to 6, each associated with a developmental age. For readers’ more intuitive understanding, we provide some items examples of the ALSOLIFE Assessment in the Appendix 1.

Figure 1

Composition of the ALSOLIFE Assessment for children with autism spectrum disorders. The structure of 6 skill-domains and 22 skills subdomains are showed. The counts in brackets are the numbers of items in the specific subdomain or domain.

Administration

Designed for caregivers with minimal professional training, ALSOLIFE Assessment can be conducted via direct testing and observation. Along with a question prompt, most assessing items also came with a short online video clip or detailed written description that addressed the uniform administration procedures and specific guidelines for preparing testing materials. If the caregivers experienced any difficulties, professional/technical assistance was available via parent support groups.

Upon completion of registration process via ALSOLIFE website (www.alsolife.com), the caregivers were asked to fill in the basic information of their child. The system then automatically delivered assessing items from each skill domain based on the child’s chronical age. ALSOLIFE Assessment used semi-adaptive testing to identify the ceiling and basal levels, so the scores best represented the child’s ability. If the child scored less than 25% of full scores of the specific level of the skill domain, the system provided items with one difficulty level lower while one level higher if the child scored more than 75%. Therefore, it was not necessary to complete all 511 items. Depending on the child’s ability, it took approximately from one hour to up to three hours to complete ALSOLIFE Assessment. Caregivers could choose to leave the assessment at any time and resume it later. All assessing items were scored on a triad scale: 0 points = not meeting the requirement, 0.5 point = partially meeting the requirement, and 1.0 point = fully meeting the requirement. Scores for items between the ceiling and basal levels were calculated to obtain the final score of each skill domain. The final total score of ALSOLIFE Assessment is the sum of the scores of the six skill domains and the maximum score of the ALSOLIFE Assessment is 511.

Psychoeducational Profile 3rd Edition (PEP-3)

The PEP-3 is an assessment tool targeting young children with ASD between 2–7 years old across multiple skills and behaviors. Multiple studies have established that the simplified Chinese version of the PEP-3 has good reliability and validity in children with ASD in China (17-19). The test administration includes the Performance Test and the Caregiver Report. The former consists of 10 subtests and a total of 172 items to measure communication ability, motor ability, and maladaptive behaviors. Three subsets assessing communication include cognitive verbal/preverbal (34 items), expressive language (25 items), and receptive language (19 items). Motor ability is measured through gross motor (15 items), fine motor (20 items), and visual-motor imitation skills (10 items). Maladaptive behaviors included four subtests; they are affective expression (11 items), social reciprocity (12 items), characteristic motor behaviors (15 items), and characteristic verbal behaviors (11 items). Additional three subtests are in the Caregivers Report, problem behavior (10 items), personal self-care (13 items), and adaptive behavior (15 items), which allows the primary caregivers to observe and report their children’s natural actions. It has a three-prong scale system, with 0= Fail, 1= Emerge, and 2= Pass.

VB-MAPP

The VB-MAPP is one of the most widely used criterion referenced assessment tool, direct training curriculum guide, and skill tracker to assess verbal and related skills of young children with ASD in China (12). Recent research has measured the reliability and validity of VB-MAPP, which suggests its efficacy to provide assessment and intervention for children with language delays (20,21). VB-MAPP is designed to assess 16 skill domains: mand, tact, echoic, intraverbal, listener, motor imitation, independent play, social and social play, visual perceptual and matching-to-sample, linguistic structure, group and classroom skills, and early academics, with a total of 170 milestones across three developmental levels (0–18, 18–30, and 30–48 months). Out of 170 items, 166 are scored using a triad system in which 0 = does not meet the requirement, 0.5 = meets half of the requirement, and 1 = meets the requirement in full. The remaining four items are scored using a binary system: 0 = does not meet the requirement and 1 = meets the requirement in full.

Procedures

Determination of content validity

Adopting content validity ratio (CVR) techniques described by Lawshe (22), we invited six experts from different disciplines to evaluate the content validity of ALSOLIFE Assessment (). While all invited experts had some extent of credentials and at least 5 years of experiences in the field of ASD (23), none of them were part of the ALSOLIFE Assessment development team. We asked the experts to rate each of the 511 ALSOLIFE items on a three-point Likert scale (3 = Essential, 2 = Useful, but not essential, 1 = Not necessary) based on their professional judgement of whether the item was “essential” for a certain skill subdomain.

Table 2

Demographics description of the expert members for content validity test

	Degree	Certification	Occupation	Years in applied experience
Expert 1	Ph.D.	BCBA-D	Director of teaching programs for children with ASD by IEPs	15
Expert 2	M.S.	BCBA	Assessment, design and teaching children with ASD by IEPs	10
Expert 3	M.S.	Special Education Teacher	Teaching children with ASD in special education center	7
Expert 4	M.D.	Pediatric Psychiatrists	Certified Pediatric Psychiatrist who has completed ADOS-2 assessment training	10
Expert 5	M.D.	Special Educationalist	Special Educationalist who engaged in the applied research for children with ASD	15
Expert 6	Ph.D.	Developmental Psychologist	Developmental Psychologist who focused on the applied research for children with ASD	15

BCBA, Board Certified Behavior Analyst; BCBA-D, Board Certified Behavior Analyst-Doctoral; ASD, autism spectrum disorders; IEP, individualized education plans; ADOS-2, Autism Diagnostic Observation Schedule (2nd edition).

Criterion-related validity

Out of all participating children, 31 had taken PEP-3 assessment (age of 2.21–6.51 years old) while another 34 had taken VB-MAPP (age of 1.8–6.04 years old), all well within two months prior to participating in this study. Upon obtaining the electronic versions of the reports from the caregivers, we used their PEP-3 and VB-MAPP scores to examine the criterion-related validity of ALSOLIFE Assessment.

Reliability tests

We recruited both parents of each of the 49 participating children with ASD to extend the inter-rater reliability check. They were asked to conduct the assessment independently and simultaneously. In addition, 87 caregivers voluntarily assessed their own child two-week after the initial assessment, using the same procedure, which permits the test-retest reliability check.

Statistical analysis

SPSS 22.0 was used for the statistical analysis of the normal distribution of data, internal consistency reliability (Cronbach’s alpha), inter-rater reliability test (Intraclass correlation coefficient, ICC), test-retest reliability test (Spearman correlation coefficient, rs), and criterion validity (Spearman correlation coefficient, rs; regression coefficients, β). Mplus 7.0 was used to conduct exploratory factor analysis (EFA) and confirmatory factor analysis (CFA).

Results

Validity of ALSOLIFE assessment

Content validity

A CRV value was calculated via the rating scores from the six invited experts. Out of a total of 511, 464 items (i.e., 90% of the items) exceeded the CVR cutoff of 0.67, which evidenced content validity (24). Specifically, 442 items had a CVR value of 1.00 and 22 of 0.67. The remaining 47 items had a CVR value of less than 0.67, suggesting insufficient evidence of content validity. After reviewing each of these 47 items, we found that they were either redundant or repetitive, which suggested that the core content of these 47 items had been included in the other 464 items. To ensure the content validity of the ALSOLIFE Assessment, we omitted these 47 items and used the remaining 464 items for the following reliability and validity tests.

Construct validity

We randomly divided the score set of all participating children in two groups and analyzed the data through EFA and CFA. The EFA analysis was performed using the Mplus7.0 robust maximum likelihood method. The results showed that the model fit well when the number of factors was 6 [χ2/df = 4.18, comparative fit index (CFI) = 0.98, Tucker-Lewis Index (TLI) = 0.97, standardized root mean square residual (SRMR) = 0.003, root mean square error of approximation (RMSEA) = 0.08], and the factor loading of each item in the oblique rotation ranged from 0.612 to 0.994 (). The factor loading distribution pattern was consistent with the structural assumption of the ALSO conception.

Table 3

EFA factor loadings of the ALSOLIFE Assessment

	Academic skills	Cognitive skills	Life skills	Living skills	Social regulation skills	Social interactive skills
Motor imitation	0.493*
Speaking and writing skills	0.810*
Tact	0.945*
Mathematical concepts	0.963*
Receptive labeling		0.846*
Sensory perception		0.917*
Samples matching		0.799*
Dressing and undressing skills			0.960*
Self-feeding			0.891*
Daily chores			0.828*
Personal hygiene skills			0.953*
Independent playing games				0.927*
Painting and handcrafting skills				0.908*
Exercise capacity				0.993*
Classroom behaviors					0.728*
Emotion management skills					0.880*
Community life skills					0.910*
Interactive language						0.872*
Understanding social games						0.926*
Demonstrating prosocial behavior						0.956*
Following directions						0.790*
Mand						0.923*

*, P<0.05. EFA, Exploratory factor analysis.

The CFA results showed that the correlation coefficient values of the six factor pairs had r-values ranged from 0.663 to 0.874. Analysis with a medium-to-high correlation coefficient (i.e., above 0.4) could benefit from a bifactor model instead of the traditional second-order factor model or correlated traits multidimensional model (25-27). Through the competition model selection method, we also found that the bifactorial model with six group factors had the best model fit indices (). The fit indices of the bifactorial model with six group factors showed significant differences from other competing models: the six-factor traditional second-order factor model and the six-factor correlated traits multidimensional model ().

Table 4

Fit indices of the CFA competition models of the ALSOLIFE Assessment

Model	χ2/df	CFI	TLI	AIC	BIC	SRMR	RMSEA (90% CI)
Six-factor correlated traits multidimensional model	11.31	0.92	0.91	44,968.21	45,312.62	0.04	0.15 (0.14, 0.15)
Six-factor second-order factor model	11.70	0.92	0.90	45,153.85	45,459.99	0.05	0.15 (0.14, 0.15)
Bifactorial model with 6 local factors	11.05	0.93	0.91	44,828.73	45,202.90	0.04	0.14 (0.13,0.14)

CFA, confirmatory factor analysis; χ2, Chi-square; df, degree of freedom; χ2/df, normed Chi-square; CFI, comparative fit index; TLI, Tucker-Lewis index; AIC, Akaike information criterion; BIC, Bayesian information criterion; SRMR, standardized root mean square residual; RMSEA, root mean square error of approximation.

Table 5

Significant difference tests between the bifactorial model with six group factors and other competing models of the ALSOLIFE Assessment

Model	△χ2	△df	P
Six-factor trait-related multidimensional model	154.21	7	<0.01
Six-factor traditional second-order factor model	357.82	16	<0.01

General factors further reflected the commonality among multiple factors. The general factors of the 22 subskills and the factor loadings and variance contribution rates of the six group factors fell within a reasonable range (27) (), suggesting that the factor structure of the ALSOLIFE Assessment was consistent with a bifactorial model with six group factors.

Figure 2

Confirmatory factor model for the ALSOLIFE Assessment. This is a bi-factorial model with six group factors (i.e., academic skills, cognitive skills, life skills, living skills, social interaction skills, and social regulation skills) and one general factor (i.e., general learning ability across six domains). The “G” in this figure is representing the general factor. Factor loadings and variance are showed in the middle of the path or next to the arrow.

Positive correlations were significant (r=0.340–0.900, P<0.05) between the scores of the 22 subskills in the ALSOLIFE Assessment and the scores of 16 milestones in VB-MAPP. Overall, the total ALSOLIFE Assessment score significantly positively predicted the total score of VB-MAPP milestones in the same sample of children with ASD (β=0.81, R2=0.923, P<0.001). The six factors of the ALSOLIFE Assessment and the three factors of the PEP-3 showed significant moderate positive correlations (r=0.578–0.627, P<0.01), and the scores for the 22 subskills of the ALSOLIFE Assessment and the 13 subtests of the PEP-3 all showed significant positive correlations (r=0.356–0.776, P<0.05).

Reliability of ALSOLIFE assessment

Internal consistency reliability

Internal consistency was measured with Cronbach’s alpha, which quantifies the degree of homogeneity among different items within a measure. The results showed that Cronbach’s alpha for the entire ALSOLIFE Assessment (total score data) was 0.942, well above the cutoff value of 0.7 (28). It indicated that ALSOLIFE Assessment had excellent internal consistency reliability.

Inter-rater reliability

Scores from both parents conducting ALSOLIFE Assessments simultaneously were used to test the inter-rater reliability. The results showed that the intraclass correlation coefficient (ICC) value of the ALSOLIFE Assessment total score was 0.92, while ICC values for the 22 skill subdomains of the ALSOLIFE Assessment ranged from 0.88 to 0.95 (P<0.01). Given ICC value over 0.80 representing good consistency, ALSOLIFE Assessment was consistent across different assessors.

Test-retest reliability

We evaluated test-retest reliability using scores from 87 participating children who were retested using the ALSOLIFE Assessment after two-week period. The results showed that the test-retest intraclass correlation coefficient (ICC) value of the ALSOLIFE assessment total score was 0.98, and the ICC value of 22 skill subdomains was between 0.94 and 0.99 (P<0.01). Thus, ALSOLIFE Assessment satisfied the requirements of test-retest reliability, suggesting stability as a skill measure (24,25).

Discussion

This study examined the reliability and validity of the ALSOLIFE Assessment based on the data collection from 1,050 children with ASD aged from 1 to 10 years old in mainland China. Psychometric evaluation of the ALSOLIFE Assessment demonstrated satisfactory internal consistency, test–retest reliability, inter-rater reliability, content validity, construct validity, and good criterion-related validity compared with VB-MAPP and PEP-3.

Through the CVR analysis, we designated 464 items out of the 511 original ones to ensure the content validity of the ALSOLIFE Assessment meets the psychometric requirement. Compared to VB-MAPP and PEP-3, the ALSOLIFE Assessment exhibited good criterion-related validity across three levels: total scores, six skill domains, and 22 skill subdomains. EFA results verified six factors making up the ALSOLIFE Assessment, which conforms to the structural assumptions of the ALSO conception (). The CFA competition models test indicated that the best fitting construct model of the ALSOLIFE Assessment score data is the bifactorial model with a general factor and six group factors (29). This finding revealed that a general factor could be extracted among the six skill factors of the ALSOLIFE Assessment and suggested that some common factors influenced the scores of the six skill domains. For instance, research found that a general factor was extracted from the construct validity model of the Wechsler Intelligence Scale for Children (30). A combination of two parts could potentially explained the general factor of ALSOLIFE Assessment. On one hand, general functioning (e.g., general learning ability) may influence to some extent of all six skill domains. On the other hand, the fact that all the domains are evaluated using the same assessment method contributing to a common methodological factor. As shown in , by comparing the values of factor loadings and variance contribution rates of the general factor and six group factors (31), we can conclude that the general factor is the dominant factor, while the six group factors also possessing its unique contributions to the construct validity model of the ALSOLIFE Assessment.

In terms of its reliability, the results showed a satisfactory internal consistency reliability, inter-rater reliability, and test-retest reliability at three levels: the entirety of ALSOLIFE Assessment, its six skill domains, and its skill subdomains. It suggested ALSOLIFE Assessment was a consistent, stable, and reliable tool.

There are some limitations of this study. First, the participants were recruited directly via online platform, as self-identified families of ASD. It might potentially influence the sample representation and accuracy of the demographic information. However, we chose online recruitment due to its benefits to overcome geographical restrictions while keeping it consistent with the natural context of ALSOLIFE platform. Through such recruitment procedure, we might potentially prevent any issues with external validity (ecological validity). In responding to the issues related to self-reported information, we asked participants to provide a copy of the official medical reports from a pediatric psychiatrist. We also confirmed any unclear demographic information through an additional telephone interview.

Second, the criterion validity testing of the ALSOLIFE Assessment was limited to the use of existing VB-MAPP and PEP-3 electronic report scores, obtained two months prior to this study. The time lag between tests might hinder the accurate analysis of concurrent validity. However, such data was still useful to determine predictive validity. Since none of the recruited children were younger than 1.5 years of age, significant developmental changes over two months period may not be a concern.

Despite the limitations outlined above, the present study is the first empirical validation study of the ALSOLIFE Assessment that provides meaningful findings. Results indicated the reliability and validity of the ALSOLIFE Assessment satisfied psychometric requirements after trimming to 464 items. In comparison with PEP-3 and VB-MAPP, the ALSOLIFE Assessment is still reliable and validated in the same sample of children with ASD. The construct validity model of the ALSOLIFE Assessment score data was firstly explored and confirmed as the bifactorial model with one general factor and six group factors, which conforms to the structural assumptions of the ALSO conception.

In conclusion, it is essential to develop ALSOLIFE Assessment from the conceptually systematic framework to evidence-based assessment tools. The validated ALSOLIFE Assessment can be an accuracy tool for caregivers of children with ASD to obtain a general picture of their child’s current functioning. Along with the aligned ALSOLIFE IEP system, this free online service system could potentially benefit caregivers in the development of a home-based intervention program, particularly for those who have trouble accessing high-quality assessment and IEPs. Future research should investigate the effectiveness of ALSOLIFE IEP system and the proper alignment of both components. It is imperative to develop validated, effective, and systematic evaluation and intervention tools, embedded on the needs of families of children with ASD unique to the cultural and societal context of China.

The article’s supplementary files as