Predictive validity of the Infant Toddler Checklist in primary care at the 18-month visit and developmental diagnosis at 3-5 years: a prospective cohort study.

OBJECTIVE: There is international variation in recommendations regarding developmental screening and growing recognition of the low sensitivity of commonly used developmental screening tools. Our objective was to examine the predictive validity of the Infant Toddler Checklist (ITC) at 18 months to predict a developmental diagnosis at 3-5 years, in a primary care setting.
METHODS: We designed a prospective cohort study, recruiting in primary care in Toronto, Canada. Parents completed the ITC at the 18-month visit and reported developmental diagnosis at 3-5 years (developmental delay, autism spectrum disorder (ASD), attention deficit hyperactivity disorder (ADHD), learning problem). We calculated screening test properties with 95% CIs. We used multivariable logistic regression analyses adjusted for important covariates.
RESULTS: In the final sample (n=488), mean age at screening was 18.5 (SD 1.1) months, and at follow-up was 46.6 (SD 10.0) months. At screening, 46 (9.4%) had a positive ITC. At follow-up, 26 (5.3%) had a developmental diagnosis, including: developmental delay (n=22), ASD (n=4), ADHD (n=1), learning problem (n=1); parents of two children each reported two diagnoses (total of 28 diagnoses). Of four children with a diagnosis of ASD at follow-up, three had a positive ITC at 18 months. The ITC specificity (92%, 95% CI: 89% to 94%) and negative predictive value (96%, 95% CI: 95% to 97%) were high; false positive rate was low (8%, 95% CI: 6% to 11%); sensitivity was low (31%, 95% CI: 14% to 52%). There was a strong association between a positive ITC at 18 months and later developmental diagnosis (adjusted OR 4.48, 95% CI: 1.72 to 11.64; p=0.002).
CONCLUSION: The ITC had high specificity, high negative predictive value, low false positive rate, and identified children with later developmental delay and ASD. The ITC had low sensitivity, similar to other screening tools underscoring the importance of continuous developmental surveillance at all health supervision visits. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

There is international variation in recommendations regarding developmental screening and growing recognition of the low sensitivity of commonly used developmental screening tools. The Infant Toddler Checklist (ITC) was developed for early identification of children, 6–24 months, who have, or are at risk of developing, a communication impairment. There is little research on the predictive validity of the ITC when used to screen children at the 18-month visit in primary care.

The ITC at 18 months had high specificity and negative predictive value, and a low false positive rate for a developmental diagnosis at 3–5 years, including developmental delay and autism spectrum disorder. Children with a positive ITC at the 18-month visit had higher odds of a developmental diagnosis at follow-up. The low sensitivity of the ITC suggests that a positive ITC at the 18-month visit cannot accurately identify those who will have a developmental diagnosis at 3–5 years.

Practitioners and policymakers may consider the ITC as a developmental screening tool at the 18-month health supervision visit. The low sensitivity of the ITC is similar to other screening tools and underscores the importance of continuous developmental surveillance at all health supervision visits.

Introduction

Early identification of young children with developmental disorders is recommended in many countries. While primary care practitioners often perform developmental surveillance, the addition of standardised developmental screening tools may lead to earlier identification and referral for intervention, as shown in a randomised trial.1 However, there is international variation in recommendations regarding screening, including type of tool (general/broadband or domain/disorder-specific tool), age of screening and one-time versus repeat screening.2–11

In Canada, the Canadian Paediatric Society recommends an enhanced 18-month visit in primary care, including the use of a developmental screening tool to ‘stimulate discussion with parents about their child’s development’.2 However, there is no consensus on which screening tool is best suited for one-time screening at this visit, and there is a growing recognition of the low sensitivity of commonly used developmental screening tools.12–17

Considering the importance of speech, language and social communication at 18 months, we examined the Infant Toddler Checklist (ITC). The ITC was developed by Wetherby and Prizant for early identification of children, 6–24 months, who have, or are at risk of developing, a communication impairment.18 Scoring provides recommendations for monitoring and referral.19 The developers also generated evidence supporting the concurrent and predictive criterion validity of the ITC for detection of a range of developmental concerns, including language delay, global developmental delay and autism spectrum disorder (ASD).20–24 However, an instrument should be validated for the target population and setting in which it will be used; we are interested in the criterion validity of the ITC in routine primary care.

Pierce et al selected the ITC for their screen–evaluate–treat model for early identification of children with ASD and have screened more than 44 000 children at 12, 18 and 24 months in a network of 203 primary care paediatricians in California, USA, demonstrating feasibility and physician satisfaction.25 26 Diagnostic evaluation was completed in those referred (39%), leading to developmental diagnoses including ASD and other delays.26 However, since not all children had a diagnostic evaluation, criterion validity of the ITC could not be examined.25 26

Concurrent criterion validity of many developmental screening tools has been evaluated, with concurrent measurement of screening and criterion measures. However, there is great interest in the predictive validity of developmental tools which ‘offers a critical illustration of whether a screening test measures dimensions of development that are enduring and have a meaningful impact on children’s long-term outcomes’.27 Three recent systematic reviews addressed the predictive criterion validity of developmental screening tools; however, of included studies, few examined screening at 18 months or younger, and none included the ITC.28–30

While the ITC may be a promising developmental screening tool for the 18-month visit in primary care, we have identified gaps in evidence regarding its predictive validity. Our objective was to examine the predictive validity of the ITC at the 18-month visit to predict a developmental diagnosis at 3–5 years of age.

Methods

Design, setting, population

Using a prospective cohort design, we recruited healthy children between 2011 and 2018 during a scheduled 18-month health supervision visit at primary care practices participating in a longitudinal cohort in Toronto, Canada called TARGet Kids! (www.targetkids.ca). The cohort profile has been published and registered at ClinicalTrials.gov (NCT01869530).31 Current Canadian recommendations define screening as use of standardised tools to search for developmental delay in asymptomatic populations.32 To ensure a healthy, asymptomatic cohort, we excluded children with chronic health conditions (except asthma), established diagnosis of developmental delay, gestational age <32 weeks, birth weight <1.25 kg and unscheduled visit. To optimise questionnaire completion, parents unable to communicate in English were not included.

ITC at 18 months

At the 18-month visit, parents completed the ITC, a one-page, 24-item checklist, which takes 5 min to complete and 2 min to score.18 19 The checklist produces three composite scores (social, speech, symbolic) and their sum produces a total score. Age-based cut-offs (at or below the 10th percentile) indicate concern/no concern for each of the four scores (three composite scores and total score). The developers recommend that a child with concern on the speech composite should be monitored with repeat ITC in 3 months and referred for evaluation if there is concern on a second ITC (monitor/refer); and a child with concern on the social composite, symbolic composite or total score should be referred for evaluation (refer).

For this study, the ITC was only measured once at the 18-month visit. We examined three components of the ITC: concern for speech delay (defined as concern on the speech composite); concern for other communication delays (defined as concern on the social composite, symbolic composite or total score); positive ITC (defined as concern for speech delay and/or other communication delays).

Developmental diagnosis at 3–5 years

At 3–5 years, using a standardised questionnaire, parents responded to the question: ‘Has your child been diagnosed with any of the following conditions?’ (response options: developmental delay, ASD, attention deficit hyperactivity disorder (ADHD), learning problem, none). Parent response was dichotomised as a developmental diagnosis (yes/no). Parent report of clinician diagnosis, using similarly worded questions, has been used extensively in national surveys of developmental disorders including ASD.33–36

Statistical analysis

We used all available data from children in the TARGet Kids! cohort meeting the eligibility criteria. Descriptive statistics were used to characterise the study participants. To assess the predictive validity of the ITC, we used two approaches.

First, we calculated the screening test properties (sensitivity, specificity, false positive rate, positive predictive value (PPV), negative predictive value (NPV)), with 95% CIs, for each of the three components of the ITC at 18 months, with a developmental diagnosis at 3–5 years as the criterion measure.

Second, we further evaluated the strength of the relationship between a positive ITC at the 18-month visit and the criterion measure, using multivariable logistic regression analyses. Three models were created corresponding to the three components of the ITC: positive ITC; concern for speech delay; concern for other communication delays. Potential confounding variables selected a priori included child age at follow-up, sex, birth weight, maternal ethnicity and family income. Models were adjusted for all covariates regardless of statistical significance.37 All potential confounders had <13% missing data. Missing covariate data were handled by multiple imputation using the fully conditional specification method.38 To reduce the potential for bias, models were run on 20 imputed data sets.39 Results of the 20 imputed data sets were combined, and the parameter estimates (95% CI) for the adjusted pooled models were reported. Statistical significance was defined as p<0.05; all statistical tests were two sided. Statistical analysis was conducted using SAS V.9.4 statistical software (SAS Institute).

Patient and public involvement

Parents and members of the public were not involved in the design, analysis or interpretation of the research presented here. TARGet Kids! has now developed a Parent and Clinician Team (https://www.targetkids.ca/pact).

Results

Participants

Of 593 children with an ITC at baseline and follow-up at 3–5 years, 488 (82%) had outcomes on developmental diagnosis and were included in the analysis (figure 1). The mean age at screening was 18.5 (SD 1.1) months and at follow-up was 46.6 (SD 10.0) months (table 1). At screening, 46 (9.4%) children had a positive ITC: concern for speech delay (n=28, 5.7%); concern for other communication delays (n=30, 6.2%); concern for both speech delay and other communication delays (n=12, 2.5%). At follow-up, 26 (5.3%) children had a developmental diagnosis, with parents of two children each reporting two diagnoses, for a total of 28 diagnoses: developmental delay (n=22); ASD (n=4); ADHD (n=1); learning problem (n=1). Of the four children with a diagnosis of ASD at follow-up (0.8% of the total sample), three had a positive ITC at 18 months, all with concern for other communication delays.

Figure 1

Study participant flow chart. ITC, Infant Toddler Checklist.

Table 1

Characteristics of study participants (n=488)

Characteristic	All participants		Infant Toddler Checklist (ITC)
			Positive	Negative
N		488	46	442
Child and family characteristics	n
Female sex	488	223 (45.7)	19 (41.3)	204 (46.2)
Birth weight, kg	456	3.2 (0.6)	3.1 (0.6)	3.3 (0.6)
Body mass index, z-score	484	0.13 (1.1)	0.17 (0.9)	0.12 (1.1)
Only child	486	220 (45.3)	18 (39.1)	202 (45.9)
Maternal age at birth, years	452	34.4 (4.0)	34.2 (3.8)	34.5 (4.0)
Maternal ethnicity*	425
European		286 (67.3)	22 (55.0)	264 (68.6)
Non-European		139 (32.7)	18 (45.0)	121 (31.4)
Maternal education	480
High school or less		24 (5.0)	2 (4.4)	22 (5.1)
College/university		456 (95.0)	43 (95.6)	413 (94.9)
Family income ($C)	480
Less than $40 000		33 (6.9)	9 (20.0)	24 (5.5)
$40 000–$79 999		54 (11.3)	8 (17.8)	46 (10.6)
$80 000–$149 999		164 (34.2)	13 (28.9)	151 (34.7)
$150 000+		229 (47.7)	15 (33.3)	214 (49.2)
Family immigration status	465
Canadian born		269 (57.9)	24 (54.6)	245 (58.2)
Immigrant, industrialised		60 (12.9)	2 (4.6)	58 (13.8)
Immigrant, non-industrialised		136 (29.3)	18 (40.9)	118 (28.0)
Family history of developmental concern†	356	29 (8.2)	4 (10.8)	25 (7.8)
Baseline
Age at baseline, months	488	18.5 (1.1)	18.3 (0.8)	18.5 (1.2)
Positive ITC screen	488	46 (9.4)	46 (100)	0 (0)
Concern for speech delay	488	28 (5.7)	28 (60.9)	0 (0)
Concern for other communication delays	488	30 (6.2)	30 (65.2)	0 (0)
Follow-up
Age at follow-up, months	488	46.6 (10.0)	47.0 (9.5)	46.6 (10.0)
Developmental diagnosis	488	26 (5.3)	8 (17.4)	18 (4.1)

Data regarding baseline characteristics are presented as mean (SD) or N (%).

*Non-European consists of 37 mixed=2 or more ethnic groups (8.2%), 33 South Asian (6.8%), 31 East Asian (6.4%), 14 African and Caribbean (2.9%), 13 Latin American (2.7%), 7 Southeast Asian (1.4%), 3 West Asian/North African (0.6%), 1 Indigenous (0.2%).

†Family history of developmental concerns includes history of ASD, ADHD, or learning disability in mother, father or siblings.

ADHD, attention deficit hyperactivity disorder; ASD, autism spectrum disorder.

Predictive validity

Sensitivity was 23%–31%, specificity 92%–95%, false positive rates 5%–8%, PPV 17%–21% and NPV 96% (table 2). Children with a positive ITC at 18 months had higher odds of a developmental diagnosis at 3–5 years (adjusted OR (aOR) 4.48, 95% CI: 1.72 to 11.64; p=0.002), as did male sex (aOR 3.05, 95% CI: 1.17 to 7.97; p=0.02) (table 3). There was a strong association between ITC concern for speech delay (aOR 4.78, 95% CI: 1.65 to 13.81; p=0.004) and ITC concern for other communication delays (aOR 4.46, 95% CI 1.71 to 11.64; p=0.002) and a developmental diagnosis at 3–5 years (table 4).

Table 2

Screening test properties of the Infant Toddler Checklist (ITC) screen at 18 months compared with developmental diagnosis at 3–5 years (n=488)

	Developmental diagnosis
ITC screen*	Yes	No
Positive	8	38	46
Negative	18	424	442
	26	462	488
Sensitivity (95% CI), %	30.8 (14.3 to 51.8)
Specificity (95% CI), %	91.8 (88.9 to 94.1)
False positive rate (95% CI), %	8.2 (5.8 to 11.3)
Positive predictive value (95% CI), %	17.4 (9.9 to 28.8)
Negative predictive value (95% CI), %	95.9 (94.8 to 96.8)
Concern for speech delay	Yes	No
Positive	6	22	28
Negative	20	440	460
	26	462	488
Sensitivity (95% CI), %	23.1 (9.0 to 43.7)
Specificity (95% CI), %	95.2 (92.9 to 97.0)
False positive rate (95% CI), %	4.8 (3.0 to 7.2)
Positive predictive value (95% CI), %	21.4 (10.8 to 38.1)
Negative predictive value (95% CI), %	95.7 (94.7 to 96.5)
Concern for other communication delays	Yes	No
Positive	6	24	30
Negative	20	438	458
	26	462	488
Sensitivity (95% CI), %	23.1 (9.0 to 43.7)
Specificity (95% CI), %	94.8 (92.4 to 96.7)
False positive rate (95% CI), %	5.2 (3.3 to 7.7)
Positive predictive value (95% CI), %	20.0 (10.1 to 35.8)
Negative predictive value (95% CI), %	95.6 (94.7 to 96.4)

*An ITC screen is positive if there is concern for speech delay and/or other communication delays.

Table 3

Logistic regression models for the association between screening with the Infant Toddler Checklist (ITC) at 18 months and developmental diagnosis at 3–5 years (n=488)

	Unadjusted analysis		Adjusted analysis*
Predictor	OR (95% CI)	P value	OR (95% CI)	P value
Positive ITC screen	4.96 (2.02 to 12.16)	<0.001	4.48 (1.72 to 11.64)	0.002
Age at follow-up, months	0.98 (0.94 to 1.02)	0.29	0.97 (0.93 to 1.02)	0.19
Sex, male	2.95 (1.16 to 7.49)	0.02	3.05 (1.17 to 7.97)	0.02
Birth weight, kg	0.98 (0.50 to 1.95)	0.96	0.95 (0.47 to 1.93)	0.88
Maternal ethnicity
Non-European	2.36 (1.05 to 5.31)	0.04	2.10 (0.83 to 5.31)	0.12
European	1.00 (ref)	—	1.00 (ref)	—
Family income ($C)
Less than $40 000	2.73 (0.82 to 9.15)	0.1	1.32 (0.33 to 5.30)	0.69
$40 000–$79 999	1.17 (0.31 to 4.33)	0.82	0.69 (0.17 to 2.89)	0.62
$80 000–$149 999	0.75 (0.27 to 2.08)	0.58	0.65 (0.23 to 1.85)	0.42
$150 000+	1.00 (ref)	—	1.00 (ref)	—

Bold=statistically significant findings at p<0.05.

*Adjusted for child age at follow-up in months, child sex, birth weight, maternal ethnicity, family income. All covariates were measured at baseline except child age which was at follow-up.

Table 4

Multivariable logistic regression models for the association between screening with the Infant Toddler Checklist (ITC) at 18 months and developmental diagnosis at 3–5 years (n=488)

	Developmental diagnosis
	Unadjusted analysis		Adjusted analysis*
Predictor†	OR (95% CI)	P value	OR (95% CI)	P value
Positive ITC screen‡	4.96 (2.02 to 12.16)	<0.001	4.48 (1.72 to 11.64)	0.002
Concern for speech delay	6.00 (2.19 to 16.44)	<0.001	4.78 (1.65 to 13.81)	0.004
Concern for other communication delays	5.48 (2.01 to 14.89)	<0.001	4.46 (1.71 to 11.64)	0.002

Bold=statistically significant findings at p<0.05.

*Adjusted for child age at follow-up in months, child sex, birth weight, maternal ethnicity, family income. All covariates were measured at baseline except child age which was at follow-up.

†Each row shows results from three separate models.

‡An ITC screen is positive if there is concern for speech delay and/or other communication delays.

Discussion

In this study, parents of 488 children completed the ITC at their child’s 18-month health supervision visit. On the basis of a 10th percentile cut-off, 48 children were expected to have a positive ITC screen. In our sample, 46 children (9.4%) had a positive ITC screen, including 5.7% with concern for speech delay, 6.2% with concern for other communication delays and 2.5% with concern for both. At follow-up, at a mean age of 4 years, approximately 5% had a developmental diagnosis, including developmental delay, ASD, ADHD and learning problems. Of four children with a diagnosis of ASD at 3–5 years, three had a positive ITC screen, all with concern for other communication delays, which is notable as this ITC component is thought to capture ASD.24–26 Children with a positive ITC at the 18-month visit had 4.48 higher odds of a developmental diagnosis at follow-up. The high specificity (92%–95%) and NPV (96%) suggest that most children with a negative ITC screen will not have a later developmental diagnosis. The low false positive rate (5%–8%) suggests that use of the ITC will result in few unintended harms related to overdiagnosis and over-referral.

We also identified low sensitivity of the ITC suggesting that a positive ITC at the 18-month visit cannot accurately identify those who will have a developmental diagnosis at 3–5 years. Low sensitivity of developmental screening tools to predict later outcomes has been recognised as a challenge, due to the dynamic nature of children’s development.27 30 Factors associated with lower sensitivity include younger age at screening and longer latency from screening to outcome.30 Therefore, it is not surprising that screening at 18 months resulted in a low sensitivity to predict developmental diagnosis at 3–5 years. This underscores the importance of continuous developmental surveillance at all health supervision visits, as recommended by professional organisations.2 5

There have been three recent systematic reviews of predictive validity of developmental screening in early childhood, highlighting the interest in understanding which tools best identify children who may benefit from early identification and intervention.28–30 Of included studies, few examined screening at ≤18 months, and none examined the ITC.

Sim et al examined the predictive validity of language screening tools at 2–6 years in six studies.28 Studies of children <18 months were not eligible, and only two studies had a time to follow-up of ≥12 months, with a mean sensitivity of 54%. Cairney et al examined the predictive value of developmental assessment at 1–5 years in 13 studies and found a positive association between poor early child development and later educational difficulties, high specificity and NPV, and low sensitivity.30 Three studies included children <18 months, which assessed association but not screening test properties.

Schonhaut et al examined the predictive validity of the Ages & Stages Questionnaire (ASQ), which is of great importance given its common use in many countries.29 Of five included studies, three screened children at 36 or 48 months; and while the remaining two studies screened some children at 18 months, they included children born preterm. Lamsal et al (not included in any systematic review) examined the predictive validity of the ASQ for parent report of a developmental diagnosis at 4–5 years.15 At 24 months (the youngest age examined), using the 1 SD cut-off for the ASQ, sensitivity was 84% and specificity was 69%; using the 2 SD cut-off, sensitivity was 32% and specificity was 91%. In summary, little is known about the predictive validity of the ASQ in healthy term infants screened at 18 months.

Low to moderate sensitivity has been found in concurrent validity studies of the ASQ.12 13 16 Warren et al found a sensitivity of 55% and specificity of 86% in a meta-analysis of four studies.16 Sheldrick et al found a sensitivity of 35% and specificity of 89% in children 9 months–5.5 years (mean 2.6 years).13 Wilson et al found a sensitivity of 55% and specificity of 95% on the ASQ Communication Scale in children 23–30 months (mean 25 months).12

Low sensitivity of ASD-specific tools has also been found. In a predictive validity study of the Modified Checklist for Autism in Toddlers with Follow-Up in children 16–26 months in primary care, Guthrie et al found a sensitivity of 39% and specificity of 95% for ASD; and a sensitivity of 12% and specificity of 97% for any delay.14

The ITC was developed by Wetherby and Prizant in US children.18 For identification of communication disorders, concurrent validity was assessed in children 12–17 and 18–24 months: sensitivity 86%–89% and specificity 74%–77%21; and predictive validity in children 12–24 months, with language assessed at 3 years: sensitivity 83% and specificity 70%.21 For identification of ASD, in children 9–24 months (n=5385), 60 who were ≥4 years received a diagnosis of ASD; of these, the ITC identified 56, for a sensitivity of 93%.24

Pierce et al selected the ITC in their screen–evaluate–treat model embedded in primary care, aiming for early ASD intervention before 2 years.25 26 A network of 203 paediatricians screened more than 44 000 children at 12, 18 and 24 months using the ITC. Approximately 39% of children with a positive ITC were referred for diagnostic evaluation. Of these, almost half received a diagnosis of ASD and about one-third received a diagnosis of developmental delay, language delay or other delays.26 Pierce et al found that a diagnosis of ASD becomes stable starting at 14 months, that the most common diagnostic transition was from language or developmental delay to ASD, and that almost 24% of children with an ASD diagnosis at 3–4 years were late identified.40 In our study, three of four children with an ASD diagnosis at 3–5 years had a positive ITC at 18 months, supporting the potential for early identification of ASD using the ITC.

Strengths of the ITC include its focus on communication, an important developmental domain at 18 months for which interventions are available. The ITC distinguishes concern for speech delay (monitor/refer) and other communication delays such as ASD (refer). Overall, the ITC screen positive rate is about 10% and false positives are low, minimising overdiagnosis and over-referral. Additional advantages include its one-page format, ease of completion and availability free of charge.

Limitations of this study include parent report of a developmental diagnosis rather than a standardised clinician assessment. However, parent report of physician diagnosis of developmental disorders including ASD has been used extensively in national surveys such as the US National Survey of Children’s Health and the National Health Interview Survey.33–36 Kogan et al have summarised the evidence supporting the validity of parent report compared with physician diagnosis.33 In our study, maternal education and family income were high, which may limit the generalisability of our findings. However, family income was included as a covariate in our analysis and the overall prevalence of a positive ITC screen in our sample was close to the expected 10th percentile cut-off score.19

Conclusion

Developmental screening at 18 months may lead to early identification of communication impairments (including speech delay and other communication disorders such as ASD) for which interventions are available. This study provides evidence on the predictive validity of the ITC in primary care at the 18-month visit, suggesting that it performs as well as other currently available developmental screening tools.