Selecting score types for longitudinal evaluations: the responsiveness of the Comprehensive Developmental Inventory for Infants and Toddlers in children with developmental disabilities.

OBJECTIVE: The objective of this study was to examine the responsiveness of the Comprehensive Developmental Inventory for Infants and Toddlers (CDIIT) in children with developmental disabilities (DD).
METHODS: The responsiveness of a measure is its ability to detect change over time, and it is fundamental to an outcome measure for detecting changes over time. We compared the responsiveness of four types of scores (ie, raw scores, developmental ages [DAs], percentile ranks [PRs], and developmental quotients [DQs]) in the five subtests of the CDIIT. The CDIIT was administrated three times at intervals of 3 months on 32 children with DD aged between 5 months and 64 months (mean =30.6, standard deviation [SD] =17.8). The CDIIT is a pediatric norm-referenced assessment commonly used for clinical diagnosis of developmental delays in five developmental areas: cognition, language, motor, social, and self-care skills. The responsiveness was analyzed using three methods: effect size, standardized response mean, and paired t-test.
RESULTS: The effect size results showed that at the 3-month and 6-month follow-ups, responsiveness was small or moderate in the raw scores and DAs of most of the subtest scores of the CDIIT, but the level of responsiveness varied in the PRs and DQs. The standardized response mean results of the 3-month and 6-month follow-ups showed that most of the subtest scores of the CDIIT had respectively moderate and large responsiveness in raw scores and DAs, but the responsiveness varied (from no to large) in PRs and DQs.
CONCLUSION: The findings generally support the use of the CDIIT as an outcome measure. We also suggest using the raw scores and DAs when using a norm-referenced pediatric developmental assessment to evaluate developmental changes and program effectiveness in children with DD.

Introduction

Developmental disabilities (DD) are a group of chronic conditions that are attributable to physical and mental impairments during the developmental period.1 Common examples are intellectual disability, cerebral palsy, and autism spectrum disorder.2 Children with DD often manifest lifelong disabilities in cognition, language, motor, social, and self-care skills.2 According to a new report from the Federal Centers for Disease Control and Prevention, the prevalence of DD is approximately one in six, which means that ~15% of children aged 3–17 years have one or more DD3 with various degrees of severity and need coordinated services for their special health care, education, and social welfare, such as early intervention and continuing special education. Therefore, a comprehensive measure is warranted to detect the area and extent of developmental delays, to evaluate the effectiveness of the early interventions or education programs, and to predict the prognosis and needs for future health care and services in children with DD.

The Comprehensive Developmental Inventory for Infants and Toddlers (CDIIT)4 is specifically designed for infants and children aged 3–71 months. It is commonly used to assess five important developmental areas: cognition, language, motor, social, and self-care skills. The CDIIT was designed to be used as a diagnostic and screening test to identify strengths and weaknesses in the five developmental areas and to establish developmental levels.4 The CDIIT is included among the recommended measures for children with DD in child developmental centers in Taiwan because of its comprehensive coverage of pediatric development, concrete and interesting materials, complete norm establishment, and clinical applicability. The CDIIT has been proved to be psychometrically sound, having good internal consistency, test–retest and interrater reliabilities, construct validity, concurrent validity, predictive validity, and diagnostic accuracy,4–10 and may have the potential for use as an outcome measure to assess and monitor developmental skills when children with DD are the subjects of intervention.

Responsiveness is fundamental to an outcome measure for detecting changes over time (its evaluative purpose).11,12 The responsiveness of a measure is its ability to detect change over time, especially in response to an intervention.11,13 Therefore, in both clinical practice and research, an outcome measure must have sufficient responsiveness to detect treatment effects.11,13–15 However, the responsiveness of the CDIIT has yet to be established, so the potential of the CDIIT for use as an outcome measure for evaluating children’s development and treatment effects longitudinally is unknown.

The purpose of this study was to examine the responsiveness of the CDIIT longitudinally and thoroughly in children with DD. We compared the responsiveness of the four types of CDIIT scores: the raw scores, developmental ages (DAs), percentile ranks (PRs), and developmental quotient (DQ). The results may serve as a reference in determining which scores to use as outcome indicators of the CDIIT and also as a reference for choosing scores for a norm-referenced pediatric developmental assessment.

Methods

Participants

A total of 32 children with DD aged between 5 months and 64 months were recruited from the Child Development and Assessment Center of the Chi Mei Medical Center in Taiwan between March 2012 and December 2014. These children were receiving early intervention programs at the time of the study. The early intervention program was individually based on the results of each child’s individualized assessments, including observation of free play or play in a group, interviews of the caregiver, and standard assessment tools, but not targeted to specific activities in the CDIIT. Written informed consent was given by their primary caregivers, and the Institutional Review Board of the Chi Mei Medical Center approved the protocol for this study.

Measures

Comprehensive Developmental Inventory for Infants and Toddlers

The CDIIT consists of two parts: the diagnostic test (CDIIT-DT) and the screening test (CDIIT-ST).4,5 Only the CDIIT-DT was used in this study. The CDIIT-DT includes five subtests and a behavior rating scale for assessing a child’s developmental capacities and behavioral characteristics in five developmental areas: cognition, language, motor, social, and self-care skills. The cognition subtest assesses a child’s mental capacities, including attention; perception; memory; reasoning; and concepts of color, shape, size, and number. The language subtest consists of expression and comprehension subdomains. The motor subtest includes two subdomains: gross motor and fine motor. The gross motor subdomain includes items to assess gravity compensation, locomotion, and body-movement coordination, while the fine motor subdomain includes items for basic hand use and visual-motor coordination. The social subtest has sections on interpersonal communication, affection, personal responsibility, and environmental adaptation. The self-help subtest comprises items about feeding, dressing, and hygiene skills.

Every item on the CDIIT-DT is scored 0 or 1, respectively, indicating whether the child “fails” or “passes” that item. Scores can be assigned based on clinical testing or home observation by the caregivers. In the present study, items in the cognition and motor subtests and part of the language subtest were individually and directly assessed by a trained administrator. The social and self-help subtests were scored by the primary caregivers. Based on the CDIIT manual, the raw score of each subtest and total score can be transformed into three other types of scores: DA, PR, and DQ. Altogether, the four types of scores were obtained for each subtest, for the gross motor and fine motor subdomains, and for the whole test.

As regard the reliability of the CDIIT-DT, the internal consistency,9 test–retest reliability (intraclass correlation coefficient =0.76–1.00), and interrater reliability (intraclass correlation coefficient =0.76–1.00) of the subtests and composites are good.5 The CDIIT-DT has good accuracy, similar to that of the Peabody Developmental Motor Scales – Second Edition for motor development evaluation in preschool children.7 With respect to its validity, the construct validity, concurrent validity, and predictive validity have been shown to be valid. The construct validity has been validated with exploratory factor analysis.8 Regarding the concurrent validity, the scores of the CDIIT subtests have been shown to be significantly and moderately correlated with the scores of the Bayley Scales of Infant Development-II in preterm and full-term infants.6,10 In addition, the CDIIT also has fairly good predictive validity for diagnostic results and later school performances or special education needs, as measured by the Child Problems Referral Survey and Preschool Children Development Checklist.16

Procedures

The children were administered the CDIIT three times at intervals of 3 months by trained administrators in clinical settings. The administrators were therapists of occupational therapy, physical therapy, speech therapy, and psychology, all of whom were trained in the standard procedures of the developmental center. Demographic information was collected from the caregivers of the children, and the administrators, children, and caregivers were blinded to the purpose of the study.

Statistical analysis

Children’s CDIIT raw scores were transformed into DAs, PRs, and DQs according to the norms of normally developing children presented in the original manual. The demographic properties of the participants and the CDIIT scores were then characterized with descriptive analysis. The four types of scores (ie, raw scores, DAs, PRs, and DQs) were used for analyzing the responsiveness.

The responsiveness of the CDIIT was examined with the effect size (ES), standardized response mean (SRM), and paired t-test. All statistical analyses were performed using SPSS 17.0 (SPSS Inc., Chicago, IL, USA).

Effect size

The ES, a measure of change, is calculated by dividing the mean difference between baseline and follow-up measurements by the pooled SD of the baseline and follow-up measurements.17 Values of 0.20, 0.50, and 0.80 indicate small, moderate, and large ES, respectively.18

Standardized response mean

The SRM is the mean difference in the scores of two consecutive measurements divided by the SD of that difference.19 Thus, the SRM gives an estimate of change in the measure that is standardized relative to the variability of change scores. As with ES, values of 0.20, 0.50, and 0.80, respectively, are considered to show small, moderate, and large responsiveness.18

Paired t-test

The statistical significance of the change in scores was determined using the paired t-test.20 The alpha level was set at 0.05.

Results

Participant characteristics

A total of 32 children with DD (23 boys and nine girls) ranging in age from 5 months to 64 months (mean: 30.6 months, SD: 17.8 months) and their caregivers participated in the study. The diagnoses of the children with disabilities consisted of psychomotor retardation (n=18), cerebral palsy (n=6), attention deficit hyperactivity disorder (n=4), Prader–Willi syndrome (n=2), Rubinstein–Taybi syndrome (n=1), and Marfan syndrome (n=1). The characteristics of the 32 children are presented in Table 1. Table 2 presents the mean and SD of the raw scores, the DAs, the PRs, and the DQs for each subtest of the CDIIT.

Table 1

Sample characteristics of children with developmental delays (N=32)

Characteristics	Statistics
Age (months); mean (SD), range	30.6 (17.8), 5–64
Sex (M/F), n (%)	23 (71.9)/9 (28.1)
Diagnosis of developmental delays, n (%)
Psychomotor retardation	18 (56.3)
Cerebral palsy	6 (18.8)
Attention deficit hyperactivity disorder	4 (12.5)
Prader–Willi syndrome	2 (6.3)
Rubinstein–Taybi syndrome	1 (3.1)
Marfan syndrome	1 (3.1)
Number of children in the family, n (%)
1	16 (50.0)
2	12 (37.5)
3	3 (9.4)
Birth order, n (%)
1	18 (58.1)
2	12 (38.7)
3	1 (13.1)
Father’s age (years); mean (SD), range	35.6 (5.0), 25–46
Mother’s age (years); mean (SD), range	34.1 (4.3), 23–42
Father’s education, n (%)
Graduate school	3 (9.4)
University	14 (4.4)
Senior high school	13 (40.6)
Junior high school	1 (3.1)
Mother’s education, n (%)
Graduate school	1 (3.1)
University	17 (53.1)
Senior high school	10 (31.3)
Junior high school	3 (9.4)

Abbreviations: SD, standard deviation; M, male; F, female.

Table 2

Descriptive statistics of the CDIIT for the children with developmental delays (N=32)

Subtest	Raw score, mean (SD)					DA (month), mean (SD)					PR, mean (SD)					DQ, mean (SD)
	Baseline	3 months	d1a	6 months	d2b	Baseline	3 months	d1a	6 months	d2b	Baseline	3 months	d1a	6 months	d2b	Baseline	3 months	d1a	6 months	d2b
Cognition	29.9 (20.1)	35.1 (21.8)	5.1 (4.2)	40.0 (23.5)	10.2 (5.7)	23.1 (17.8)	27.5 (19.2)	4.3 (4.0)	31.7 (20.6)	8.7 (5.1)	11.6 (17.5)	14.3 (21.0)	2.7 (14.1)	21.1 (24.9)	9.3 (18.8)	72.7 (16.0)	75.7 (17.1)	3.0 (15.0)	80.7 (17.9)	8.1 (15.8)
Language	25.9 (18.5)	29.0 (19.1)	3.1 (5.6)	32.7 (20.3)	6.9 (5.0)	22.2 (16.3)	25.8 (18.5)	3.6 (5.1)	29.6 (20.1)	7.4 (5.8)	10.4 (15.0)	15.0 (21.2)	4.6 (13.7)	19.4 (29.0)	8.7 (24.9)	72.4 (15.1)	74.7 (18.0)	2.3 (12.3)	77.3 (20.5)	4.4 (16.9)
Motor	49.8 (25.8)	57.0 (24.0)	7.3 (5.9)	59.7 (24.5)	10.6 (5.4)	22.3 (15.5)	26.3 (17.0)	4.0 (3.5)	29.3 (19.0)	7.5 (4.9)	4.2 (6.1)	8.3 (12.0)	4.1 (11.5)	10.5 (13.6)	6.9 (12.8)	65.7 (11.6)	69.7 (14.6)	4.0 (13.8)	71.3 (15.8)	6.3 (14.2)
Social	27.5 (13.8)	31.2 (14.6)	3.8 (9.5)	32.5 (14.4)	5.3 (9.3)	21.8 (17.7)	27.2 (18.6)	5.5 (8.8)	28.8 (18.8)	7.1 (9.2)	24.5 (30.8)	25.9 (26.1)	1.5 (23.0)	26.0 (28.2)	1.0 (25.2)	81.9 (21.4)	84.3 (17.6)	2.4 (14.1)	84.1 (20.4)	2.1 (17.1)
Self-care	20.6 (14.6)	23.9 (15.6)	3.3 (6.9)	25.1 (15.1)	5.0 (7.8)	22.9 (17.1)	27.1 (19.1)	4.1 (5.3)	28.2 (19.2)	5.9 (8.1)	14.1 (22.0)	17.5 (23.6)	3.3 (19.9)	19.2 (25.9)	4.7 (25.2)	72.6 (21.2)	77.1 (17.9)	4.5 (16.3)	75.7 (20.5)	2.5 (18.2)
Total	153.6 (86.2)	176.2 (91.6)	22.5 (18.5)	190.0 (94.9)	38.0 (21.5)	23.9 (18.4)	26.4 (18.2)	2.4 (13.3)	29.5 (19.6)	5.6 (14.3)	8.1 (15.7)	9.9 (17.6)	1.8 (15.5)	14.4 (23.0)	6.1 (20.3)	67.9 (15.3)	70.2 (16.2)	2.3 (13.7)	73.4 (18.6)	5.6 (15.4)

Notes:

Mean difference (3-month follow up – baseline).

Mean difference (6-month follow up – baseline)

Abbreviations: CDIIT, Comprehensive Developmental Inventory for Infants and Toddlers; DA, developmental age; PR, percentile rank; DQ, developmental quotient; SD, standard deviation.

Responsiveness

Table 3 shows the responsiveness for the four types of scores (raw scores, DAs, PRs, and DQs) for each subtest of the CDIIT.

Table 3

The responsiveness of the CDIIT: ES, SRM, and paired t-test (N=32)

Subtest	Raw score			DA (month)			PR			DQ
	ES	SRM	Paired t-test	ES	SRM	Paired t-test	ES	SRM	Paired t-test	ES	SRM	Paired t-test
Three-month follow-up
Cognition	0.24	1.22	0.000*	0.25	1.07	0.000*	0.14	0.19	0.294	0.18	0.20	0.261
Language	0.16	0.55	0.004*	0.21	0.71	0.000*	0.25	0.34	0.194	0.14	0.19	0.302
Motor	0.29	1.24	0.000*	0.25	1.13	0.000*	0.43	0.36	0.223	0.30	0.29	0.112
Social	0.26	0.39	0.033*	0.30	0.62	0.001*	0.05	0.06	0.295	0.12	0.17	0.347
Self-care	0.22	0.48	0.011*	0.23	0.78	0.000*	0.15	0.17	0.368	0.23	0.28	0.126
Six-month follow-up
Cognition	0.47	1.79	0.000*	0.45	1.68	0.000*	0.43	0.49	0.010*	0.48	0.51	0.008*
Language	0.35	1.38	0.000*	0.41	1.29	0.000*	0.38	0.35	0.061	0.24	0.26	0.161
Motor	0.42	1.96	0.000*	0.43	1.54	0.000*	0.66	0.54	0.005*	0.44	0.44	0.020*
Social	0.38	0.57	0.004*	0.39	0.78	0.000*	0.04	0.04	0.821	0.10	0.12	0.500
Self-care	0.34	0.65	0.001*	0.32	0.72	0.001*	0.19	0.19	0.310	0.12	0.14	0.442

Note:

P<0.05.

Abbreviations: CDIIT, Comprehensive Developmental Inventory for Infants and Toddlers; ES, effect size; SRM, standardized response mean; DA, developmental age; PR, percentile rank; DQ, developmental quotient.

At 3-month follow-up, all the subtests had small responsiveness in the raw scores, except for the language subtest, which was not responsive. Regarding the DAs, all the subtests had small responsiveness (0.21–0.30). However, in the PRs, only the language and the motor subtests had small responsiveness. The other subtests were not responsive in the PRs. For the DQs, only the motor (0.30) and self-care (0.23) subtests had small responsiveness.

At 6-month follow-up, all the subtests had small responsiveness in the raw scores (0.34–0.47). Regarding the DAs, all the subtests had small responsiveness, but greater than that at 3-month follow-up (0.32–0.45). However, in the PRs, except for the social and self-care subtests, which had no (0.04 and 0.19, respectively) responsiveness, the cognition and language subtests had small responsiveness (0.43 and 0.38, respectively) and the motor subtest even had moderate responsiveness (0.66). For the DQs, the social and self-care subtests had no responsiveness (0.10 and 0.12, respectively), and the other three subtests had small responsiveness (0.24–0.48).

At 3-month follow-up, with regard to the SRMs of the raw scores of the subtests, the social and self-care subtests had small responsiveness (0.39 and 0.48), the language subtest had moderate responsiveness (0.55), and the cognition and motor subtests had large responsiveness (1.22 and 1.24). For the DAs, all subtests had responsiveness that was better than moderate; the cognition and motor subtests had large responsiveness (1.07 and 1.13) and the other three had moderate responsiveness (0.62–0.78). As regard the SRM of the PRs, only the language and motor subtests were responsive (0.34 and 0.36). For the DQ, the subtests of cognition, motor, and self-care were responsive (0.20–0.29) and the other two were not responsive (0.17 and 0.19).

At 6-month follow-up, all the SRMs of the raw scores of the subtests had extremely large responsiveness (1.38–1.96), except for the social and self-care subtests, which were moderately responsive (0.57 and 0.65). For the DAs, all the subtests had responsiveness that was better than moderate; the cognition, language, and motor subtests had extremely large responsiveness (1.29–1.68) and the other two had moderate responsiveness (0.72 and 0.78). As regard the SRMs of the PRs, the motor subtest was moderately responsive (0.54) and the cognition and language subtests had small responsiveness (0.49 and 0.35). The other two were not responsive (0.04 and 0.19). For the DQs, the cognition subtest was moderately responsive (0.54) and the language and motor subtests had small responsiveness (0.26 and 0.44). The other two were not responsive (0.12 and 0.14).

At 3-month follow-up, all the changes in subtest scores were significant (P<0.01) for the raw scores and DAs, but not for the PRs and DQs. Furthermore, at 6-month follow-up, the results were similar to those at 3-month follow-up, but with additional significant changes in the PRs and DQs in the cognition and language subtests.

Discussion

We believe that this is the first study to examine the responsiveness of the CDIIT in children with DD. In this study, the responsiveness of the CDIIT was thoroughly analyzed. The raw scores DAs, PRs, and DQs were examined with three statistical methods of responsiveness. Regarding the variability of the scores of the initial assessment (ES), the results of the 3-month and 6-month follow-ups showed that most of the subtest scores of the CDIIT had small responsiveness in raw scores and DAs, but the responsiveness varied in PRs and DQs. Regarding the variability of the change scores (SRM), the results of the 3-month and 6-month follow-ups showed that most of the subtest scores of the CDIIT had moderate and large responsiveness, respectively, in raw scores and DAs, but the responsiveness varied (from no to large) in PRs and DQs. These findings about responsiveness support the use of the raw scores and DAs of the CDITT by clinicians and researchers as an outcome indicator to track change over time and to evaluate program effectiveness and developmental changes for children with DD.

Based on the results, both the raw scores and the DAs of the CDIIT are suggested for evaluative purposes because the two types of scores have different purposes. A raw score represents how many items a child passes (1) or fails (0) in a subtest. The DA refers to a child’s level of development within a subtest.21 Therefore, changes in raw scores reflect the degree to which the child has mastered items of functional skills and behaviors in relation to the results of a previous assessment. On the other hand, a change in DA reflects the degree to which the level of development has changed in the intervening time between repeated assessments. Therefore, both the raw scores and the DAs can be used for different purposes to track changes in children’s performance over time, depending on the focus on mastery or development of skills/behaviors.

In this study, the PRs and DQs were less responsive than the raw scores and DA or not responsive at all. Thus, the PR of the CDIIT is not recommended for use as an outcome measure. The PR is the percentage of scores in its normative sample that are better than, the same as, or lower than it, which might explain why PRs were less responsive to children’s changes. The functional performance and behaviors of the children with DD did improve, possibly due to intervention or normal development, as indicated by the raw scores and DAs. However, these improvements of the children with DD did not surpass those of normally developing children of the same age in the normative sample provided in the CDIIT manual.4

Because no well-accepted index has been acknowledged for evaluative purposes,18 especially in the pediatric field, we used three indices to examine the responsiveness of the CDIIT. We found that in general, the values of ES were smaller than those of SRM. This systematic difference can be ascribed to the different denominators of the formulas. The formula for ES is ES = Xchange/SDpooled; its counterpart for SRM is SRM = Xchange/SDchange. The two formulas have the same numerators (Xchange, mean change between baseline and follow-up measurements). The denominators, however, are different. That for ES is the pooled SD of the baseline measurement and follow-up measurement (SDpooled), while that for SRM is the SD of change in scores (SDchange). In our study, SDchange was smaller than SDpooled for every type of score (raw scores, DAs, PRs, and DQs). Thus, in our study, the ES values were smaller than the SRM values. From these observations, it appears that multiple indices should be used to examine the responsiveness of a measure for better interpretation in different contexts.18

One possibility might explain why the responsiveness of the social and self-care subtests was generally smaller than those of the other three subtests, especially those of the PRs and DQs. The social and self-care subtests, which are composite skills, are comparably advanced and based on the fundamental/basic component skills in the other three developmental areas (cognition, language, and motor). Children’s component skills improve and should be integrated, and then their advanced skills can improve. Therefore, children’s social and self-care skills are unlikely to improve a great deal in a short period of time (eg, 6 months in this study) along with the other three developmental areas. The children in our study, as expected, improved a smaller amount in the development of social and self-care skills.

Limitations

This study has several limitations. First, the children with DD with various diagnoses were recruited from a single medical center in southern Taiwan, so the representativeness of our sample was limited. Second, we did not examine whether differential responsiveness existed in subgroups with different diagnoses because of the small sample size. The responsiveness of the CDIIT may require further investigation in specific groups of children with DD or with a population-based sample. Third, despite the interval of 3 months between assessments, the possibility of a practice effect cannot be excluded nor can the possible inflation of the responsiveness of the CDIIT as a result of this effect. Fourth, although the interrater reliability has been examined in the clinical setting,5 it was not specifically examined in the present study. The fifth limitation is the small sample size of this pilot study. Therefore, additional studies in a larger cohort with equal representation across diagnostic categories need to be carried out to generalize the findings and recommendations.

Conclusion

Our results revealed that the CDIIT was responsive in terms of raw scores and DAs, and they supported the use of the CDIIT as an outcome measure for assessing the developmental areas at intervals of 3 months and 6 months in children with DD. In addition, the raw scores and DAs are suggested for evaluative purposes in norm-referenced pediatric developmental measures. Additional studies with a larger sample size are needed to support our findings.