Validity of the ages and stages questionnaires in Korean compared to Bayley Scales of infant development-II for screening preterm infants at corrected age of 18-24 months for neurodevelopmental delay.

This study aimed to evaluate the validity of the ages and stages questionnaire in Korean (ASQ 1st edition, Korean Questionnaires, Seoul Community Rehabilitation Center, 2000) for premature infants. The study population consisted of 90 premature infants born between January 1, 2005, and December 31, 2011, who were tested using the ASQ (Korean) and Bayley Scales of Infant Development (BSID) (II) at a corrected age of 18-24 months. The validity of the ASQ (Korean) using cut-off values set at < -2 SD was examined by comparing it to the BSID (II) components, namely, the mental developmental index (MDI) or psychomotor developmental index (PDI), which were both set at < 85. The calculation of the sensitivities, specificities, positive predictive values, and negative predictive values of the ASQ (Korean) components revealed that they detected infants with neurodevelopmental delay with low sensitivity and positive predictive values, however, the communication domain showed moderate correlations with MDI. The failure in more than one domain of the ASQ (Korean) was significantly correlated with the failure in MDI. The ASQ (Korean) showed low validity for screening neurodevelopmentally delayed premature infants.

INTRODUCTION

The mortality rates of premature infants and extremely low birth weight infants have dropped owing to the significant advances in perinatal medicine and neonatology. However, the rate of morbidity in high-risk premature infants has not been improved (1, 2). Indeed, a meta-analysis showed that such infants still have poor long-term neurodevelopmental outcomes and growth delay (3). These poor neurodevelopmental outcomes are related to the fact that, in particular, the basal ganglia, hippocampus, cerebellum, and periventricular white matter are frequently impaired in premature infants (4, 5). However, because infants and toddlers have the highest neuronal plasticity among all age groups, the timely diagnosis of neurodevelopmental delay in premature infants and the institution of appropriate early interventions could improve their neurodevelopmental outcomes in the preschool period (6, 7, 8).

The Bayley Scales of Infant Development (BSID) is presently the gold standard test for assessing neurodevelopmental delay in high-risk infants up to the age of 42 months. It has proven concurrent validity with the Denver Developmental Screening Test (DDST), Differential Ability Scales (DAS), and McCarthy Scales of Children's Abilities (8). However, the disadvantages of the Bayley Scales of Infant Development, 2nd edition (BSID [II]) include its long test duration, requirement of highly skilled personnel for interpreting the results, and low cost-effectiveness (9, 10).

Unlike BSID (II), the Ages and stages questionnaire (ASQ) is cost-effective and easy to use (11). ASQ in Korean (ASQ [Korean], ASQ 1st edition, Korean Questionnaires, Seoul Community Rehabilitation Center, 2000) is available for screening neurodevelopmental delay in Korean children (11, 12). Since 2007, standardized and culturally modified Korean ASQ (K-ASQ (2006), ASQ 2nd edition) has been used exclusively in primary care settings in Korea as a screening tool to detect infants with developmental delay (13). While several studies have assessed the validity of K-ASQ (2006) as a screening tool for neurodevelopmental delay in high-risk premature infants (14, 15, 16), the validity of ASQ (Korean) and K-ASQ (2006) for assessing the neurodevelopmental outcomes of high-risk preterm infants has not been fully confirmed. The present study aimed to assess the validity of ASQ (Korean) relative to BSID (II) for premature Korean infants at high risk of neurodevelopmental delay.

MATERIALS AND METHODS

Study population and design

The study cohort consisted of 90 infants. The inclusion criteria were patients with gestational age less than 37 weeks admitted to the neonatal intensive care unit of Asan Medical Center between January 1, 2005, and December 31, 2011, and to whom both BSID (II) and ASQ (Korean) were administered at 18-24 months of corrected age. If both measures were repetitively assessed during 18 to 24 months of corrected age, only the first assessment was used. Outborn infants meeting the above criteria were also included. All BSID (II) tests were performed by the same designated board-certified neonatologist. We used the questionnaire that most closely matched the infant's corrected age (i.e., 18-, 20-, and 24-month questionnaires). The questionnaire was administered by a primary caretaker at either a scheduled clinic visit or at the time of BSID (II) testing. The neonatal characteristics including gestational age; birth weight, height, and head circumference; maternal education level; 1 min and 5 min APGAR scores; twinning; intrauterine growth restriction (IUGR, defined as weight below the 10th percentile for the gestational age); respiratory distress syndrome (RDS), chronic lung disease (CLD, defined as bronchopulmonary dysplasia by Jobe and Bancalari, National Institute of Child Health and Human Development/National Heart, Lung, and Blood Institute Workshop) (17); retinopathy of prematurity (ROP, defined by the International Classification of Retinopathy of Prematurity (ICROP), stage 2 and above) (18); intraventricular hemorrhage (IVH, defined by Papile's grading system (19); sepsis; hearing loss (including mild hearing loss of 20-40 dB); cerebral palsy (CP, defined as abnormal muscle tone, movement, and posture in more than one extremity as diagnosed by a board-certified pediatric rehabilitation medical doctor); and the need for more than 6 hr of oxygen therapy were reviewed (Table 1).

Table 1

General characteristics of the premature infants in the study (n = 90)

Characteristics	No. (%) or Mean ± standard deviation	Range
Female	44 (49)
Male	46 (51)
Gestational age (weeks)	31.9 ± 2.3	25-36
Birth weight (g)	1,663 ± 495	595-3,160
Height (cm)	42.0 ± 5.1	22-54
Head circumference (cm)	29.3 ± 3.1	22-43
Maternal education (graduation)	83 (92.2)
Elementary school	9 (10)
Middle school	0 (0)
High school	16 (17.8)
College	45 (50)
Graduate school	13 (14.4)
APGAR score, 1 min	5.5 ± 2.0	1-8
APGAR score, 5 min	7.5 ± 1.4	3-9
Twin	29 (32.2)
Intrauterine growth restriction	18 (20)
Respiratory distress syndrome	18 (20)
Intraventricular hemorrhage ≥ stage 1	30 (33.3)
Sepsis	9 (10)
Chronic lung disease	9 (10)
Retinopathy of prematurity ≥ stage 2	8 (8.9)
Hearing loss	1 (1.1)
Cerebral palsy	0 (0)
O2 supply	44 (48.9)

Diagnosis of developmental delay was defined as a mental developmental index (MDI) or psychomotor developmental index (PDI) score of <85 (<-1 standard deviation [SD]) on BSID (II) (8). To assess the ability of ASQ (Korean) to identify infants with developmental delay, the critical cut-off scores for five domains (communication, gross motor, fine motor, problem-solving, and personal-social interrelation) were set at 2 SD under the mean. The validity of ASQ (Korean) was then determined by calculating its sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) relative to MDI <85 or PDI <85 on BSID (II).

Statistical analysis

The quantitative data of the continuous variables were described as mean±SD. The categorical variables were described as frequencies with percentile. The correlation between the outcomes of BSID (II) and ASQ (Korean) components was analyzed using the chi-square test for more than 1 domain failure of ASQ (Korean). Spearman's rank correlation was used to assess the relationship between individual ASQ (Korean) domains and BSID (II) domains. All statistical analyses were performed using the Statistical Package for Social Sciences version 21.0 (SPSS, Chicago, IL, USA).

Ethics statement

The study was approved by the institutional review board (IRB) of Asan Medical Center (IRB No. 2014-0596). The written informed consent was waived by the IRB.

RESULTS

Patient characteristics

The clinical characteristics of the 90 premature infants are shown in Table 1. Of these infants, 46 (51.1%) were male. The mean±SD of gestational age, weight, height, and head circumference at birth were 31.9±2.3 weeks, 1,663±495 g, 42.0±5.1 cm, and 29.3±3.1 cm, respectively (Table 1). 24 (26.7%) and 10 (11.1%) infants were diagnosed to have neurodevelopmental delay by MDI and PDI, respectively. 19 (21.1%), 3 (3.3%), 5 (5.6%), 5 (5.6%), and 7 (7.8%) infants were detected to have neurodevelopmental delay in the communication, gross motor, fine motor, problem-solving, and personal-social interrelation domains of ASQ (Korean), respectively (Table 2).

Table 2

Frequency of infants who failed Bayley Scales of Infant Development-II (BSID [II]) or Ages and stages questionnaires in Korean (ASQ [Korean])

Tests	Domain	No. (%)	Mean ± standard deviation	Range
BSID (II)	MDI<85	24 (26.7)	78.5 ± 4.0	68-84
	PDI<85	10 (11.1)	73.0 ± 9.1	53-84
ASQ (Korean) <-2SD	Communication	19 (21.1)	23.7 ± 10.3	0-35
	Gross motor	3 (3.3)	26.7 ± 14.4	10-35
	Fine motor	5 (5.6)	34.0 ± 2.2	30-35
	Problem-solving	5 (5.6)	17.0 ± 7.6	5-25
	Personal-social interrelation	7 (7.8)	22.1 ± 9.5	10-35

MDI, mental developmental index; PDI, psychomotor developmental index.

ASQ (Korean) and MDI

The validity of each ASQ (Korean) domain relative to MDI <85 is shown in Table 3. Compared to MDI <85, the sensitivities, specificities, PPVs, and NPVs of the five ASQ (Korean) domains were in the range of 0.04-0.46, 0.82-0.99, 0.33-0.86, and 0.74-0.82, respectively (Table 3). If equal to or than one ASQ (Korean) domain failed, the sensitivity, specificity, PPV, and NPV of ASQ (Korean) relative to MDI <85 were 0.58, 0.80, 0.52, and 0.84, respectively (Table 3).

Table 3

Validity of Ages and stages questionnaires in Korean (ASQ [Korean]) among premature infants with delayed mental development

ASQ (Korean) <-2 standard deviation	Mental developmental index < 85
	Sensitivity	Specificity	Positive predictive value	Negative predictive value
Communication	0.458	0.879	0.579	0.817
Gross motor	0.042	0.970	0.333	0.736
Fine motor	0.083	0.955	0.400	0.741
Problem-solving	0.083	0.955	0.400	0.741
Personal-social interrelation	0.250	0.985	0.857	0.783
1 domain failed	0.375	0.848	0.474	0.789
2 domains failed	0.125	0.970	0.600	0.753
3 domains failed	0.042	0.985	0.500	0.739
4 domains failed	0.042	1.000	1.000	0.742
≥ 1 domain failed	0.583	0.803	0.519	0.841

ASQ (Korean) and PDI

The validity of each ASQ (Korean) domain relative to PDI <85 is shown in Table 4. When patients had lower scores than the cut-off, set as <-2 SD, for individual ASQ (Korean) domains, the sensitivities, specificities, PPVs, and NPVs were in the range of 0.1-0.3, 0.8-1.0, 0.16-1.0, and 0.89-0.92, respectively (Table 4). When more than one ASQ (Korean) domain failed, the sensitivity, specificity, PPV, and NPV were 0.5, 0.73, 0.19, and 0.92, respectively (Table 4).

Table 4

Validity of Ages and stages questionnaires in Korean (ASQ [Korean]) among premature infants with delayed psychomotor development

ASQ (Korean) <-2 standard deviation	Psychomotor developmental index < 85
	Sensitivity	Specificity	Positive predictive value	Negative predictive value
Communication	0.300	0.800	0.158	0.901
Gross motor	0.300	1.000	1.000	0.920
Fine motor	0.100	0.950	0.200	0.894
Problem-solving	0.100	0.950	0.200	0.894
Personal-social interrelation	0.200	0.938	0.286	0.904
1 domain failed	0.300	0.800	0.158	0.901
2 domains failed	0.0001	0.938	0.0001	0.882
3 domains failed	0.100	0.988	0.500	0.898
4 domains failed	0.100	1.000	1.000	0.899
≥ 1 domain failed	0.500	0.725	0.185	0.921

Correlations between ASQ (Korean) and BSID (II)

Communication showed moderate correlation (r=0.385) and personal-social interrelation (r=0.217) and total score (r=0.282) of ASQ (Korean) showed weak correlation with MDI. Gross motor (r=0.25) weakly correlated with PDI (Table 5). Failure in more than one domain of ASQ (Korean) significantly correlated with failure in MDI (P<0.001) (Table 5).

Table 5

Correlation between Ages and stages questionnaires in Korean (ASQ [Korean]) domains and Bayley Scales of Infant Development-II (BSID [II])

ASQ (Korean)	BSID (II)
	Mental developmental index		Psychomotor developmental index
	P	r	P	r
Communication	< 0.001	0.385*	0.454	-0.080
Gross motor	0.941	0.008	0.018	0.250*
Fine motor	0.081	0.185	0.389	0.092
Problem-solving	0.241	0.125	0.942	0.008
Personal-social	0.040	0.217*	0.425	0.085
≥ 1 domain failure	< 0.001†		0.159

*Spearman's rank correlation analysis; †Chi-square test, ≥1 domain failure compared to MDI<85 or PDI<85, P<0.05.

DISCUSSION

Premature infants have a higher risk of motor, cognitive, and speech delays and behavioral disorders compared to term infants (20, 21, 22, 23, 24, 25). Screening premature infants for neurodevelopmental delay and providing early interventional therapy improves their short-term neurodevelopmental outcomes in the preschool assessments (7, 25, 26). Several standardized developmental screening tests have been shown to be valid for detecting infants who are at high risk of neurodevelopmental delay, including DDST, the Early Language Milestone (ELM) scale for children, Brigance developmental screening, and BSID (24). BSID (II) is a standardized screening test that has been shown to have concurrent validity and to correlate strongly with DAS, McCarthy Scales of Children's Abilities, and Wechsler Preschool & Primary Scale of Intelligence (WPPSI)-Revised (8). Similarly, when Park et al. performed standardization research on Korean children to develop the Korean version of BSID (II) (K-BSID [II]), they found that K-BSID (II) correlated strongly and exhibited concurrent validity with Korean WPPSI (28, 29).

However, BSID (II) is an expensive and time-consuming test. For more cost-effective monitoring of the neurodevelopmental outcomes of infants and toddlers, several questionnaires have been developed as parent-completed developmental screening tools. The most frequently used questionnaires include the Health Status Questionnaire of the United Kingdom, ASQ of Australia, National Children's Study in the US, and Birth to Twenty Study in South Africa (25, 30, 31, 32). In Korea, many studies have examined the neurodevelopmental outcomes of infants and toddlers as assessed by screening instruments, namely, the short-term longitudinal study on the development of Korean infants, panel study of Korean children, panel survey of Korean children and youth, and K-ASQ (2006) (13, 32, 33, 34).

It has been suggested that a screening test has optimal validity when it has a sensitivity of at least 70%-80% and specificity, PPV, and NPV of at least 80% (17, 35, 36). In our study, the sensitivity and specificity of individual ASQ (Korean) domains were <46% and >80%, respectively. In contrast, the specificity and NPV of ASQ (Korean) satisfied the optimal screening test criteria with respect to both the mental and the psychomotor aspects. This finding is in partial agreement with the study that evaluated the validity of K-ASQ (2006) in preterm infants (14). In this study of Korean preterm infants born at gestational age of less than 32 weeks or weight of less than 1,500 g, the sensitivity, specificity, PPV, and NPV of K-ASQ (2006) at a corrected age of 18 months were 0.72, 0.94, 0.88, and 0.84, respectively, in comparison to the BSID (III) results. Furthermore, the sensitivity and specificity at a corrected age of 18 months were higher than those at a corrected age of 4, 8, and 12 months (14). Most distinguishably, our study showed that the sensitivity of ASQ (Korean) at a corrected age of 18-24 months was in the range of 0.1-0.3, and it is far below the optimal values in all ages that were examined in contrast to specificity of 0.8-1.0 and NPV of 0.74-0.92. There may be several reasons for the poor sensitivity of ASQ (Korean). First, both ASQ (Korean) and K-ASQ (2006) were designed to screen the entire public population for general developmental delay, which means that sensitivity tends to be low while specificity tends to be relatively high (37). Second, neurosensory impairments are complex and often subtle in premature infants (7), and therefore, the under detection of the mildly delayed population might have increased the false negativity. Third, different versions of BSID were used in the previous study: BSID (III) subcategorizes expressive language and receptive language, gross motor skills, and fine motor skills. This detailed categorization may have increased the sensitivity for BSID (III) than for BSID (II). Fourth, our study cohort had smaller number of patients with severe neurodevelopmental delay than the cohort in the previous study. Lastly, the usage of different versions of ASQ is another issue. The data collection of BSID (II) and ASQ (Korean) of the developmentally delayed premature infants started from January 2005 in our study, at which time only the Korean version of ASQ, 1st edition copyrighted by Paul H. Brooks Publishing Company in 1995 (11), was available. The lower sensitivities of our study in comparison to K-ASQ (2006) may have arisen from different cut-off points for each domain, of which 9 out of 15 cut-off values were higher in ASQ (Korean) (38). The differences in cut-off values set for each test, according to Heo et al., are based on different characteristics of the sample population, cross-cultural adaptation, and child-rearing customs between Korea and the US (37).

Correlation analysis of the individual domains of ASQ (Korean) and BSID (II) showed that recognition of failure in more than one domain of ASQ (Korean) was significantly correlated with delayed mental development. Especially, the communication domain showed moderate correlations with MDI.

The Mother and Child Health Law and Childcare Act in Korea mandated the use of K-ASQ (2006) as a tool for screening neurodevelopmental delay in all Korean infants and children in the primary outpatient care setting since 2007 (13). However, it is well known that the original ASQ, 2nd edition has limited screening ability in screening social-personal interrelation in neurodevelopmentally delayed formal premature infants (39). Moreover, there are issues with the copyright. Therefore, K-ASQ (2006) and ASQ (Korean) are being replaced by the Korean Developmental Screening Test from 2014 (40).

This study had several limitations. First, the study population had a limited number of infants with severe neurodevelopmental delay. Owing to the limited number of patients under the cut-off score set at <-2 SD, enough data was not available to analyze the validity of each domain of ASQ (Korean) at different corrected ages. Second, there may have been recall bias as the ASQ (Korean) was completed by the parent (16, 31) and selection bias owing to the retrospective observational design of the study. Third, although we sought to administer the ASQ (Korean) questionnaires and BSID (II) at the same corrected age, this was not always possible.

In conclusion, the ASQ (Korean) showed low validity in comparison to MDI and PDI of BSID (II). Thus, it suggests the need for additional measures to increase the yield of screening neurodevelopmentally delayed premature infants at a corrected age of 18-24 months.