Reliability, validity and developmental sensitivity of the Language Use Inventory (LUI) in the Chinese context.

BACKGROUND: Pragmatics has generally been defined as the ability to use language in social situations, it is commonly regarded as the third major component of language ability. To date, there is no tool for assessing early pragmatic development of Chinese-speaking children. AIMS: To describe the translation of the Language Use Inventory (LUI) from English to Mandarin Chinese and to report findings on the Chinese version's reliability, validity and developmental sensitivity. METHODS & PROCEDURES: The original English version of the LUI was translated into Mandarin Chinese. Parents of 177 typically developing (TD) toddlers and preschool children completed the inventory to examine its internal reliability and construct validity and how scores differed across ages and sexes. A total of 31 parents out of the 177 completed the LUI-Mandarin, again within 4 weeks, to assess test-retest reliability. To examine discriminative validity, 43 parents of age- and sex-matched TD children and children with autism spectrum disorder (ASD) recruited from Nanjing Brain Hospital affiliated with Nanjing Medical University completed the LUI-Mandarin. OUTCOMES &
RESULTS: Cronbach's alpha values for the LUI-Mandarin's three parts and for 11 of 12 LUI-Mandarin subscales were 0.707-0.992, with most values in the 0.825-0.992 range. Test-retest reliability ranged from 0.66 to 0.95, indicating good to excellent reliability. Factor analysis of the LUI-Mandarin revealed two different factors, and the total variance explained was 74.38%. The LUI-Mandarin total scores and subscale scores increased with age for both boys and girls, providing evidence of the inventory's developmental sensitivity. Girls, however, had higher total scores than boys at earlier ages (18-23 months). The results of the discriminant validity study revealed that performance was significantly lower in the ASD group than in the TD group with respect to LUI total scores and subscale scores (except for subscale A). CONCLUSIONS & IMPLICATIONS: The LUI-Mandarin is the first and only questionnaire available in China that evaluates the pragmatic language skills of children aged between 18 and 47 months. The results of the study show that the LUI-Mandarin is a valid and reliable tool for Chinese toddlers and preschool children. WHAT THIS PAPER ADDS: What is already known on this subject The LUI is a parent-report questionnaire that can provide comprehensive information about very young children's communicative competence. It is widely used both for assessment and to guide intervention. Additionally, it has been translated into French, Italian, Polish, Arabic, Portuguese and Norwegian and it shows good reliability and validity. What this paper adds to existing knowledge In the present study we describe the translation of the LUI from English to Mandarin Chinese and report findings on the Chinese version's reliability, validity and developmental sensitivity. What are the potential or actual clinical implications of this work? The LUI-Mandarin is the first and only questionnaire available in China that can evaluate pragmatic language skills of children aged between 18 and 47 months. The results show that the LUI-Mandarin is a valid and reliable tool for use with Chinese toddlers and preschool children.

INTRODUCTION

Pragmatics has been generally defined as the ability to use language in social situations (Bates, 1974). With the form (phonology and grammar) and the content (semantics) of language, pragmatics is considered to be the third major component of language competence (Blankenstijn, 1996). It encompasses non‐verbal means of communication (such as eye contact and facial expressions), conversational skills and narrative abilities (Fahey et al., 2001). Traditionally, pragmatics requires not only the linguistic ability to express and understand the words of others but also the cognitive and social skills to respond to different situations (Eigsti et al., 2011). Broadly speaking, pragmatics includes the ability to use language to convey information in different contexts (Adams, 2002; Prutting & Kittchner, 1987), as well as the cognitive, emotional and social aspects of language use (Adams, 2005; Hart et al., 2004; Huang, 2015), which are influenced by individuals’ social experience and cultural circumstances (Hyter, 2007). Most of the studies of pragmatics have focused on the use of language in context and have placed a great emphasis on the functions or uses of language (Matthews et al., 2018).

Based on previous studies, some neurodevelopmental disorders are associated with greater pragmatic impairment, including autism spectrum disorder (ASD) (Baixauli‐Fortea et al., 2017; Bauminger‐Zviely et al., 2013; Dennis et al., 2001; Sturrock et al., 2020; Tager‐Flusberg & Anderson, 1991; Ying Sng et al., 2018), intellectual disability (ID) (Abbeduto & Hesketh, 1997; Cummings, 2017; Thomas et al., 2010), attention‐deficit/hyperactivity disorder (ADHD) (Geurts et al., 2004; Kim & Kaiser, 2000; Mathers, 2006), learning disability (LD) (Lapadat, 1991) and development language disorder (DLD) (Andrés‐Roqueta & Katsos, 2020; Andrés‐Roqueta et al., 2021; Norbury et al., 2004). Studies have shown that the effects of pragmatic impairment are far‐reaching. Pragmatic impairment has been found to be associated with a greater incidence of behavioural problems, especially destructive and externalizing behaviours (Ketelaars et al., 2010). Among children, the long‐term negative effects of pragmatic impairment include poor peer relations (Cummings, 2016), being excluded and rejected by peers in school (Laws et al., 2012; Mok et al., 2014), and having difficulty forming good social relations in adulthood (Whitehouse et al., 2009). With the introduction of a new diagnostic category to the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM‐5), namely, social (pragmatic) communication disorder (SPCD), and the proposals for adding pragmatic language impairment (PLI) to the International Classification of Diseases—11 (ICD‐11) (World Health Organization, 2018), an emphasis on the development of pragmatic abilities in children and methods for assessing these abilities has grown. In addition, the early recognition of social pragmatic communicative impairment has taken on greater urgency to allow earlier intervention and aid in diagnosis.

Current tools used to assess non‐verbal communication and conversational skills related to pragmatics include the Parent Report Rating Scale (Girolametto, 1997), Communication and Symbolic Behavior Scales (CSBS) (Wetherby & Prizant, 1993), Social Interactive Coding System (SICS) (Rice et al., 1990) and Language Use Inventory (LUI) (O'Neill, 2009). Additionally, tools used to assess narrative skills include The Renfrew Bus Story (Cowley & Glasgow, 1994), Narrative Assessment Protocol (NAP) (Justice et al., 2010) and Narrative Language Measures (NLM) (Spencer & Petersen, 2010). Narrative skills involve many aspects of pragmatics, such as reasoning, presupposition, cohesion and coherence, as well as social cognition and working memory. For children aged 0–5 years, in addition to the assessments of pragmatics above, some assessment tools include a few aspects of pragmatics, such as the MacArthur‐Communication Development Inventory (MCDI) (Fenson et al., 1993), Language Development Survey (LDS) (Rescorla, 1989) and Preschool Language Proficiency—5 (PLS‐5) (Zimmerman et al., 2011).

Currently, there are relatively few pragmatic assessment tools for children aged 0–5 years compared with the number available for older children and adolescents at a global level (Hyter, 2007), and this lack is especially obvious in the Chinese context. At present, tools used in China include only the Chinese version of the CSBS‐Developmental Profile (DP) and the Chinese Infant Communication Development Inventory (CCDI). Lin and Chiu (2014) found that there were significant differences in the symbolic behaviour scores between 1‐ and 2‐year‐old Taiwanese children and American children using the Chinese version of the CSBS‐DP, suggesting that cultural differences might have been a potential factor influencing children's scores. However, the upper age limit of the CSBS‐DP is 24 months, and thus it is not designed to evaluate the pragmatic competence of preschool‐aged children. On the other hand, for the development of the CCDI in Mandarin Chinese, Hao et al. (2008) collected data regarding the communicative development of Chinese infants and toddlers aged 12–30 months from 884 Chinese families in Beijing. Although the age range of the CCDI is wider than that of the CSBS‐DP, it is designed primarily to assess vocabulary and grammar rather than pragmatics.

The LUI is used to measure the social pragmatic communication of children aged 18–47 months. It is composed of three parts: (1) how children use gestures to communicate; (2) how children use words to communicate; and (3) how children use longer sentences and stories. Ten of the subscales (comprising 161 of the 180 items) contribute to the LUI total score. The remaining four subscales ask about gestures and the child's interests in play and talk. The applicable age with the LUI is broader than that with the CSBS‐DP and the CCDI. Its content focuses on uses of language that emerge as a joint function of language development and children's developing social cognitive understanding that takes place during the toddler and preschool years (O'Neill, 2007). Moreover, in contrast with criterion‐referenced measures, the LUI was normed on a large Canadian sample (N = 3653) and provides norms for every month from 18 to 47 months (O'Neill, 2009). The LUI mainly focuses on the use of language in social interactions rather than on vocabulary and grammar. As most LUI items do not focus on the child's production of specific words, instead they ask about language use more generally, followed by examples of what a young child might say. These examples are intended to help parents recognize and focus on the purpose or function of their child's language (rather than on its form). Additionally, the LUI is a parent‐report questionnaire taking into account that parents and/or caregivers have interacted with the child for a long time, and they have extensive experience regarding their child's language skills in a variety of natural contexts and everyday situations. Parent‐report measures enable a more comprehensive assessment of the child's pragmatic language competence, increasing the reliability of the information about the child's communicative behaviour occurring in everyday communication (Law & Roy, 2008; Boudreau, 2005), which surpasses structured testing (Hirsh‐Pasek et al., 2005). In addition, another advantage is that parent‐report measures, compared with informal or naturalistic assessment methods, possess greater ecological validity (Lund & Duchan, 1983). The LUI is widely used in clinical practice in Canada, the United States and all English‐speaking countries both for assessment and to guide intervention. Portuguese (Guimarães et al., 2013), Italian (Longobardi et al., 2017), French (Pesco & O'Neill, 2016), Polish (Biaecka‐Pikul et al., 2019) and Norwegian (Helland & Mllerhaug, 2020) adaptations, all with good psychometric properties, are also available. The cross‐cultural versions of the LUI have underscored more similarities than differences (e.g. similar Cronbach's alpha for the main parts of the LUI and similar developmental trajectories by age and sex) in children's pragmatic language development across different countries and languages.

Research on the original LUI‐English has shown good to excellent internal reliability and well as test–retest reliability. In addition, the LUI was demonstrated to distinguish children with and without language delay with high sensitivity and specificity (O'Neill, 2007). Children's scores on the LUI were strongly correlated with their scores on a subset of scales from the CSBS (Wetherby & Prizant, 1993), supporting its concurrent validity (O'Neill, 2009). Furthermore, a study of the LUI's predictive validity (Guimarães et al., 2013) further showed that children's LUI scores predicted later language outcomes at age 5–6 years on a battery of laboratory‐administered standardized language assessments (e.g. Clinical Evaluation of Language Fundamentals—Preschool, 2nd Edition—CELF‐P2; Wiig et al., 2004).

ASD is a common, highly heritable and heterogeneous neurodevelopmental disorder characterized by deficits in social communication and social interaction and the presence of restricted, repetitive patterns of behaviour, interests or activities present during early periods of development (American Psychiatric Association, 2013). Pragmatics is frequently demonstrated to be a key component of the difficulties exhibited by children diagnosed with ASD (Baixauli‐Fortea et al., 2017; Bauminger‐Zviely et al., 2013; Dennis et al., 2001; Sturrock et al., 2020; Tager‐Flusberg & Anderson, 1991; Ying Sng et al., 2018). Individuals with ASD also show developmental differences in related social pragmatic communicative behaviours, such as the use and understanding of gestures and attention to social partners and common topics (Mundy et al., 1990). In addition, approximately 30% of autistic individuals may demonstrate only minimal verbal communication (Tager‐Flusberg & Kasari, 2013). For children with ASD who develop language abilities, the previously described pragmatic difficulties persist and can include fewer and often less skilled attempts at communication, a narrower range of communicative behaviours and difficulties in learning new languages (Aldred et al., 2004). In research studies, the LUI has been used to assess the pragmatic competence of many different clinical groups of children (Abbott‐Smith et al., 2015; Miller et al., 2014; Pesco & O'Neill, 2012; Tager‐Flusberg et al., 2009). It is one of the most commonly used tools to assess pragmatics in children with ASD (Bland‐Stewart et al., 2013), in part due to the recommendation by an expert panel as a means to assess the expressive language progress of children with ASD in the context of interventions (Ingersoll et al., 2005).

Currently, there are no standardized measures of pragmatics for Chinese‐speaking toddlers and preschoolers, so it is of great significance and value to translate the pragmatic tools widely used around the world into Mandarin Chinese and ensure that these translations incorporate any cultural adaptations for use in China. Therefore, the purpose of this study was to develop a version of the LUI in Mandarin for use in China, to test its reliability, to explore its ability to distinguish typically developing (TD) and autistic children and to detail the developmental growth of Chinese children's pragmatic competence.

MATERIALS AND METHODS

Translation and adaptation process

After acquiring permission from the original author, translation and cross‐cultural adaptation processes were conducted following the guidelines the original author had provided. Two graduate students whose mother tongue was Chinese independently translated the English LUI into Mandarin Chinese. The graduate students were selected based on their background knowledge of child development (i.e., major in child psychology and work with children) and fluency levels in Mandarin and English. Translation typically follows back or forward translation. In back translation, the source is translated to a target language by one translator and then translated back to the source language by a second translator. Back translation can be difficult because you may not know if the incongruence is due to poor back translation or an actual issue with an item. It can also encourage overly literal translations if the initial translator is aware that back translation will follow; consequently, translated items might sound less natural or be more puzzling than the source version (American Psychiatric Association, 2005; Zucker et al., 2005). Forward translation, which involves multiple rounds of assessing the draft questionnaire with a few people in different user groups, such as parents, speech–language pathologists and language researchers, allows revisions to be made accordingly. Forward translation is now generally favoured (American Psychiatric Association, 2005; Zucker et al., 2005) and was adopted in the present study.

These translations included all instructions, subscales and subscale items. A consensus committee comprising the two translators, six researchers from psychology and psychiatry, a linguistics professor and the principal investigator reviewed the translations item by item. The committee discussed and made suggestions regarding the wordings used and adjusted the words and sentences of the translated version to make the meaning of the items as accurate and clear as possible. We stuck to the original format in the vast majority of cases. Nonetheless, because of the linguistic, grammatical and cultural structural differences in the Chinese context, some of the item wordings were adapted (for all changes, see Appendix A). One minor modification was performed on item 2 of subscale A (put a toy or book in your lap, or climb into your lap with a toy), and the phrase ‘when he/she wants you to accompany’ (想让您陪他/她时) was added in front of the sentence to make it easily understandable by giving background information. Parents might make an incorrect choice based on their child's ability to put a book or climb into their lap rather than focusing on their child's ability to use gestures to express a need. In addition, ‘me’ was converted into ‘you’ (您) on several items (items 20 and 21 of subscale C; items 5 and 6 of subscale F) to keep personal pronouns consistent in context while preserving the meaning of the terms from the original English version. Additionally, to make the terms easily understandable by the general Chinese population, we adopted Chinese names as ‘xiao ming’ (小明) and ‘xiao hong’ (小红) instead of the original English names on items 1, 8 and 16 of subscale H.

TABLE A1

Adaptions of items in the LUI‐Mandarin

Items of the original LUI‐English	Altered items of the LUI‐Mandarin (changes are shown in italics)	Altered items translated into Chinese (changes are shown in italics)
Subscale A
2. put a toy or book in your lap, or climb into your lap with a toy	2. put a toy or book in your lap, or climb into your lap with a toy (When he/she wants you to accompany)	2. (想让您陪他/她时) 把玩具或书放在您的腿上, 或者带着玩具爬到您的腿上
Subscale C
20. it is fairly easy for me to teach my child a new word	20. it is fairly easy for you to teach your child a new word	20. 教孩子一个新词语对您来说相当容易
21. it is fairly easy for me to know when my child and I are both talking about the same thing	21. it is fairly easy for you to know when your child and you are both talking about the same thing	21. 当孩子和您在说同一件事情时, 您很容易就明白
Subscale F
5. my child uses words to ask me to look at him/her, or at what he/she is doing	5. your child uses words to ask you to look at him/her, or at what he/she is doing	5. 孩子会通过语言让您看着他, 或看他在做什么
6. my child uses words to ask me to look at something he/she is interested in	6. your child uses words to ask you to look at something he/she is interested in	6. 孩子通过语言让您看他所感兴趣的东西
Subscale H
1. what his/her own name is (e.g. My name's Alicia; I'm Brendan.)	1. what his/her own name is (e.g. My name's xiao ming; I'm xiao hong.)	1. 他的名字叫什么？ (如:我叫小明;我叫小红。)
8. what someone else wants or doesn't want (e.g. Ben wants the truck.)	8. what someone else wants or doesn't want (e.g. xiao ming wants the truck.)	8. 别人想要或不想要的东西 (如:小明想要卡车。)
16. how someone else is behaving (e.g. Jamie’s being mean.; That boy's nice.)	16. how someone else is behaving (e.g. xiao ming’s being mean.; That boy's nice.)	16. 其他人的行为 (如:小明太凶了;那个男孩真好。)

Five mothers were recruited to complete the LUI‐Mandarin for their children (aged 2–4 years) and to comment on the clarity, completeness and ease of responding to the questions. Feedback from the five mothers was positive; the LUI‐Mandarin was described as clear, comprehensive and easy to complete. Given their positive feedback, no further adjustments were made to the LUI‐Mandarin. As a result, the final LUI‐Mandarin version did not differ from the original LUI‐English in the number of parts, subscales or subscale items, and the LUI total score remained the same at 161.

Validation process

Materials

Like the original LUI, the LUI‐Mandarin comprises three main parts and 14 subscales. Two subscales (A and B) comprise Part 1 and assess a child's early use of gestures, which are included in the inventory for parents of children with very low (or no) expressive ability. If a child masters no words at all, parents are asked to stop following subscale B. Part 2 is comprised of three subscales (C–E) that assess a child's communication at approximately the one‐ to two‐word stage. Nine subscales (F–N) comprise Part 3 and assess a variety of different uses of language that appear once children start to use longer sentences.

The scoring of the LUI‐Mandarin items followed the scoring procedure of the original LUI (O'Neill, 2009). A child's total LUI‐Mandarin score was calculated from the sum of all items in the subscales of Parts 2 and 3 (161 items, excluding subscales E and L). The information provided on subscales E and L may be of particular value when evaluating a child for further in‐depth assessment for the possible presence of a pervasive developmental disorder (PDD). ‘Yes’ responses and frequency ratings of ‘sometimes’ or ‘often’ were awarded 1 point. ‘No’ responses and frequency ratings of ‘never’ or ‘rarely’ or ‘not any more’ (the latter for gestures) received a score of 0 in accordance with the LUI Manual (O'Neill, 2009).

In addition to the main part of the LUI‐Mandarin, parents provided information about children's substantive birth complication(s) or other major health problem(s) generally recognized as resulting in delay or impairment in language and/or cognitive development and exposure to other languages. Exposure to a second language exceeding 20% of waking hours was established as the threshold for exclusion. With respect to how missing data were handled, a questionnaire was excluded if, among the 10 scored subscales in Parts 2 and 3 (subscales C, D, F–K, M and N) making up a child's LUI total score, a response was missing for two items within one of these 10 subscales or a response was missing for more than two items across all these 10 subscales (O'Neill, 2009).

Participants and procedures

This study was approved by Nanjing Brain Hospital affiliated with the Nanjing Medical University Ethics Committee (certificate number 2019‐KY021‐01). Parents or legal guardians who agreed to participate in the study signed the informed consent form.

For TD individuals, data were collected through daycare centres and health centres for women and children from Nanjing and Wuxi, both of which are in southern Jiangsu province of China. Researchers introduced the purposes of the study to the parents or legal guardians of TD children, and they were asked to complete the LUI‐Mandarin in a single day if possible, or two at most. To participate, a child had to be in the age range of 18–47 months, irrespective of days. The inventory needed parents to provide demographic information and to respond to questions about their child's health, development and language exposure. Responses to these questions were used to implement exclusionary criteria applied in our study, similar to those used with the original LUI‐English (O'Neill, 2009). Specifically, children were excluded from the TD group if they were (1) exposed to a language other than Chinese more than 20% of the time since birth; (2) born 2 or more weeks prematurely and had low birth weight (< 2500 g); (3) diagnosed with a language delay, developmental delay, hearing impairment, autism or medical condition likely to affect language development; or (4) receiving any type of intervention.

The questionnaires were distributed to 219 parents of TD children between the ages of 18 and 47 months. In total, 189 (86.30% return rate) of the questionnaires were returned. All the TD children had acquired Chinese as their first language, although 35% were reported to be exposed to a second language (English) in the home but fewer than 20% of their waking hours. After a review of the returned questionnaires, 12 were dropped for the following reasons: medical reasons (n = 2) and missing data (n = 10). This resulted in a final sample size of 177 participants (age range = 18–47 months; male to female ratio = 88:89) (see Table 1 for sample characteristics). The age range was divided into five 6‐month intervals (18–23, 24–29, 30–35, 36–41 and 42–47 months) for further data analysis. Of the 177 participants, 31 parents completed the questionnaire twice within 4 weeks of completion of the initial time 1 questionnaire to examine test–retest reliability.

TABLE 1

Descriptive characteristics of all TD participants

Age group (months)	Mean age (months)	Boys/girls	Total
18–23	19.26	22/17	39
24–29	26.77	17/14	31
30–35	32.18	13/20	33
36–41	38.31	16/20	36
42–47	45.44	20/18	38
Total		88/89	177

For the patient group, individuals with ASD were recruited via Nanjing Brain Hospital affiliated with Nanjing Medical University. The diagnoses for ASD were based on the Diagnostic and Statistical Manual of Mental Disorders—Fourth Text—Revision (DSM‐IV‐TR) diagnostic criteria and were supported by standardized clinical assessments, including the Autism Diagnostic Observation Scale (ADOS) (Lord et al., 1989) and Autism Diagnostic Inventory—Revised (ADI‐R) (Lord et al., 1994). Children with ASD were excluded if they were (1) exposed to a language other than Chinese more than 20% of the time since birth; (2) born 2 or more weeks prematurely and had low birth weight (< 2500 g); or (3) receiving any type of intervention. Questionnaires were initially distributed to 60 parents of the ASD children, and 43 (age range = 18–47 months; male to female ratio = 29:14) were returned (71.67% return rate). A total of 17 questionnaires were dropped for the following reasons: prematurity (n = 1), missing data (n = 3), received language intervention (n = 5) and older than 47 months (n = 8).

Statistical analysis

All statistical analyses were performed using Statistical Product and Service Solutions (SPSS, version 22.0). Internal consistency was analysed using Cronbach's alpha coefficients, and values > 0.70 were considered acceptable. Construct validity was analysed by exploratory factor analysis (EFA) with varimax rotation. The LUI‐Mandarin total score was analysed with univariate analysis of variance (ANOVA) for age then sex. This strategy was preferred to two‐way ANOVA (crossing age and sex) because univariate procedures in SPSS accommodate heterogeneous variance across the levels of a factor. Such heterogeneity was present in our sample, and Levene's test for equal variances confirmed that the homogeneity of variance assumption for age was not met (p = 0.004). Independent t‐tests were conducted to compare the LUI‐Mandarin total scores of boys and girls at each age. Pearson correlation analysis was conducted to demonstrate the intercorrelations among the subscale scores. The test–retest reliability was assessed by the Pearson correlation coefficient. Additionally, independent t‐tests were conducted to compare the LUI‐Mandarin total and subscale scores between the ASD group and the TD group to show the discriminant validity. Moreover, discriminant function analysis was conducted with age taken into account. p‐values < 0.05 (two‐sided) were considered as being statistically significant.

RESULTS

Internal reliability

Coefficient alpha values were calculated for all three parts and all subscales of the inventory. The resulting alpha values are shown in Table 2. The coefficient alpha values of each of the three parts and 8 of the 12 scored subscales met the clinically most desirable level of 0.9 or above. The alpha values for all remaining subscales were all above acceptable levels of 0.70. The alpha values for Parts 2 and 3 combined, which comprised a child's total score on the LUI‐Mandarin, was 0.99.

TABLE 2

Cronbach's alpha for all parts and subscales of the LUI‐Mandarin

LUI parts and subscales	Cronbach's alpha	Number of items
Part 1: How your child communicates with gestures	0.903	13
A: How your child uses gestures to ask for something	0.915	11
B: How your child uses gestures to get you to notice something	0.737	2
Part 2: Your child's communication with words	0.938	28
C: Types of word's your child uses	0.935	21
D: Your child's requests for help	0.707	7
E: Your child's interests	Not scored	2
Part 3: Your child's longer sentences	0.992	133
F: How your child uses words to get you to notice something	0.745	6
G: Your child's questions and comments about things	0.943	9
H: Your child's questions and comments about themselves/other people	0.981	36
I: Your child's use of words in activities with others	0.929	14
J: Teasing and your child's sense of humour	0.825	5
K: Your child's interest in words and language	0.920	12
L: Your child's interests when talking	Not scored	4
M: How your child adapts conversation to other people	0.952	15
N: How your child is building longer sentences and stories	0.975	36
Total score	0.992	161

Factor analysis

EFA was conducted to identify the dimensions and examine the construct validity of the inventory. Before performing EFA, the Kaiser–Meyer–Olkin (KMO) test and Bartlett's test of sphericity were calculated to assess the suitability for EFA. The results of the KMO test (KMO = 0.938) and Bartlett's test of sphericity (F = 2298.75, p < 0.001) demonstrated the reasonability of conducting EFA. An EFA with varimax rotation supported a two‐factor solution, with the first factor corresponding to the 10 subscales of Parts 2 and 3 assessing verbal communication (eigenvalue = 7.658) and a second factor corresponding to the subscales A and B in Part 1 (eigenvalue = 1.267). The first factor explained 63.82% of the variance, while the second factor explained an additional 10.56% of the variance, and the total variance explained was 74.38%. No subscales overlapped in their loadings, and all loadings for the first factor were > 0.726.

Growth with age across subscales

ANOVA showed an effect of age group on LUI‐Mandarin total scores: Welch's F = 68.24, p < 0.001. Figure 1 displays increases in total scores by age group. Post‐hoc comparisons were conducted following ANOVA using Dunnett's T3 procedure (as appropriate in the case of unequal variance) and are summarized in Table 3. As shown, generally the younger groups differed from the older groups, as anticipated, with several exceptions: the mean scores of 24–29 were not significantly different from those of 30–35, the mean scores of 30–35 were not significantly different from those of 36–41, the mean scores of 36–41 were not significantly different from those of 42–47, but in all cases the differences were in the expected direction (i.e. lower in the lower age group). ANOVA showed an effect of age on LUI‐Mandarin subscale scores with the exception of subscale B: for subscale A, Welch's F = 4.56, p = 0.002; for subscales C–D and F–N, Welch's F = 6.45–61.89, p < 0.001. Figure 2 provides the LUI‐Mandarin subscale scores expressed as percentages to facilitate comparisons across subscales with differing numbers of items. As shown, scores on gesture subscale A decreased with age. For subscales C–D and F–G, scores increased from 18–23 to 30–35 months and then reached or approached the ceiling. For the remaining subscales H–K and M–N, scores increased up to 42–47 months. The results of Dunnett's T3 post‐hoc tests identified which of the five age groups differed significantly in their performance on each subscale (see the summary in Table 4).

FIGURE 1

Language Use Inventory (LUI)‐Mandarin total score by age group

TABLE 3

Post‐hoc comparisons of LUI‐Mandarin total score by age group

Age group (months)	Number	N	Mean (SD)	Significant differences
18–23	39	51.05	29.01	24–29; 30–35; 36–41; 42–47
24–29	31	98.65	38.59	18–23; 36–41; 42–47
30–35	33	121.58	25.09	18–23; 42–47
36–41	36	136.67	25.01	18–23; 24–29
42–47	38	143.53	23.63	18–23; 24–29; 30–35

Note: The final column indicates significant differences between scores of children in the age group for that row and the age groups denoted.

FIGURE 2

LUI‐Mandarin subscale scores (%) by age group (months)

Note: Full names of the subscales are provided in Table 2. Error bars are 95% CI.

TABLE 4

Summary of Dunnett's T3 analyses showing, for each age group, the older age groups with a significantly different mean score on a particular subscale

Comparison age groups	Subscales with significantly different mean scores
18–23 versus 24–29	Subscales C, D, F–K, N
30–35	Subscales C, D, F–K, M, N
36–41	Subscales C, D, F–K, M, N
42–47	Subscales A, C, D, F–K, M, N
24–29 versus 30–35	Subscale M
36–41	Subscales G, H, K, M, N
42–47	Subscales F–H, K, M, N
30–35 versus 36–41	Subscale N
42–47	Subscales H, K, N
36–41 versus 42–47	–

Sex differences

As displayed in Figure 3, girls scored significantly higher than boys at the ages of 18–23 months; t = –2.82, p = 0.008 (two‐tailed t‐test); girls also scored higher than boys at the ages of 36–41 months, but this difference was not significant: t = –0.76, p = 0.45. Boys’ mean total scores appeared higher at the ages of 24–29, 30–35 and 42–47 months but were not significantly different from girls’ mean total scores: 24–29 months, t = 0.49, p = 0.63; 30–35 months, t = 0.51, p = 0.61; and 42–47 months, t = 0.85, p = 0.41.

FIGURE 3

LUI‐Mandarin total score by age group and gender

Note: Error bars are 95% CI. *p < 0.05.

Intercorrelations among the subscales

Table 5 shows the intercorrelations among all 12 subscales, controlling for age (months). The scores on subscales A and B were significantly positively correlated (r = 0.214, p < 0.05). Apart from this correlation, subscales A and B showed no significant correlations with any of the other subscales, which is consistent with the factor analysis results. Regarding the remaining 10 language subscales of the inventory, the intercorrelations among the scores were all significantly and positively correlated (r = 0.260–0.855, p < 0.001), reflecting the excellent interrelatedness of the language subscales of the LUI‐Mandarin.

TABLE 5

Intercorrelations between all scored LUI‐Mandarin subscales from the typically developing (TD) sample (N = 177) controlling for age

	A	B	C	D	F	G	H	I	J	K	M
B	0.214*
C	−0.043	0.089
D	−0.026	0.174	0.644**
F	−0.039	0.051	0.675**	0.585**
G	−0.052	0.039	0.634**	0.634**	0.740**
H	−0.085	0.085	0.571**	0.619**	0.714**	0.855**
I	−0.077	0.092	0.466**	0.548**	0.642**	0.708**	0.797**
J	−0.167	0.014	0.319**	0.331**	0.449**	0.514**	0.555**	0.583**
K	−0.098	0.035	0.400**	0.460**	0.546**	0.629**	0.677**	0.722**	0.703**
M	−0.020	0.040	0.260**	0.330**	0.434**	0.516**	0.567**	0.626**	0.623**	0.617**
N	−0.109	0.018	0.320**	0.313**	0.522**	0.439**	0.584**	0.630**	0.640**	0.658**	0.612**

Note:

p < 0.05.

p < 0.001.

Test–retest reliability

The difference between the first and the second completion of the LUI‐Mandarin by parents was determined with t‐tests in independent groups. Table 6 shows significant increases in scores on all subscales from C to N within the short 4‐week period. Test–retest correlations were provided using Pearson correlation coefficients, and test–retest stability and reliability were excellent, with all subscales demonstrating significant Pearson correlations between the test and retest scores that ranged from 0.66 to 0.95 (p < 0.001), with the exception of subscale B.

TABLE 6

Mean score differences and test–retest Pearson correlation coefficient between Time 1 and Time 2 from the typically developing (TD) sample (N = 31)

Subscale	Time 1 mean (SD)	Time 2 mean (SD)	t	p	Pearson correlation coefficient
A	9.00 (2.60)	9.23 (3.26)	0.53	0.60	0.70**
B	1.84 (0.52)	2.00 (0.00)	1.72	0.10	–
C	18.77 (4.09)	19.45 (3.52)	2.19	0.04*	0.91**
D	6.36 (0.95)	6.52 (1.18)	0.90	0.38	0.68**
F	4.90 (1.35)	5.13 (1.48)	1.07	0.29	0.66**
G	6.84 (3.05)	6.90 (3.40)	0.20	0.85	0.85**
H	23.10 (12.57)	24.48 (13.62)	1.12	0.27	0.86**
I	9.52 (4.46)	9.97 (4.72)	0.86	0.39	0.80**
J	2.29 (1.92)	2.58 (1.88)	1.66	0.11	0.87**
K	7.32 (4.21)	7.45 (4.47)	0.26	0.79	0.81**
M	8.29 (5.69)	9.07 (6.03)	1.40	0.17	0.86**
N	14.52 (12.04)	16.19 (12.58)	2.36	0.02*	0.95**

Note:

p < 0.05.

p < 0.001.

Discriminant validity

When the total scores for the ASD group were compared with those for the TD group, the difference was strikingly large. Figure 4 shows the total scores of the 43 age‐ and sex‐matched child pairs in order of increasing age. The mean total score for the ASD group was 28.44 (SD = 34.88) compared with 115.91 (SD = 36.08) for the TD group. This difference was significant, t = –11.43, p < 0.001. Moreover, Figure 4 reveals that this difference remained relatively stable across the entire age range from 18 to 47 months. Figure 5 shows the mean performance for each group over all 12 scored subscales. With the notable exception of their mean performance on subscale A, the mean performance of the ASD group fell far below that of the TD group on all remaining 11 subscales (B–N), t = –5.29 to –13.27, p < 0.001. Results of a discriminant function analysis using total scores, conducted with age taken into account, correctly classified 90.7% of the original 86 individuals, sensitivity was 88.4% and specificity was 93%, which strongly supported the discriminant validity of the LUI‐Mandarin.

FIGURE 4

Total scores on the LUI‐Mandarin plotted against age (months) for the 43 age‐ and sex‐matched children in the ASD and TD groups

FIGURE 5

Mean performance of the 43 children in ASD and age‐ and sex‐matched TD group on all 12 scored subscales of the LUI‐Mandarin

DISCUSSION

The goal of this study was to adapt the original English version of the LUI to Mandarin Chinese and investigate its reliability, validity and developmental sensitivity in Chinese TD and ASD toddlers and preschool children. The LUI‐Mandarin was developed through strict translation processes and cross‐cultural adaptations taking into account the cultural and environmental characteristics of China. The psychometric evaluation of the LUI‐Mandarin indicated good reliability, validity and developmental sensitivity in the Chinese context.

The Cronbach's alpha values indicated adequate to excellent internal reliability: all values were in the 0.707–0.992 range (most were > 0.90), which is higher than the cut‐off threshold of 0.70 (Bland & Altman, 1997; Sharpe, 2004), meaning that items are homogenous and that they conjointly measured the same underlying construct. Analysis of the alpha coefficients indicated that the results obtained in the translated version (LUI‐Mandarin) were similar to the original version, with good reliability. This proximity is explicit with respect to the total score of Parts 2 and 3 (161 items), where the alpha coefficient was 0.99 for both the original LUI‐English (O'Neill, 2007) and LUI‐Mandarin. Regarding the factor analysis, EFA with varimax rotation was conducted. The results indicated, as was found for the original LUI‐English, that separating the gesture subscales from the language subscales is supported. The loading of subscales on both factors implied that the LUI‐Mandarin subscales evaluated the same underlying construct of pragmatics. The correlations between the language subscales presented in Table 5 were positive and strong, reflecting close interrelatedness. In contrast, the gesture subscale A correlated negatively with all the language subscales, demonstrating the trend for gestures to decrease as language increases. Likewise, the gesture subscale B did not correlate significantly with the language subscales as pointing and showing remain across the age range, and it has only two items, similar to the results for the LUI‐English (O'Neill, 2007), LUI‐French (Pesco & O'Neill, 2016), LUI‐Norwegian (Helland & Mllerhaug, 2020) and LUI‐Arabic (AlKadhi, 2015).

The effects of age on the LUI‐Mandarin total and subscale scores were also examined. There was a significant main effect for age on all subscales except for subscale B. The differences in total scores were not significant between the age groups of 24–29 versus 30–35, 30–35 versus 36–41, and 36–41 versus 42–47 months, but the scores were in the expected direction, with each older age group's total score increasing above the previous group's score. It is possible that with a larger sample, as planned for a norming study, that significant differences may appear between more of these age groups. The increases in the total scores from the 18–23‐ to the 24–29‐month age groups were the most dramatic. Our findings were consistent with those obtained with the LUI‐French (Pesco & O'Neill, 2016) and confirmed by the norming study (N = 3563) of the original LUI‐English (O'Neill, 2009), which also reported that the increases in the total scores were the most dramatic from 18 to 30 months, and the increases were not as large after that point.

Our results showed that the subscale scores assessing verbal language use in Parts 2 and 3 increased linearly with age, suggesting a development toward more sophisticated pragmatic skills. For subscales C–G, scores approached the ceiling at 30–35 months, while for subscales H–N, scores increased up to 42–47 months. This pattern was not surprising since the subscales were developmentally ordered in the original LUI‐English. In contrast, for the two gesture subscales in Part 1, scores decreased with increases in age. Iverson et al. (1994) found that gestures were more prevalent in children's communication at 16 months, but by 20 months, a clear shift toward a preference for verbal production was observed. Consistent with their work, our findings also suggested that the enhancement of expressive use of language paralleled a reduction in gestural engagement. Overall, our findings demonstrate that the LUI‐Mandarin can be used as a means of assessing and evaluating the development of pragmatics in toddlers and preschool children.

In the present study, we found sex differences at 18–23 months on the LUI‐Mandarin total scores, and the differences were no longer significant after that period. A systematic literature review (Etchell et al., 2018) also confirmed the existence of sex differences in communication, language and speech development among TD children, and all significant effects were in favour of girls. In addition, Lange et al. (2016) found that the advantage in language competence for females decreased with age. Previous adaptations of the LUI to other languages showed that girls scored higher than boys in the age range of 24–36 months, but in the older age groups, the difference disappeared with the LUI‐French (Pesco & O'Neill, 2016). Similarly, regarding the LUI‐Italian (Longobardi et al., 2017), girls scored higher than boys in the younger age groups (24–35 months), while the opposite pattern was observed in the oldest age groups (42–47 months). However, sex differences were not evident in all age groups in the LUI‐Arabic (AlKadhi, 2015) and LUI‐Norwegian (Helland & Mllerhaug, 2020). Regarding the results with the original LUI‐English (O'Neill, 2009) that was normed on a large sample, significant sex differences were found in the direction of higher performance by girls. Findings of a much larger ongoing standardization study will more accurately determine whether and when sex differences exist with respect to LUI‐Mandarin total scores.

In a previous study, preschool‐age siblings of children with ASD scored lower on the LUI than siblings of TD children, confirming a risk of social pragmatic communication deficits in the ASD‐sibling group (Miller et al., 2014). Consistent with this research, our study also showed that children with ASD scored lower on the LUI‐Mandarin than TD children, supporting the strong discriminant validity of the LUI‐Mandarin. Additionally, our results of the discriminant function analysis showed that the overall correct classification rate was comparable to the classification rate with the original LUI‐English (O'Neill, 2007).

CONCLUSIONS AND LIMITATIONS

Alpha coefficient values for the subscales very strongly confirmed their internal consistency and reliability. Test–retest stability and reliability were also strongly supported. In addition, the discriminant validity of the LUI‐Mandarin was strongly supported by its ability to distinguish between TD children and a clinical sample of children with ASD. In addition, our study's inclusion of children across the full age range from 18 to 47 months, similar to the ages tested with the original LUI‐English, led to a more comprehensive picture of the developmental sensitivity of the LUI‐Mandarin. Overall, our preliminary results suggest that the LUI‐Mandarin is a promising tool for assessing pragmatics in early childhood. Nevertheless, the study also had some limitations. First, our study did not establish the concurrent validity of the LUI‐Mandarin by examining the relationship between children's performance on the LUI‐Mandarin and their performance on another standardized pragmatic competence test, such as the CSBS. Second, although our sample size is comparable to other studies exploring the validity and reliability of the LUI (Helland & Mllerhaug, 2020; O'Neill, 2007; Pesco & O'Neill, 2016), it is nonetheless relatively moderate. We intend to proceed with norming the instrument for clinical use, which will involve a larger sample in the future.

CONFLICT OF INTEREST

The authors (Lu Qian, Huan Shao, Hui Fang, Ting Xiao, Ning Ding, Bei Sun, HuiYun Gao, Min Tang, Mei Ye, XiaoYan Ke) declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The last author, Daniela O'Neill, is founder and president of Knowledge in Development, Inc., the company that holds the copyright to and publishes the original English version of the Language Use Inventory (LUI) commercially (https://languageuseinventory.com/). The company receives all proceeds from the LUI, and thus, Daniela K. O'Neill is a beneficiary of proceeds from the LUI. This is a continuing relationship.

FUNDING

This work was supported by the National Key Research and Development Program of China (grant number 2016YFC1306200), the National Natural Science Foundation of China (grant number 81771478) and the ‘Special Disease Cohort’ Research Project of Nanjing Medical University (grant number NMUC2018010A).