The Dutch well child language screening protocol for 2-year-old children was valid for detecting current and later language problems.

AIM: A little is known about predictive validity of and professionals' adherence to language screening protocols. This study assessed the concurrent and predictive validity of the Dutch well child language screening protocol for 2-year-old children and the effects of protocol deviations by professionals.
METHODS: A prospective cohort study of 124 children recruited and tested between October 2013 and December 2015. Children were recruited from four well child clinics in urban and rural areas. To validate the screening, we assessed children's language ability with standardised language tests following the 2-year screening and 1 year later. We assessed the concurrent and predictive validity of the screening and of protocol deviations.
RESULTS: At 2 years, the sensitivity and specificity of the language screening were 0.79 and 0.86, and at 3 years 0.82 and 0.74, respectively. Protocol deviations by professionals were rare (7%) and did not significantly affect the validity of the screening.
CONCLUSION: The language screening protocol was valid for detecting current and later language problems. Deviations from the protocol by professionals were rare and did not affect the concurrent nor predictive validity of the protocol. The 2-year language screening supports professionals working in preventive child health care and deserves wider implementation in well child care.

American Academy of Pediatrics

MacArthur‐Bates Communicative Development Inventories

Language Development Survey

Schlichting tests for Language Comprehension

Schlichting tests for Sentence Production

Schlichting tests for Word Production

Little is known about predictive validity of language screening protocols and effects of protocol deviations by professionals.

The Dutch 2‐year language screening protocol identified children with current and later language problems accurately, with rather few protocol deviations by the professionals.

The 2‐year language screening supports professionals working in preventive child health care and deserves wider implementation in well child care.

INTRODUCTION

Problems in speech and language are one of the most reported developmental problems in children with an estimated prevalence of 7%. These problems may impact children's emotional functioning, academic success and social relationships , and early detection and subsequent treatment may significantly reduce their impact. Therefore, the American Academy of Pediatrics has recommended developmental surveillance and regular screening of children in order to identify developmental problems early.

According to the American Academy of Pediatrics' developmental screening recommendations, well child care professionals, such as nurses and paediatricians, require valid language screening protocols. Some of these are specific for language, such as the MacArthur‐Bates Communicative Development Inventory and Language Development Survey. There are also screening tools for the whole development of young children, such as the Ages and Stages Questionnaire, but, none of them include guidelines for the referral of children who failed the screening. Additionally, follow‐up was not evaluated for any of the screening protocols. Evidence on the concurrent validity of such screening protocols is frequently reported and expressed in sensitivity and specificity in a research setting. , Sensitivity is the percentage of individuals with a health condition, in this case children, who experience speech and language problems and are correctly identified as having the condition. Specificity is the percentage of healthy people who are correctly identified as not having the condition. Concurrent validity regards the degree to which the results of the screening test are parallel with a reference standard that is administered at the same time. However, evidence is scarce for the performance of a screening instrument regarding health outcomes in the long term, the so called predictive validity. This lack of evidence implies that the efficacy of protocols to monitor the development of a child over time is not known. Moreover, there is insufficient evidence on the effect of protocol deviations of professionals in daily practice, a setting in which the sensitivity and specificity of the screening may differ from the research settings. For example, a professional refers a child with a negative screening outcome because the professional has raised concerns regarding the language development of the child or is requested to do so by the parents. A clinical decision by the professional that deviates from the screening protocol might improve or worsen the validity of the protocol. Evidence regarding the concurrent and predictive validity of a language screening protocol in routine practice is thus highly needed.

In the Netherlands, well child care professionals monitor the development of children using the Van Wiechenschema. It is a brief schedule that aids a professional in monitoring a child’s development regarding communication, fine motor skills, gross motor skills, adaptive behaviour, social behaviour, and personality from birth to the age of 54 months. The skills that the child is expected to have mastered at specific ages are elicited in 2.5‐3.0 minutes by using a small amount of materials. However, guidelines for referral are only available for children from birth to 12 months. For children aged 12 months and over, a referral depends on the interpretation of the assessor. To assist assessors in the identification and referral of children with possible language problems, a practice‐based language screening protocol was implemented in well child care by the Dutch preventive child healthcare services. This protocol consists of tasks and questions to assess children's attainment of developmental language milestones and is performed by trained well child paediatricians and nurses. It provides a clear cut‐off score with directions for referral in <10 minutes. However, the concurrent and predictive validity of this language screening protocol as well as the frequency and effect of deviating professionals' clinical decisions are not known. Evaluation of this protocol and the effect of protocol deviations provide insights that can contribute to the optimisation of language screening in well child care in other countries. Therefore, the aim of this study was to assess the concurrent and predictive validity of the Dutch well child language screening protocol for 2‐year‐old children and the effects of protocol deviations by professionals.

METHODS

Sample and procedure

This was a prospective cohort study. We obtained the sample from four well child clinics that had implemented the screening protocol in their routine procedures. The clinics were selected from urban and rural areas in order to obtain a representative sample for the country as a whole. Well child paediatricians and nurses of these clinics asked all parents of children failing the screening to participate in the study. For each child failing the screening that was recruited, a gender‐matched child that passed the screening was recruited as well.

All of the participating children had Dutch as their first language. The study was conducted between October 2013 and December 2015.

Protocol language screening at 2.0 years of age

The purpose of the 2‐year screening is to detect language delays in children by assessing the children's ability to use two‐word sentences and comprehend basic, common nouns. It is administered by trained well child nurses and paediatricians and concerns the observation of two milestones, specifically, production of a two‐word sentence and pointing out five body parts on a doll. If the professional cannot observe the desired behaviour during the well child visit, the professional asks the parent about this in a standardised manner. Both items are scored with a maximum of two points for a child’s correct response. There will be no referral if a child achieves four points. If a child scores less than four points, an additional question is asked with regard to the child's playing behaviour. If the child plays together with the parent and can also play alone, one additional point is added. Any child with fewer than four points is referred. A total score of 0‐1 points results in a referral to an audiological centre for diagnostic assessment. A total score of 2‐3 points results in a referral for guidance by a preventive speech language pathologist or well child nurse and a follow‐up consultation is offered when the child is 2.5 years.

Procedure and measures

For the validation of the 2‐year language screening protocol, we assessed the language development of the child at age 2 years (concurrent validity) and at age 3 years (predictive validity). We used age appropriate standardised tests on language comprehension, word production, and sentence production. We defined atypical language as a deviant score on two or three tests or a moderate to severe deviation on one test. A reference standard was operationalised as follows: two or more test scores below minus one standard deviation of the norm score or one test score below minus one and a half standard deviation of the norm score resulted in atypical language. We used the Schlichting tests for Language Comprehension (SLC), Word Production (SWP) and Sentence Production (SSP) as reference tests. These are language tests for children from approximately 2‐7 years of age. The SLC is an 85‐item test assessing comprehension of grammatical constructions using toys, pictures, and tokens. The SWP is a 70‐item test to evaluate expressive vocabulary using a stimulus booklet with pictures. The SSP is a 40‐item test to determine expressive grammatical constructions by using imitation of expressions visualised in a stimulus booklet with, in some cases, associated toys. Age‐standardised scores for each test (mean = 100; standard deviation = 15) can be calculated according to the manuals in which entry levels per age and cut‐off rules are also described. , The SLC, SWP and SSP have excellent internal consistency (lambda‐2 = 0.93, 0.89 and 0.90, respectively) and demonstrate significant association with subtests of the Dutch version of the Clinical Evaluation of Language Fundamentals (0.63, 0.47 and 0.59 respectively) , .

A speech language pathologist (LD), who was blinded for the screening outcome, tested all of the children within 4 months, 90% within 2 months, of the 2‐year screening and 1 year later. Testing occurred during a home visit of, on average, 2 hours. Parents provided background information regarding birthweight and pregnancy duration.

The well child professionals reported the screening outcome and their clinical decision to the first author (MVB). We defined protocol deviations by professionals as the degree of adherence of the professional to the protocol and influence of deviations from the protocol on concurrent and predictive validity.

Analyses

First, we described the background characteristics of the sample. Next, we assessed the concurrent and predictive validity of the 2‐year language screening protocol and the effect of protocol deviations. We did so by calculating the sensitivity and specificity of the screening against the reference standard at the age of 2 years (concurrent validity) and at the age of 3 years (predictive validity). Next, we calculated the sensitivity and specificity of the clinical decision of the professional against the reference standard at the age of 2 years and at the age of 3 years. In addition, logistic regression analyses were performed with the reference standard at age two and at age three as the dependent variables and the screening result as the independent variable. These analyses were repeated using the clinical decision by the professional as the independent variable.

Ethics

The Medical Research Ethics Committee of Groningen (METc2013/103) approved the study and written, informed consent was obtained from parents or guardians of all children participating in this study.

RESULTS

Sample

In total, 124 children participated in the study at the age of 2 years. This regarded 61 children that were identified by the language screening protocol and 63 children that were not identified by the screening protocol. In advance, 132 parents agreed to participate in the study. However, eight parents withdrew from the study before the first test moment. This regarded five withdrawals from parents with children that were identified by the screening and three withdrawals from parents with children that were not identified by the screening. One child from the group that was identified by the protocol was absent in the follow‐up, resulting in a sample of 123 children at the age of 3 years of which 78% were males. The children identified by the protocol did not significantly differ from the children that were not identified by it on age at the first test moment (P = .85) and birthweight (P = 0.081) (Table 1). Groups differed for length of pregnancy (P = 0.021) as all of the three children born before 36 weeks were in the group with identified problems.

TABLE 1

Characteristics of the sample at baseline for children identified, and not identified by the Dutch 2‐y language screening protocol, and total sample

	Identified by screening protocol	Not identified by screening protocol	Total
	n = 61	n = 63	n = 124
Boys/girls (%)	50/11 (82/18)	47/16 (75/25)	97/27 (78/22)
Age at first test moment, months, mean (sd)	26 (1)	26 (1)	26 (1)
Birthweight, grams, mean (sd)	3300 (573)	3440 (514)	3370 (546)
Length pregnancy, weeks, mean (sd)	39 (2)	40 (2)	40 (2)

Concurrent and predictive validity of the 2‐year language screening protocol

A total number of 67 children (54% of the total sample) had atypical language at the age of two years according to the reference standard; 53 of these children (43% of the total sample) were identified by the screening protocol, and 14 children (11% of the total sample) were missed by the screening protocol (Table 2). One year after the screening, 50 children (41% of the total sample) had atypical language of which 41 children (33% of the total sample) were identified by the 2‐year screening protocol, and nine children (7% of the total sample) were missed (Table 2). The 2‐year screening compared to the reference standard at 2 years had a sensitivity of 0.79 and a specificity of 0.86 (concurrent validity). Regarding predictive validity, with reference testing 1 year later at age three, these values were 0.82 and 0.74, respectively (Table 3).

TABLE 2

Results of the 2‐y screening protocol and the speech language therapists' (SLT) reference standards at age 2 y and at age 3 y

		Identified by screening protocol n = 61	Not identified by screening protocol n = 63
Results of SLT reference	Atypical language	53 (43%)	14 (11%)
standard at age two	Typical language	8 (6%)	49 (40%)
Results of SLT reference	Atypical language	41 (33%)	9 (7%)
standard at age three	Typical language	19 (15%)	54 (44%)

Percentages refer to total percentage of total sample of n = 124 at age two years and n = 123 at age three years.

TABLE 3

Sensitivity, specificity for screening protocol with and without professionals' clinical decision and the reference standard at age two years (concurrent validity) and at age three years (predictive validity).

	2‐y standard: protocol	2‐y standard: protocol + professional	3‐y standard: protocol	3‐y standard: protocol + professional
Sensitivity	0.79	0.72	0.82	0.80
Specificity	0.86	0.86	0.74	0.79

Effect of protocol deviations

The professionals demonstrated strong adherence to the protocol. Their clinical decision deviated from it in only nine cases (7%). They did not refer seven children that were identified by the screening protocol and referred two children who were not identified by it (Figure 1). Deviations from the protocol had no added value regarding concurrent validity. We found minimal difference in specificity and sensitivity for the screening protocol compared to the clinical decision of the professional (Table 3). In accordance with findings for sensitivity and specificity, the logistic regression analyses also yielded quite similar findings for detection by the screening and detection by the clinical decision of the professional, slightly in favour of the screening (Table 4)

FIGURE 1

Screening, referral, and outcomes on the reference standards at age 2 y and age 3 y of all children (n = 124). Dark grey refers to atypical language established by the reference tests; white refers to typical language development according to these tests. *one child missing on follow‐up.

TABLE 4

Results from univariate logistic regression analysis on atypical language development at age 2 y and at age 3 y

Dependent: Language	Language problem at age 2 y Univariate (n=124)	P‐value	Language problem at age 3 y Univariate (n=123)	P‐value
Covariables	OR (95%CI)		OR (95%CI)
2‐y protocol	23.19 (8.95‐60.05)	<.01	12.95 (5.31‐31.56)	<.01
2‐y protocol + professional	15.47 (6.19‐38.71)	<.01	15.47 (6.32‐37.89)	<.01

Regarding predictive validity, 1 year after screening, again, we found hardly any difference in specificity and sensitivity for the screening protocol compared to the (deviations in) clinical decisions of the professional (Table 3). The odds ratios for detection by the screening protocol decreased from 23 to 13 and remained 15 for the clinical decision of the professional. Although all odds ratios differed significantly from 1, the 95% confidence intervals almost completely overlapped, indicating that the screening protocol and the professional's clinical decision had comparable predictive validity (Table 4). At an individual child level, the deviation of the professional was confirmed by the reference test after 1 year for six children and was not confirmed for three children.

DISCUSSION

The aim of this study was to assess the concurrent and predictive validity of the Dutch well child language screening protocol for 2‐year‐old children and the effects of protocol deviations by professionals. We found that the 2‐year language screening protocol had good concurrent and predictive validity. The professionals showed strong adherence to the protocol, and any deviations from it that were made by the professionals did not improve its sensitivity and specificity.

We found that the 2‐year language screening protocol had good concurrent validity, a sensitivity of 0.79; and a specificity of 0.86, confirming previous studies concluding that atypical language can be identified by using a combination of milestones at the age of two. , The two language milestones and one milestone regarding social interaction in play in the protocol thus seem to be adequate for a screening in well child care. The first language milestone in the protocol, says two‐word sentences, aligns with the international evidence that this milestone is strongly related to language problems. , , The second language milestone in the protocol, pointing out five body parts on a doll, accords with the red flag for immediate referral for evaluation at age 2 years, does not point to pictures or body parts when named. The finding on the protocol including a third milestone regarding the child's playing behaviour, concerning social interaction, confirms that interaction is part of language development. These three milestones are thus pivotal for language screening at age 2 years.

We further found a high predictive validity of the screening, in particular a sensitivity of 0.82 and a specificity of 0.74 after 1 year, confirming the few studies that reported that results of language screening at age 2 years indeed predicts later language status. However, in this meta‐analysis of Sim et al, the strongest overall predictive validity was reported for parent report measures compared to direct‐child‐assessment, whereas we found high predictive validity for a screening based on direct‐child‐assessment in combination with parent report. Reported predictive sensitivity and specificity for the MacArthur‐Bates Communicative Development Inventories 1 year after the screening was 0.61 and 0.94, respectively . For the Language Development Survey, the predictive sensitivity and specificity was 0.67 and 0.96 respectively, at an average of 23 days after the screening . Both parent report instruments showed high specificity with a moderate sensitivity. Our protocol had poorer specificity than the CDI and LDS but had better sensitivity 1 year after screening, indicating that our protocol showed a higher proportion of correct detections and had more over‐referrals than did the LDS and CDI. With our study we provide additional evidence for good predictive validity of language screening at 2 years based on a screening protocol performed by professionals. However, higher predictive specificity is desirable to minimise over‐referrals.

We found that the professionals showed strong adherence to the protocol and that deviations from it could not improve the sensitivity and specificity at ages two and three; but, a higher predictive specificity is desirable for community‐based screening. Such an improvement might be realised in two ways. A first option would be to add an extra step to the screening protocol that might improve its performance. Such an additional step could be to include the assessment of risk indicators, such as parental concerns or a family history of language problems , or to include a more extensive language screening with high specificity for flagged children. In the Netherlands, that could be a first‐stage diagnostic instrument, such as the Language Standard. Specificity should exceed 0.90 in order to make the protocol suitable for population based screening as this minimises over‐referrals with its negative effects, for example discomfort for parents and children as well as costs. A second option to improve specificity could be to extensively train professionals and increase the amount of time that they have for screening. Costs for training as well as time and implementation of a protocol in the workflow are barriers for successful screening that were noted in earlier studies. , The professionals participating in our study were already trained, and the protocol was implemented in the workflow which contributed to the success of our 2‐year screening protocol. Further research will be required to determine whether an extra step could improve sensitivity and specificity of the language screening protocol.

Our study had a number of major strengths, in particular that the actual referral by the professional could be assessed which provided insights into the validity of the protocol in routine practice. Also, the same extensive language tests were used in a longitudinal design, providing a sound reference standard for validation. Last, we achieved almost 100% retention, highly restricting potential bias in our study. A limitation of our study was that the sample was too small to differentiate the referred group into referral for further assessment or referral to guidance groups. This limited the potential to determine sensitivity and specificity for these graded referral rules, therefore, overall results should be interpreted with caution. Moreover, sensitivity and specificity were calculated within a predetermined sample and not one that was population based. Therefore, the positive and negative predictive values could not be estimated. Another limitation was that the well child professionals were not asked what, according to them, could improve sensitivity and specificity of this screening. Last, the protocol included a parent report in the event that professionals could not observe the child’s ability regarding two‐word sentences and pointing out body parts. The reliability of the answers of the parents was not investigated, but, parent report is generally accepted as reliable for assessing language abilities in young children. Therefore, a screening protocol based on professional observation supplemented with parent report, if needed, appears to provide a valid overview of a young child's language abilities.

Multiple studies have reported the positive effect of interventions to support the communication skills of children. , One way to promote language development at an early age could be to deliver the Hanen Program in which certified speech therapists assist parents in stimulating the speech and language development of their child. The potential benefits of adding this intervention to the protocol should be investigated. Moreover, several studies show relationships between the development of children on domains, such as language, feeding, psychosocial and motor development. Future research could relate the outcomes of children on the language protocol to outcomes of assessments on these other developmental domains.

CONCLUSION

Our findings on concurrent and predictive sensitivity and specificity indicate that the 2‐year screening protocol was valid for the detection of language problems in children. However, the specificity should be further improved for population based screening. Professionals demonstrated strong adherence to the protocol and any deviations from it made by the professionals did not improve its sensitivity and specificity. The protocol can provide support to professionals working in preventive child health care and thus deserves wider implementation in well child care.

Conflict of interest

The authors report no conflicts of interest.