ESSENCE-Q - used as a screening tool for neurodevelopmental problems in public health checkups for young children in south Japan.

BACKGROUND: Screening for developmental disorders is an important task for Child Health Care. The concept of ESSENCE (early symptomatic syndromes eliciting neurodevelopmental clinical examinations) was created to cover all types of early developmental disorders and the ESSENCE-Questionnaire (ESSENCE-Q containing 12 questions with possible total scores ranging from 0 to 22) was developed as a tool for early detection of these disorders. The aim of this study was to perform a validation study in a public health situation in Japan.
METHODS: The psychometric properties of the ESSENCE-Q, completed by mothers, public health nurses (PHNs), and psychologists at 18-month (n=143 children) and 36-month (n=149 children) checkups were evaluated in a small city of Japan. Results were validated against clinical ESSENCE diagnoses. Receiver operating characteristic curves were generated and compared by using the area under the curve (AUC). Optimal cutoff values were explored.
RESULTS: At the 18-month checkup, AUC by mothers was 0.72, by PHNs 0.86, and by psychologists 0.82. An optimal cutoff was 3 with a high negative predictive value (NPV). At the 36-month checkup, AUC by mothers was 0.57, by PHNs 0.82, and by psychologists 0.87. Optimal cutoff was 2 with high NPV.
CONCLUSION: The ESSENCE-Q completed by PHNs and psychologists had good diagnostic validity. The results suggested that almost all children scoring under cutoff would not have any ESSENCE problems/diagnoses.

Introduction

Screening for chronic diseases has been defined as: If “disease or defect” is changed to “neurodevelopmental disorder or problem” and “to their physician” to “for neurodevelopmental assessment,” then this definition can also be applied to developmental screening. The main aim of developmental screening was to assess a child’s risk of developmental disorders.2,3 Early identification of neurodevelopmental problems is one of the most important tasks for public health.4 Many outcome studies of children with developmental disorders have demonstrated that early tailored interventions are important to maximize the individual child’s potential.3 There is growing evidence that, even when a precise diagnosis cannot be established, early intervention taking the child’s basic impairment into account is important.5–8

[…] the presumptive identification of unrecognized disease or defect by application of tests, examinations or other procedures, which can be applied rapidly […]. Persons with positive or suspicious finding must be referred to their physician for diagnosis and necessary treatment.1

ESSENCE (early symptomatic syndromes eliciting neurodevelopmental clinical examinations) is a concept that was introduced with a view to alert clinicians and researchers to the complexity and overlap of different neurodevelopmental disorders/problems including autism spectrum disorder (ASD), attention-deficit/hyperactivity disorder (ADHD), oppositional defiant disorder (ODD), tic disorders (TDs), developmental coordination disorder (DCD), speech and language disorder (SLD), intellectual developmental disorder (IDD), borderline intellectual functioning (BIF), and other non-specific learning difficulties (LDs).9 If a child before 5 years of age has at least one (usually several) problem lasting more than several months in the following 12 areas – general development, communication/language, social interrelatedness, perception, motor coordination, attention/“listening,” activity, “behavior,” mood, feeding, sleep, and/or episodes/absences – this should be seen as a red flag for possible neurodevelopmental disorders. Unrecognized and “untreated” ESSENCE may predispose to adult psychiatric and/or physical disorders. With regard to prevalence, at least 10% of children aged <18 years are affected by ESSENCE.9 Identifying problems under the ESSENCE concept in young children and monitoring child development would possibly reduce the risk of later maladjustment.

The ESSENCE-Questionnaire (ESSENCE-Q) is a screening tool designed to help clinicians and researchers to identify children with ESSENCE (or neurodevelopmental) problems.10 The ESSENCE-Q is intended for use in both clinical practice and population research. It is proposed to be useful as a questionnaire to be completed by parents and/or caregivers or as a brief interview by a specialist. It has already been shown that the ESSENCE-Q can be useful as a parent questionnaire in a neurodevelopmental clinic.11 It could also be useful in a public health setting as a parent questionnaire or as an interview questionnaire and/or as the basis for an observation record.

Screening at child health care centers has proven to be a valuable method to identify developmental disorders in young children. Previous studies12 have demonstrated the use of a language screening, performed by the health nurse, and that such screening identifies also children with other ESSENCE problems. Such screening methods include the professional assessment supplemented by reports from parents. The value of the ESSENCE-Q lies in its short format and its broad approach, covering different developmental areas. It can be used to obtain both parents’ and professionals’ concerns at the child’s health checkups and will thereby increase the validity of the screening.

In Japan, according to the Maternal and Child Health Law,13 all children are invited to take part in health checkups, first during 18–24 months (“18-month checkups”) and second during 36–48 months (“36-month checkups”). Municipalities have responsibility for these checkups, and the rate of attendance is ~95%.14 Therefore, these checkups provide good opportunities to identify the children with neurodevelopmental disorders.15 The ESSENCE-Q in 18-month and 36-month checkups in a small city of Japan was used as a parent questionnaire to mothers and as a parent-interview and child-observation tool used by public health nurses (PHNs) and psychologists who were specialized in neurodevelopmental disorders (specialized psychologists).

The aims of the present study were to 1) evaluate the validity of the ESSENENCE-Q completed by mothers, PHNs, and specialized psychologists in a public health setting and 2) explore an optimal cutoff score of the ESSENCE-Q at 18- and 36-month checkups.

Methods

This is a prospective, population-based, observational cohort study. Data on two separate groups of children who came to health care centers in Kami City, Kochi, Japan, from April 2014 to March 2015 were collected. In Kami City, parents of children who were 18 or 42 months of age during the period were invited to 18- or 36-month checkups. PHNs in the city have been working with specialized psychologists, and they have had on-the-job training in screening of neurodevelopmental disorders for several years. They also participated in seminars about ESSENCE and related problems, including about the ESSENCE-Q.

ESSENCE-Q

Procedures

The ESSENCE-Q is a brief one-page “quick and easy” questionnaire containing 12 items covering concerns relating to the above-mentioned 12 areas. Response options of “Yes,” “Maybe/A little,” or “No” should be checked for each item. It is not a diagnostic instrument or an instrument that can be used as a proxy for diagnoses, but a screening tool to be used in order not to overlook children who need further assessments by a developmental specialist.10

All ESSENCE-Q data from mothers who came with their child to the 18-month or the 36-month checkup were collected. PHNs and specialized psychologists were engaged in these checkups. The mothers, PHNs, and the specialized psychologists scored the child’s ESSENCE-Q blind to each other.

The ESSENCE-Q was sent to mothers before the checkup, and they were asked through letter to complete the questionnaire. These ESSENCE-Qs (ESSENCE-Q-M) were gathered by receptionists at the checkups. Then, at the checkups, the PHNs completed the ESSENCE-Q (ESSENCE-Q-N) through interview with the mothers and through direct observation of the child without any knowledge about the ESSENCE-Q-M. Third, specialized psychologists completed the ESSENCE-Q (ESSENCE-Q-P) through interview with the mothers and through direct observation of the child without any knowledge about the ESSENCE-Q-M and the ESSENCE-Q-N.

The PHNs and the specialized psychologists used the ESSENCE-Q statements as a template for their interviews, and sometimes added simple questions (eg, “When did your child start walking?”). After the interview and the observation, they noted their concerns in the ESSENCE-Q.

ESSENCE diagnoses

At the end of the checkups, a pediatrician with training in the field of ESSENCE examined the child and interviewed the mother. The child’s entire records with regard to development from birth were checked as were the three ESSENCE-Qs. Taking all this information into consideration, the pediatrician made a decision as to whether or not the child needed to go for further neurodevelopmental examinations. Children targeted for further examinations were then examined – on another day at the same health center – by a specialized child neuropsychiatrist (including interview with the mother). If there was any remaining concern regarding neurodevelopmental disorders, the child was referred to a neurodevelopmental clinic for further investigation by the psychiatrist and a team including occupational therapists and psychologists. The time periods between the secondary checkup and the first assessment at the neurodevelopmental clinic were <4 months (in most cases, 2 months). The child neuropsychiatrist was not blind to the results of the ESSENCE-Qs, but these results did not form part of his referral decision.

At the neurodevelopmental clinic, assessments covered all developmental areas included under the ESSENCE umbrella. Overall development, motor and perceptual performance, social communication and related behaviors, social interest, joint attention, imitation, play, reciprocal affective behavior, and insistence on sameness/stereotypies16–18 were assessed for all children at the first two to three visits. For cognitive assessment, the Kyoto Scale of Psychological Development 2001 (KSPD2001)19 was used. KSPD is an individualized face-to-face test to assess a child’s development in the areas of fine and gross motor functions, non-verbal reasoning, visuospatial perceptions, interpersonal relationships, socializations, and verbal abilities.19 KSPD2001 has excellent psychometric properties, and the results are closely correlated with the Bayley Scales of Infant Development.20 For the assessment of social and communication development, the Diagnostic Interview for Social and Communication Disorders (DISCO)21 was used. The DISCO is widely used as an assessment tool for autism spectrum conditions. The interview is semistructured and covers a wide range of behaviors associated with the ASD phenotype. It is suitable for use with all ages and levels of ability. It enables to identify specific features found in ASD. The Strengths and Difficulties Questionnaire (SDQ)22 for parents of 2- to 4-year olds was used. SDQ covers child mental health and developmental aspects, including child behavior, emotions, and relationships. It also addresses impact and duration of symptoms, distress in the child, impairment in different settings, and burden to others. SDQ can be used as a screening tool for several types of neurodevelopmental disorders, such as ASD and ADHD. Unstructured clinical observations at the clinic and reports/interviews from parents and preschool teachers were collected throughout the examination period. Motor-perceptual performance was examined at clinical observations. The diagnostic evaluations were done at least at five different sessions separated by a minimum of 2 weeks.23 For the children who were referred from the 18-month checkups, definite diagnoses were given at or after 30 months. The International Statistical Classification of Diseases-10/Diagnostic and Statistical Manual of Mental Disorders-IV criteria for diagnoses of child psychiatric/neurodevelopmental disorders were used throughout.

Statistical analysis

ESSENCE-Q items were rated as 0 for “No,” 1 for “Maybe/a little,” and 2 for “Yes.” The range of possible scores was 0–24. If four or more (>10%) of the 36 items, collapsed from the three ESSENCE-Qs, were unchecked by mother, PHS, or specialized psychologist, then this case was excluded. Unchecked items were otherwise rated as 0. These overall ESSENCE-Q scores were used as continuous variables and generated receiver operating characteristic (ROC) curves and compared the area under the curve (AUC) to evaluate the validity of ESSENCE-Q-M, ESSENCE-Q-N, and ESSENCE-Q-P separately. In addition, the optimal cutoff values for screening in health checkups from the ROC curves were explored. For a developmental screening, the best sensitivity and specificity balance is around 0.70–0.80 for both.5,24 Sensitivity should be higher than specificity so as not to miss children with problems.25 When an optimal cutoff value fulfilling these conditions was found, the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated. The data from 18- and 36-month checkups were analyzed separately. All statistical analyses were done by R version 3.1.3 (The R Foundation for Statistical Computing, Vienna, Austria). Sensitivity, specificity, PPV, and NPV were expressed with a 95% confidence interval (CI).

Ethics

The institutional ethical boards of Kochi Prefectural Medical and Welfare Centre approved the study with parents’ written informed consent, No 25-2056.

Results

Characteristics of study population

18-month checkups (Figure 1)

During the study period, all 152 children (79 boys and 73 girls) were invited to the 18-month checkup and 143 children (75 boys and 68 girls, mean age 18.7, standard deviation [SD] 0.9) participated. The attendance rate was 94%. Of the 143 children, 50 (27 boys and 23 girls) (35%) were invited to come to the secondary checkup due to developmental concerns. However, eight (three boys and five girls) did not come to the secondary checkup. Of the remaining 42 children, 21 (10 boys and 11 girls) were regarded to have normal development and 21 (14 boys and 7 girls) were referred to the neurodevelopmental clinic. One boy’s family moved out from the city and, as a result, 20 (13 boys and 7 girls) came to the clinic.

Neurodevelopmental diagnoses in the children of the 18-month checkups

Of the 20 children who came to the clinic, five (three boys and two girls) were considered to have a normal developmental trajectory and 15 children (10 boys and 5 girls) were given neurodevelopmental diagnoses (10% of all participants). One third had intellectual problems (IDD, BIF) and one third had ADHD. Two had ASD. Table 1 shows diagnostic and other information of the 20 children who came to the clinic.

Table 1

Diagnosis and background information of the 21 children referred to the clinic from 18-month checkups

Number	Gender	Age at the first checkup (months)	Age at the first visit (months)	ESSENCE diagnoses/conditions at age (months)
1	Boy	19	21	Normal developmental trajectory (44)
2	Boy	18	21	Normal developmental trajectory (33)
3	Boy	18	19	ASD (33)
4	Boy	18	26	ADHD (41), IDD (41)
5	Boy	18	20	ADHD (39), DCD (39)
6	Boy	19	38	BIF (39)
7	Boy	19	20	ASD (36)
8	Boy	18	21	Normal developmental trajectory (30)
9	Boy	21	22	IDD (30)
10	Boy	19	22	IDD (33)
11	Boy	19	25	ADHD (32)
12	Boy	18	29	SLD (30)
13	Boy	18	20	IDD (30)
14*	Boy	18	N/A	N/A
15	Girl	18	21	ADHD (42), SLD (43)
16	Girl	19	23	Normal developmental trajectory (38)
17	Girl	18	22	SLD (37)
18	Girl	19	28	Normal developmental trajectory (30)
19	Girl	18	20	SLD (42)
20	Girl	18	21	ADHD (36)
21	Girl	18	19	IDD (30)

Notes:

Lost to follow-up due to changing address.

Abbreviations: ADHD, attention-deficit/hyperactivity disorder; ASD, autism spectrum disorder; BIF, borderline intellectual functioning; DCD, developmental coordination disorder; ESSENCE, early symptomatic syndromes eliciting neurodevelopmental clinical examinations; IDD, intellectual developmental disorder; SLD, speech and language disorder; N/A, not applicable.

Three of the 143 children (three boys without a diagnosis) were excluded because of missing data. The eight children who did not come for the second assessment and the child who moved from the area were excluded (because it could not be ascertained whether or not they actually had an ESSENCE diagnosis), leaving 131 (63 boys and 68 girls, mean age 18.7, SD 0.8, 92% of all the participants) for statistical analyses.

“Thirty six-month checkups” performed at a mean age of 42 months (Figure 2)

During the period, 158 children (77 boys and 81 girls) were invited to the “36-month checkups” (which in reality occurred at 42 months in the majority of cases) and 149 children (73 boys and 76 girls, mean age 42.0 months, SD 1.0) participated. The attendance rate was 94%. Of the 149 children, 38 (19 boys and 19 girls) (26%) were invited to come to the secondary checkup due to developmental concerns. However, five (five boys) did not come to the secondary checkup. In the remaining 33 children, 17 were regarded to have a normal development (6 boys and 11 girls) and 16 (8 boys and 8 girls) were referred to the neurodevelopmental clinic. A family of a boy and another family of a girl moved out from the area and, as a result, 14 (7 boys and 7 girls) came to the clinic.

Neurodevelopmental diagnoses in the children of the 42-month checkups

All children who came to the clinic were given neurodevelopmental diagnoses (9% of all participants). Twelve had ADHD and two had ASD. Ten had two or more diagnoses. Table 2 shows diagnostic and other information of the 14 children who were referred to the clinic.

Table 2

Diagnosis and background information of the 16 children referred to the clinic from 36-month checkups

No	Gender	Age at the first checkup (months)	Age at the first visit (months)	ESSENCE diagnoses/conditions at age (months)
1	Boy	41	46	ASD (48), DCD (48), Tics (48)
2	Boy	42	47	BIF (49), ADHD (56)
3	Boy	42	43	ASD (45), ADHD (58)
4	Boy	42	43	DCD (44), ADHD (51)
5	Boy	41	45	ADHD (51), SLD (51)
6	Boy	41	44	ADHD (57), SLD (57)
7*	Boy	43	N/A	N/A
8**	Boy	42	44	ADHD (52), BIF (52)
9	Girl	43	51	ADHD (59)
10	Girl	43	45	DCD (45), Tics (45), SAD (45), Congenital tremor (45)
11	Girl	41	46	ADHD (56)
12	Girl	43	44	ADHD (59), BIF (59)
13	Girl	42	45	ADHD (59)
14	Girl	42	45	ADHD (59)
15*	Girl	43	N/A	N/A
16	Girl	41	47	ADHD (53), RAD (53)

Notes:

Lost to follow-up due to changing address.

Excluded from ROC analysis because of missing data.

Abbreviations: ADHD, attention-deficit/hyperactivity disorder; ASD, autism spectrum disorder; BIF, borderline intellectual functioning; DCD, developmental coordination disorder; ESSENCE, early symptomatic syndromes eliciting neurodevelopmental clinical examinations; N/A, not applicable; RAD, reactive attachment disorder; SAD, social anxiety disorder; SLD, speech and language disorder; ROC, receiver operating characteristic.

Three of the 149 children (three boys, one of whom had a diagnosis) were excluded because of missing data. Five children who did not come to the secondary assessment and two children who did not come for full clinical assessment were also excluded, leaving 139 (64 boys and 75 girls, mean age 42.0, SD 1.0, 93% of all the participants) for statistical analyses.

ESSENCE-Q scores

18-month checkups (Figure 3, Table 3)

Overall ESSENCE-Q-M, ESSENCE-Q-N, and ESSENCE-Q-P scores produced AUC values (95% CI) of 0.69 (0.52–0.86), 0.92 (0.86–0.97), and 0.85 (0.74–0.96), respectively. For ESSENCE-Q-M, an optimal cutoff value that fulfilled the conditions mentioned above was not found. For ESSENCE-Q-N, the optimal cutoff value was set at 3, which showed a sensitivity of 0.93 (95% CI: 0.66–1.00) a specificity of 0.74 (95% CI: 0.66–0.82), a PPV 0.30 (95% CI: 0.17–0.46), and an NPV 0.99 (0.94–1.00). For ESSENCE-Q-P, an optimal cutoff value was set as 3, which showed a sensitivity of 0.86 (95% CI: 0.57–0.98), a specificity of 0.75 (95% CI: 0.66–0.83), a PPV of 0.29 (95% CI: 0.16–0.46), and an NPV of 0.98 (95% CI: 0.92–1.00).

36-month checkups (Figure 4, Table 4)

Overall ESSENCE-Q-M, ESSENCE-Q-N, and ESSENCE-Q-P scores produced AUC values (95% CI) of 0.63 (0.46–0.81), 0.84 (0.73–0.95), and 0.82 (0.69–0.94). For ESSENCE-Q-M, an optimal cutoff value that fulfilled the conditions mentioned above was not found. For ESSENCE-Q-N, the optimal cutoff value was set as 2, which showed a sensitivity of 0.86 (95% CI: 0.57–0.98), a specificity 0.70 (95% CI: 0.61–0.76), a PPV of 0.24 (95% CI: 0.13–0.38), and an NPV of 0.98 (95% CI: 0.92–1.00). For ESSENCE-Q-P, the optimal cutoff value was set as 2, which showed a sensitivity of 0.86 (95% CI: 0.57–0.98), a specificity of 0.66 (95% CI: 0.57–0.75), a PPV of 0.22 (95% CI: 0.12–0.36), and an NPV of 0.98 (95% CI: 0.92–1.00).

Differences between boys and girls

Results were subdivided according to gender. There was a clear trend toward boys having “better” validity data than the girls. In the 18-month checkups, sensitivity was 0.89 and 0.89 for boys compared to 1.00 and 0.80 for girls at ESSENCE-Q-N and ESSENCE-Q-P, respectively. Specificity was 0.75 and 0.71 for boys compared to 0.74 and 0.79 for girls. PPV was 0.35 and 0.32 for boys compared to 0.25 and 0.25 for girls. NPV was 0.98 and 0.98 for boys compared to 1.00 and 0.98 for girls. In the 36-month checkups, sensitivity was 1.00 and 0.86 for boys compared to 0.71 and 0.86 for girls at ESSENCE-Q-N and ESSENCE-Q-P, respectively. Specificity was 0.75 and 0.70 for boys compared to 0.65 and 0.63 for girls. PPV was 0.33 and 0.26 for boys compared to 0.17 and 0.19 for girls. NPV was 1.00 and 0.98 for boys compared to 0.96 and 0.98 for girls. The significance of the differences was not tested because of small numbers.

Attrition results

Of the nine children in the 18-month checkups and the seven children in the 36-month checkups who could not be diagnostically assessed (and who were suspected to be cases), five and six respectively were screened positive. This rate (69% of all who dropped out after initial screening) is higher than the rate (36%) for those who were retained for the statistical analyses.

Discussion

According to the results obtained from these 18- and 36/42-month checkups – with AUCs for ESSENCE-Q-N and ESSENCE-Q-P of 0.8–0.9 and NPVs of almost 1.0 – the ESSENCE-Q used by PHNs and specialized psychologists seems to hold promise as a general population screening tool for neurodevelopmental disorders in young children. However, the validity of ESSENCE-Q-M (showing an AUC of 0.69 in the 18-month and 0.63 in the 36-month checkup) seems not sufficient for a screening tool when used alone. Given that individuals who dropped out were much more often screen positive than were those who remained in the study (and hence much more likely to be real cases), the estimates of psychometric validity presented here are probably conservative. An underlying reason for the better validity of the ESSENCE-Q rating by PHNs and specialized psychologists, compared to the mothers, might be that the professionals’ experiences contributed to a more realistic assessment of the children’s development. The ESSENCE-Q in its present format appears to possibly have a better validity for screening boys – at least at the 36/42-monthcheckup – as compared with girls with neurodevelopmental disorders in the general population, but numbers after subdivision according to gender were too small to make other than very tentative suggestions. However, the possibility that symptoms of a developmental disorder in some girls may not be obvious enough at the time of screening was not excluded.

Nurses working in maternal and child health are the key professionals for early identification of neurodevelopmental disorders in many western countries,26 and, in Japan, PHNs are expected to take this role.27 If a trained PHN’s identification of children with ESSENCE at the first stage of screening can be shown to be valid when compared with the result of the ESSENCE-Q assessment by a specialized psychologist, the PHN might then even be suggested to take the role of “specialist” when it comes to referral for full neurodevelopmental assessment. Given that the PHNs who joined this study had enough knowledge and experiences with regard to neurodevelopmental disorders in young children, further studies are needed to determine whether or not the results of this study can be generalized to other, larger general population samples.

In the 18-month checkup, a cutoff value of 3 satisfied our requirement (ie, that both sensitivity and specificity be at least 0.7–0.8, and sensitivity > specificity) for ESSENCE-Q-N and ESSENCE-Q-P. The high NPVs (0.98 in ESSENCE-Q-N and in ESSENCE-Q-P) suggested that almost all children with score <3 in the ESSENCE-Q by a PHN (or by a psychologist) would be free of ESSENCE problems/diagnoses. On the other hand, the low PPVs (0.30 in ESSENCE-Q-N and 0.29 in ESSENCE-Q-P) suggested that the result on the ESSENCE-Q should not be taken to mean that all children over the cutoff value would have neurodevelopmental diagnoses. Nevertheless, screen-positive children in this cohort will be followed-up, and if later concerns regarding neurodevelopmental problems arise, they will be referred for new neurodevelopment assessment. In the 36/42-month checkup, with a cutoff of 3, the sensitivity was 0.71–0.64 and the specificity was 0.81–0.79 for ESSENCE-Q-N and ESSENCE-Q-P. This was slightly under our required level. When the cutoff was set at 2, the sensitivity and the specificity almost satisfied our requirement for ESSENCE-Q-N and ESSENCE-Q-P. NPVs were almost 1.0 and PPVs (0.24 in ESSENCE-Q-N and 0.22 in ESSENCE-Q-P) were similar to levels at 18-month checkups. These cutoff values (3 in 18-month checkups and 2 in 36-month checkups) are almost identical to those suggested by the originator of the ESSENCE-Q (“Yes” ≥1 or “Maybe/a little” ≥3).

The results of the ESSENCE-Q by mothers, PHNs, and specialized psychologists were known to the pediatrician who made the decision to refer or not for secondary checkup. This is a clear limitation of the present study. However, given that the study was performed in a routine clinical environment that already included some kind of explicit screening for neurodevelopmental disorders, it would have been ethically problematic to recruit a representative population if parent-reported or observed problems would not have influenced the decision to refer for in-depth assessment. The strength of the study includes its population-based nature, the very thorough clinical examination of the children, and the fact that the screening procedure used is clinically applicable as it stands (“quick and easy” and based on a one-page questionnaire).

There are several studies that have claimed that the developmental concerns expressed by parents should be taken seriously by specialists.28,29 It was previously shown that the ESSENCE-Q completed by parents of children referred for neurodevelopmental assessment can be a useful screening tool for neurodevelopmental disorders in a clinical setting.11 However, given that the numbers were small in both the present and the clinical study, further research is needed before generalizable conclusions can be drawn.

Table 3

AUC, sensitivity, specificity, PPV, and NPV of the three ESSENCE-Qs in 18-month checkups

18 months (n=131)	ESSENCE-Q-M	ESSENCE-Q-N	ESSENCE-Q-P
AUC	0.69 (0.52–0.86)	0.92 (0.86–0.97)	0.85 (0.74–0.96)
Optimal cutoff score	N/A	Ntotal ≥3	Ptotal ≥3
Sensitivity	N/A	0.93 (0.66–1.00)	0.86 (0.57–0.98)
Specificity	N/A	0.74 (0.66–0.82)	0.75 (0.66–0.83)
PPV	N/A	0.30 (0.17–0.46)	0.29 (0.16–0.46)
NPV	N/A	0.99 (0.94–1.00)	0.98 (0.92–1.00)

Note: Figures in parentheses indicate 95% CI.

Abbreviations: AUC, area under the curve; CI, confidence interval; ESSENCE-Q, ESSENCE-Questionnaire; ESSENCE-Q-M, ESSENCE-Q completed by mother; ESSENCE-Q-N, ESSENCE-Q completed by public health nurse; ESSENCE-Q-P, ESSENCE-Q completed by specialized psychologist; NPV, negative predictive value; N/A, not applicable; PPV, positive predictive value.

Table 4

AUC, sensitivity, specificity, PPV, and NPV of the three ESSENCE-Qs in 36-month checkups

36 months (n=139)	ESSENCE-Q-M	ESSENCE-Q-N	ESSENCE-Q-P
AUC	0.63 (0.46–0.81)	0.84 (0.73–0.95)	0.82 (0.69–0.94)
Optimal cutoff score	N/A	Ntotal ≥2	Ptotal ≥2
Sensitivity	N/A	0.86 (0.57–0.98)	0.86 (0.57–0.98)
Specificity	N/A	0.70 (0.61–0.76)	0.66 (0.57–0.75)
PPV	N/A	0.24 (0.13–0.38)	0.22 (0.12–0.36)
NPV	N/A	0.98 (0.92–1.00)	0.98 (0.92–1.00)

Note: Figures in parentheses indicate 95% CI.

Abbreviations: AUC, area under the curve; CI, confidence interval; ESSENCE-Q, ESSENCE-Questionnaire; ESSENCE-Q-M, ESSENCE-Q completed by mother; ESSENCE-Q-N, ESSENCE-Q completed by public health nurse; ESSENCE-Q-P, ESSENCE-Q completed by specialized psychologist; NPV, negative predictive value; N/A, not applicable; PPV, positive predictive value.