A parent-reported standardised checklist is not sensitive to screen for motor problems at school age following neonatal critical illness.

AIM: As nowadays more children survive neonatal critical illness, evaluation of long-term morbidities becomes more important. We determined whether the parent-reported Movement Assessment Battery for Children-Second Edition (MABC-2) Checklist is a proper tool to screen for motor problems in school-aged children born with severe anatomical anomalies and/or treated with neonatal extracorporeal membrane oxygenation.
METHODS: We analysed data of 190/253 children (60.0% male) participating in our multidisciplinary follow-up programme who were routinely assessed at the ages of five, eight and/or 12 years. Parents completed the Checklist prior to assessment of the child's actual motor performance by a physical therapist using the MABC-2 Test. The sensitivity and specificity of the Checklist with a cut-off point of the 16th percentile were determined.
RESULTS: The sensitivity of the MABC-2 Checklist was 57.1%, which implies that 42.9% of the children at risk for motor problems were not identified. The specificity was 79.1%.
CONCLUSION: The low sensitivity of the MABC-2 Checklist suggests that this instrument does not suffice to screen for motor problems in children who survived neonatal critical illness. Yet, it may help to gain insight in parental perceptions of the child's motor performance and to provide tailored advice on lifestyle.

computerised adaptive testing

extracorporeal membrane oxygenation

Motor Assessment Battery for Children‐Second Edition

negative predictive power

percentile

positive predictive power

A proper tool to identify children who need extensive follow‐up for long‐term morbidities is lacking.

We found low sensitivity of the Movement Assessment Battery for Children‐Second Edition Checklist (57.1%), which suggests that this instrument does not suffice to identify children at risk for motor problems.

This instrument helps to gain insight in parental perceptions of their child's motor performance and thus supports clinicians to provide lifestyle advice.

INTRODUCTION

Due to improved surgical and neonatal management, a child born with a severe congenital anomaly and/or having suffered from neonatal critical illness has a greater probability to survive.1, 2, 3 Identifying children at risk for long‐term morbidity, such as motor function problems,3, 4, 5, 6 is a next‐level challenge.

Since healthcare systems are increasingly burdened by the need of follow‐up of these children, risk stratification for the development of long‐term sequelae becomes more important. Objective, standardised and norm‐referenced tests administered by experienced healthcare professionals are usually lengthy and costly to administer.7 Moreover, testing in the hospital setting can be stressful for children and parents alike.

Evaluating a child's performance in daily life activities with the use of proxy‐reported outcome measures may help identify deficits. It is important to identify those children who actually have motor problems and those who are at risk for motor impairments. Extensive follow‐up assessments and tailored advice could then be restricted to those children.

National and international collaborations to develop evidence‐based guidelines for follow‐up are increasingly being established, especially for children with rare diseases. Determining the prevalence of children with definite functional problems is an important first step towards standardisation of assessments and management.8, 9, 10 Population‐based data, preferably from multicentre studies, are needed for this purpose.

The Movement Assessment Battery for Children‐Second Edition (MABC‐2) Test and the associated MABC‐2 Checklist are internationally validated instruments to evaluate motor function in children aged three to 16 years.11 The Checklist is a questionnaire designed to be filled out by parents or professionals (eg doctors, teachers, therapists).11 To the best of our knowledge, only Schoemaker and co‐workers have evaluated the correlation between the MABC‐2 Test and its associated Checklist.12 Their study found low sensitivity of the MABC‐2 Checklist, which had been filled out by teachers.

We aimed to evaluate whether the MABC‐2 Checklist filled out by parents is valuable as screening tool for follow‐up assessments and/or as instrument for multicentre outcome studies. Therefore, we analysed whether the proxy‐reported MABC‐2 Checklist is sensitive and specific enough to identify motor problems in school‐aged children born with severe anatomical anomalies and/or treated with extracorporeal membrane oxygenation (ECMO).

METHODS

Patients, procedures and study design

Since 1999, we have offered a prospective longitudinal follow‐up programme for children born with anatomical anomalies and those who underwent ECMO treatment.13, 14 A multidisciplinary team follows these children and their parents from the age of six months till the age of 17 years.

Between May 2015 and May 2018, we asked the parents of all children who were being evaluated within the framework of this standardised programme at ages five, eight or 12 years to fill out the MABC‐2 Checklist immediately prior to assessment of the child's actual motor performance by the paediatric physical therapist using the MABC‐2 Test.

For the purpose of this study, we did not use the data of the children with unreliable results on the MABC‐2 Test,11 the data of the children whose parents had insufficient command of Dutch language to fill out MABC‐2 Checklist and the data of the children whose parents did not fully complete the MABC‐2 Checklist or whose answers were considered unreliable as indicated by the guidelines of the manual.

The Medical Ethics Review Board of the Erasmus University Medical Center stated that the rules laid down in the Dutch Medical Research Involving Human Subjects Act did not apply to this research proposal (MEC‐2016‐111). The parents of all included children had been informed about the study and had provided permission to use the data for research purposes.

The following baseline data were retrieved from the medical records: gender, age, diagnosis, gestational age, birthweight and duration of initial ventilation.

Proxy‐reported motor performance‐MABC‐2 Checklist

Perspectives regarding a child's motor performance in daily life activities can be assessed with the MABC‐2 Checklist. This Checklist is a questionnaire designed to be filled out by professionals or parents and contains two 15‐item sections. Section A evaluates movement in both a static and/or predictable environment. Section B evaluates movement in a dynamic and/or unpredictable environment. In this study, the Checklist was filled out by the parents of the children involved.

The items in both sections are rated on a four‐point Likert rating scale ranging from very well to not close. The total result was considered unreliable and excluded from analysis if more than three items in either section A or B were not rated, or if the parents had specified that they had not seen the movement or activity of the items.

The MABC‐2 Checklist Total Motor Score is the sum of the scores for the two sections; the higher the score, the poorer the performance. The psychometric properties of the MABC‐2 Checklist for the use in healthy Dutch children have been published previously.12 Age‐specific norm values are available for Dutch children from the age of five years up till 12 years.15

Actual motor performance‐MABC‐2 Test

The children's actual motor performance was evaluated by an experienced paediatric physical therapist using the MABC‐2 Test. The test contains three age bands. In each age band, children perform eight motor activities. Three domains are tested: manual dexterity, ball skills and balance. Summation of the item scores produces a total impairment score, which is then converted into an age‐related percentile score for Dutch children.15 For both the Checklist and the Test, a percentile score of ≤5 indicates definite motor problems; a percentile score of >5 and ≤16 indicates borderline performance; and a percentile score >16 indicates normal motor development.

Statistical analysis

Differences in baseline data between the included participants and the excluded participants were tested with Mann‐Whitney U tests for continuous variables and chi‐square tests for categorical variables. The sensitivity, specificity, positive predictive power (PPP) and negative predictive power (NPP) of the Checklist were calculated using the MABC‐2 Test as gold standard.16 To determine whether the MABC‐2 Checklist is an adequate screening tool for motor problems, we set a percentile score of 16 on both the MABC‐2 Test and the MABC‐2 Checklist as a cut‐off to indicate risk for motor problems. To determine the validity of the MABC‐2 Checklist for outcome studies, we set a percentile score of 5 on both the MABC‐2 Test and the MABC‐2 Checklist as a cut‐off to indicate a definite motor problem.

All statistical tests were two‐sided with a significance level of 0.05.

RESULTS

During the study period, 270 children were invited for a follow‐up visit, of whom 253 actually attended, and were assessed with the MABC‐2 Test by our paediatric physical therapist (response rate 93.7%). The parents of all 253 children completed the MABC‐2 Checklist. The result of the MABC‐2 Checklist was considered unreliable in 54 cases (21.3%); however, the data of five other children (2.0%) were incomplete. A reliable MABC‐2 Test could not be obtained in four other children (1.6%). Thus, data of 190 children (75.1%) were analysed.

Background characteristics of both the included children (60.0% boys) and excluded children (55.6% boys) are presented in Table 1. None of the background characteristics significantly differed between the included children and excluded children after exclusion of two outlying data: a ventilation duration of 285 days in one child and a very preterm birth at 25 weeks of another child.

Table 1

Patient characteristics included participants and excluded participants

Background	Included participants n = 190	Excluded participants n = 63
Diagnosis, (%)
Congenital anomaly
Oesophageal atresia	60 (31.6)	14 (22.2)
Diaphragmatic hernia	64 (33.7)a	24 (38.0)b
Congenital cystic adenomatoid malformation (resected)	18 (9.5)	5 (8.0)
Giant omphalocele	7 (3.7)	3 (4.8)
Other	2 (1.0)	2 (3.2)
ECMO treated patients without diaphragmatic hernia	39 (20.5)	15 (23.8)
Gender, (%)
Boy	114 (60.0)	35 (55.6)
Girl	76 (40.0)	28 (44.4)
Gestational age, (wk)	38.7 (35.9‐40.4)	38.3 (35.9‐40.0)
Birthweight, (kilograms)	3.1 (2.6‐3.5)	3.0 (2.6‐3.2)
Duration of initial ventilation, (d)	7.0 (2.0‐11.0)	8.0 (3.0‐14.0)

Data are presented as mean ± standard deviation, median (range) or number (percentage), as appropriate.

Abbreviation: ECMO, extracorporeal membrane oxygenation.

9 ECMO/55 non‐ECMO patients with diaphragmatic hernia.

5 ECMO/19 non‐ECMO patients with diaphragmatic hernia.

The parents' scores on the MABC‐2 Checklist indicated definite motor problems in 27 children (14.2%) and borderline performance in 33 children (17.4%). The MABC‐2 Test results indicated definite motor problems in 25 children (13.2%) and borderline performance in 31 children (16.3%) (Table 2).

Table 2

MABC‐2 Checklist versus MABC‐2 Test at two different cut‐off levels at 5, 8 and 12 y

Cut‐off 16th percentile	Test ≤P16	Test >P16
Checklist ≤P16
Total group	32 (57.1)	28 (20.9)
5 y	9 (50.0)	8 (30.8)
8 y	15 (60.0)	12 (20.0)
12 y	8 (61.5)	8 (16.7)
Checklist >P16
Total group	24 (42.9)	106 (79.1)
5 y	9 (50.0)	18 (69.2)
8 y	10 (40.0)	48 (80.0)
12 y	5 (38.5)	40 (83.3)

Data are presented as number of patients/parents (percentage). The sensitivity and specificity for the total group are shown in bold font.

Abbreviations: MABC‐2, Movement Assessment Battery for Children‐Second edition; P, percentile.

When we applied the cut‐off point of the 16th percentile, the sensitivity of the MABC‐2 Checklist was 57.1%; this implies that 42.9% of children at risk for motor problems were not identified (Table 2). The specificity at this cut‐off point was 79.1% (Table 2). At this cut‐off point, the PPP was 53.3% and the NPP was 81.5%.

When we applied the 5th percentile cut‐off point, which identifies children with definite motor function problems, the sensitivity was 40.0% and the specificity was 89.7% (Table 2). The PPP at the cut‐off point of the 5th percentile was 37.0% and the NPP was 90.8%.

The calculated sensitivities and specificities were comparable for the various age groups (Table 2) and differed for boys and girls (Table 3). The sensitivity for girls was lower than that for boys: 36.4% versus 70.6% at a cut‐off point of the 16th percentile and 25.0% vs 42.9% at the 5th percentile.

Table 3

MABC‐2 Checklist versus MABC‐2 Test at two different cut‐off levels for boys and girls separately

Cut‐off 16th percentile	Boys
	Test ≤P16	Test >P16
Checklist ≤P16	24 (70.6)	17 (21.2)
Checklist >P16	10 (29.3)	63 (78.8)

Data are presented as number of patients/parents (percentage). The sensitivity and specificity are shown in bold font.

Abbreviations: MABC‐2, Movement Assessment Battery for Children‐Second edition; P, percentile.

DISCUSSION

In this study, we analysed the sensitivity, specificity, PPP and NPP of the MABC‐2 Checklist in a group of routinely assessed school‐aged children born with congenital anatomical anomalies and/or neonatal ECMO treatment. In line with earlier studies, the majority of the children had normal motor function development evaluated with the MABC‐2 Test.4, 5 For both cut‐off points applied, the sensitivity and PPP were moderate to low‐moderate, respectively, and the specificity and NPP were high to very high, respectively.16 From a perspective of care, it is important that children at risk for motor problems are identified. For this reason, we applied the cut‐off level of the 16th percentile. The corresponding low sensitivity of 57.1% suggests that the MABC‐2 Checklist does not suffice as a screening tool for this purpose for the studied population. For the purpose of population‐based, multicentre outcome studies, a cut‐off point of the 5th percentile score is probably more appropriate. The sensitivity of the MABC‐2 Checklist at a cut‐off level of the 5th percentile was 40.0%, and the specificity at this cut‐off level was 89.7%. This Checklist may well serve to evaluate the prevalence of children with normal motor function, estimated by the parents, and hence to determine whether actual assessment of motor performance should be prioritised in a multidisciplinary follow‐up programme.

Only one previous study, in healthy Dutch and Flemish children aged from five to eight years with and without motor impairments, has evaluated the sensitivity and specificity of the MABC‐2 Checklist.12 In that study, the children's teachers had filled out the Checklist. They found a moderate, but significant correlation between the MABC‐2 Checklist and the MABC‐2 Test (−.38, P < .001); the sensitivity was only 41% and the specificity was 88% across all ages when the 16th percentile cut‐off was applied for both the Test and the Checklist.

Looking at the results of our study and the limited data from the one previous study, we can pose two questions. First, is the MABC‐2 Test an appropriate gold standard for the Checklist? Griffiths and co‐workers recently published a systematic review on the psychometric properties of gross motor assessment tools and concluded that the currently available gross motor assessment tools for children, including the MABC‐2 Test, have good to excellent content and construct validity.17 Kennedy et al evaluated the correlation between the MABC‐2 Checklist and the Bruininks‐Oseretsky Test of Motor Proficiency, Second Edition, in 38 healthy Australian children aged from eight to 12 years, and found a moderate positive correlation.18 All domains of that test, except that of the Fine Manual Control area, correlated with the MABC‐2 Checklist. Nevertheless, that study covered only a small sample of healthy children, and the sensitivity and the specificity of the Checklist were not calculated.

Second, can parents reliably report on the physical activities of 5 to 12‐year‐old children? Although the MABC‐2 Checklist is suitable to be applied by parents and professionals,11 some parents may have difficulty responding to specific items, especially those relating to physical activities that take place outside the home environment. In this respect, parental reports may be more reliable for younger children than for school‐aged children. Majewska et al demonstrated that maternal reports of developmental milestones of under 3‐year‐olds are sufficiently reliable to be used in clinical judgement.19 In that study in 387 3‐year‐old healthy Polish children, maternal reports on milestones attainment correlated with the scores on the Bayley Scales of Infant and Toddler Development‐Third Edition.19 In our study, we had to exclude data of 62 children (24.5%). The parents of 54 children, almost one‐quarter of all participating parents, indicated for at least three item in either section A or B of the Checklist that they had not seen the movement or activity described. As school‐aged children spend much time at school, it is not unthinkable that a child's teacher is better suited than the parents to complete the Checklist. In the study of Schoemaker and co‐workers, the Checklist was filled out by the children's teachers, and the resulting sensitivity was in line with the sensitivity we found.12 The specificity reported by Schoemaker and co‐workers was lower, however, than the specificity we found, which suggests more false‐positive reports by the parents in our study. Our data do not allow drawing conclusions as to whether the parents correctly observed their child's motor function, or whether the child's medical history had influenced their judgements.

The discrepancy in outcomes between the MABC‐2 Test and the MABC‐2 Checklist in our population might also be the result of a response shift. Parental internal standards of physical health may have changed after having seen their child's critical health state after birth. Indeed, in a study on survivors of meningococcal septic shock, these children's parents reported the long‐term physical and psychosocial health‐related quality of life of their children as better than did the parents of the Dutch normative population.20 Those parents reported that their child's stay in the intensive care unit had made them appreciate life more fully and that they were less worried about futilities in life.20 A positive response shift was found also in parents of children with less life‐threatening situations, such as new‐onset epilepsy.21

Our findings implicate that the MABC‐2 Checklist cannot be recommended as a screening tool for risks of impaired motor function in school‐aged children with congenital anatomical anomalies and/or neonatal ECMO treatment. Further studies in other patient populations may be needed to evaluate the merits of this Checklist as a screening tool. We have no explanation for the differences in calculated sensitivities and specificities between boys and girls in this study. Future studies may clarify the potential difference between boys and girls. Nevertheless, the instrument is certainly suitable to objectify parent's perception of their child's physical performance in daily life activities and thus be of value for clinicians to provide tailored advice on lifestyle. This is especially useful when outcomes as evaluated from clinicians' assessments differ from parental perceptions and parents overestimate the child's performance.

Parental perceptions on motor performance of their child could also be valuable to determine whether actual assessment of motor performance should be prioritised in a multidisciplinary follow‐up programme. Initiatives are being taken to establish, internationally, a standardised long‐term follow‐up programme for children with rare diseases and multi‐morbidity.9 For practical and financial reasons, it would be worthwhile to reach consensus on the main focus of such a programme. International standardisation may be difficult because of differences between countries in geographical distances for home‐hospital transfers and differences in resources to provide adequate multidisciplinary follow‐up. If multicentre outcome studies make clear that only a minority of parents consider their child to have abnormal motor function, as established with this Checklist, priority may initially be given to other assessments; for example, lung function or neuropsychological testing.

Nevertheless, clinicians should be aware that motor problems might deteriorate when children with severe anatomical anomalies and/or neonatal ECMO treatment grow older.5

A strength of our study is the inclusion of a large cohort of children assessed as standard of care at school age. Nevertheless, several limitations need to be addressed. First, data of 62 children (24.5%) had to be excluded because either the MABC‐2 Checklist was not filled out correctly and/or completely (23.3%) or the MABC‐2 Test results were unreliable (1.2%). Still, as included children and excluded children had similar background characteristics, we assume that our data set is generalisable for school‐aged children with severe congenital anatomical anomalies. However, data on socio‐demographic parameters, such as parental educational level, were not available to support this assumption. Second, the setting and timing of administering the Checklist might be considered sub‐optimal. All parents filled out the Checklist in the hospital immediately prior to the child's physical assessment. Should they have completed the Checklist at home, they might have had the opportunity to closely observe their child's motor performance prior to scoring the Checklist items. But it might also have given them the opportunity to let the child practice the motor skills first and then fill out the Checklist, which would have influenced the results. Third, although the study cohort was quite large, the sample sizes of the separate age groups were relatively small. Lastly, computerised adaptive testing (CAT) has not been used in this study, but might have been of additional value to objectify proxy‐reported outcome on physical functioning. A CAT algorithm selects items directly tailored to the child's ability level, based on responses to previous items.22 In a study of children with cerebral palsy, a parent‐report CAT programme was found useful in the assessment of the children's physical functioning.23 Future studies are needed to determine whether a CAT programme optimises tailored aftercare and is of added value for the evaluation of physical function in single‐centre and multicentre outcome studies.

CONCLUSION

The moderate‐to‐low sensitivity of the MABC‐2 Checklist in our study population suggests that this proxy‐reported Checklist does not suffice as a screening tool to identify actual motor problems in school‐aged children born with severe congenital anatomical anomalies and/or treated with ECMO. However, the Checklist may well serve to indicate the prevalence of normal motor function, estimated by the parents, in multicentre studies in this population, and hence to determine whether actual assessment of motor performance should be prioritised in international multidisciplinary follow‐up programmes. Moreover, this instrument is valuable for clinicians to evaluate parental perceptions on motor performance of their child. It can be of value for tailoring aftercare, especially for children whose parents tend to overrate physical performance.

CONFLICT OF INTEREST

The authors have no conflicts of interest to declare.