Prediction of outcomes of extremely low gestational age newborns in Australia and New Zealand.

OBJECTIVE: To determine the accuracy of the National Institute of Child Health and Human Development (NICHD) calculator in predicting death and neurodevelopmental impairment in Australian and New Zealand infants.
DESIGN: Population-based cohort study.
SETTING: Australia and New Zealand. PATIENTS: Preterm infants 22-25 completed weeks gestation.
INTERVENTIONS: Comparison of NICHD calculator predicted rates of death and death or neurodevelopmental impairment, with actual rates recorded in the Australian and New Zealand Neonatal Network cohort. MAIN OUTCOME MEASURES: Infant death and death or neurodevelopmental impairment rates.
RESULTS: A total of 714 infants were included in the study. Of these infants, 100 (14.0%) were <24 weeks, 389 (54.5%) male, 529 (74.1%) were singletons, 42 (5.9%) had intrauterine growth restriction, 563 (78.9%) received antenatal steroids and 625 (87.5 %) were born in a tertiary hospital. There were 288 deaths (40.3%), 75 infants (10.5%) with neurodevelopment impairment and 363 (50.8%) with death or neurodevelopmental impairment. The area under the curve (AUC) for prediction of death and the composite death or neurodevelopmental impairment by the NICHD calculator in our population was 0.65(95% CI 0.61 to 0.69) and 0.65 (95% CI 0.61 to 0.69), respectively. When stratified and compared with gestational age outcomes, the AUC did not change substantially for the outcomes investigated. The calculator was less accurate with outcome predictions at the extreme categories of predicted outcomes-underestimation of outcomes for those predicted to have the lowest risk (<20%) and overestimation for those in the highest risk category (>80%).
CONCLUSION: In our recent cohort of extremely preterm infants, the NICHD model does not accurately predict outcomes and is marginally better than gestational age based outcomes.

Tools to predict death and neurodevelopmental outcomes of extremely preterm newborns are limited and are typically based on gestational age in completed weeks.

Prediction models such as the National Institute of Child Health and Human Development (NICHD) calculator have been validated in several cohorts and are routinely used in clinical practice.

The NICHD model does not provide an accurate prediction of outcomes for extremely preterm infants born between 22 weeks and 25 weeks in Australia and New Zealand.

The NICHD model showed inaccuracies at the highest and lowest predicted probabilities of outcomes for extremely preterm infants in the Australian New Zealand cohort.

Introduction

The initiation of resuscitation and provision of intensive care for extremely preterm infants at the margins of viability between 22 weeks and 25 weeks gestational age is an area of major controversy and raises many ethical questions for parents and medical staff. Even with improvements in intensive care, these infants continue to be at high risk of death, major morbidities and neurodevelopmental impairment.1–3 Clinicians in the neonatal intensive care unit (NICU) routinely provide antenatal counselling to parents and guide management decisions based on available estimates of survival and neurodevelopmental outcomes of these infants.4

Tools to predict death and neurodevelopmental outcomes of extremely low gestational age newborns are limited and are based on best estimates of gestational age in completed weeks.5–9 The US National Institute of Child Health and Human Development (NICHD) Neonatal Research Network developed a model for predicting outcomes for extremely low gestational age newborns (22–25 weeks) based on five risk factors including gestational age in completed weeks, gender, estimated birth weight, prenatal steroids and plurality.6 This study demonstrated that each of these factors had an impact on survival and neurodevelopmental outcome similar to that of a single week of gestational age. This prediction model has since been validated by several other groups and is routinely used in clinical practice with the creation of an internet-based calculator that estimates survival and survival without neurodevelopmental impairment (https://www.nichd.nih.gov/about/org/der/branches/ppb/programs/epbo/pages/epbo_case.aspx).10 11

The generalisability of the NICHD model to non-academic and non-US setting deserves further study as it is based on data collected from inborns at 19 US tertiary academic institutions.10 11 Survival rates and outcomes based on single and selected clusters of NICUs may be biased by local patient variations and heterogeneous medical practices. The objective of our study was to determine if the NICHD calculator published in 2009 is valid in predicting death and neurodevelopmental impairment for a contemporary population cohort of extremely low gestational age newborns admitted for neonatal intensive care during 2009 to 2010 in Australia and New Zealand.

Methods

Study population

The data for this study were obtained from the Australian and New Zealand Neonatal Network (ANZNN) database that began in January 1995. All ANZNN registrants born between 2009 and 2010, 22–25 weeks gestation and admitted to one of the 29 tertiary NICUs in Australia and New Zealand were included in this study. The Network monitors the care of high-risk newborns by pooling data for all infants admitted to tertiary level NICUs during the neonatal period (≤28 days) for any of the following indications: gestational age of <32 completed weeks, birth weight of <1500 grams, need for assisted ventilation (mechanical ventilator or continuous positive airway pressure device (CPAP)) for ≥4 consecutive hours, received major surgery or received therapeutic hypothermia. The care of extremely preterm infants has been regionalised since the early 1990s, where the ANZNN contains 99% of all live births <29 weeks in both countries. Neonatal, maternal and perinatal data are collected from each NICU and compiled into a central ANZNN database located at the University of New South Wales, Australia. Each NICU has an audit officer who collects and checks the data before submission into the central database. The accuracy of the data collection is validated by random data crosschecking by the ANZNN coordinator.

Outcome definitions

Variables were defined according to the ANZNN data dictionary (https://anznn.net/dataresources/datadictionaries). Gestational age is defined as the best obstetric estimate of completed weeks based on obstetric history, clinical examination and prenatal ultrasound. Infants were considered outborn if they were born outside of a hospital with a tertiary level NICU. Major morbidity was defined as the presence of any of the following: bronchopulmonary dysplasia,12 grade 3 or grade 4 intraventricular haemorrhage,13 periventricular leucomalacia, medically or surgically treated necrotising enterocolitis or grade 3 or grade 4 retinopathy of prematurity.14 The composite outcome of death and/or major morbidity included infants who died or had at least one of the major morbidities.

Survival and neurodevelopmental outcomes were assessed at 2–3 years corrected age as per the ANZNN study protocol. The presence of neurodevelopmental impairment was defined as blindness, deafness requiring amplification, cerebral palsy with severity graded by Gross Motor Function Classification System (GMFCS) levels 2 to 5, or moderate to severe developmental delay. Moderate to severe developmental delay was determined by a score of >2 SD below the mean on either the cognitive, language or motor scale of the Bayley Scales of Infant Development-Third Edition (BSID-III) or Griffith Mental Developmental Scales; or by individual review by the ANZNN follow-up subcommittee of developmental outcomes from other formal developmental assessments or clinical assessments of development by a clinician.

The NICHD risk prediction for the outcomes investigated for each infant was generated based on the logistic mixed model equation provided by the authors.6 The outcomes predicted by the calculator were compared with the actual outcomes for each patient in our study cohort. The following outcomes were compared: death, death or neurodevelopmental impairment.

Statistical analysis

Receiver operative characteristics (ROC) curves for death and the composite outcome of death or neurodevelopmental impairment were constructed to assess the accuracy of the NICHD calculator for predicting these outcomes in the ANZNN study population. Calculation of the area under the curve (AUC) and its 95% CI were used to compare the performance of the NICHD calculator. The predicted rates of death and the composite outcome from the NICHD calculator were compared with the actual rates in the ANZNN study population. Separate ROC curves were generated for the different gestational age in weeks to compare the prediction of the calculator with gestational age outcomes, which are typically used for outcome prediction of extremely preterm infants. Statistical analyses were performed by using IBM SPSS Statistics V.22.0. (IBM Corp, Armonk, New York, USA)

Results

Of the 957 ANZNN registrants who were born at 22–25 weeks gestation and at ≥401 grams and <1000 grams, 714 were eligible for inclusion in the study cohort. The overall follow-up rate for neurodevelopmental assessment was 78%. Exclusions were made according to figure 1. The baseline clinical characteristics and outcomes are described in table 1. A total of 100 infants (14.0%) were born at less than 24 weeks. More than a third (36.0%) of the infants were born of mothers who were either ≥35 years or <20 years of age. A total of 590 (82.6%) mothers presented in preterm labour, 240 (33.6%) with antepartum haemorrhage and 52 (7.3%) with hypertension in pregnancy. The distribution of the infants by gestational age: 22 weeks—10 (1.4%), 23 weeks—90 (12.6%), 24 weeks—268 (37.5%), 25 weeks—346 (48.5%). Of the infants, 529 (74.1%) were singleton births, 42 (5.9%) had intrauterine growth restriction and 625 (87.5 %) were born in a tertiary care hospital. Vaginal delivery accounted for 57.1% of all births with the proportions decreasing from 22 weeks to 25 weeks gestation. Antenatal steroids were given to 78.9% of all infants prior to delivery and 84.7% were intubated at delivery.

Figure 1

Flow diagram of study participants.

Table 1

Clinical characteristics and outcomes of study participants as stratified by gestational age

Characteristics	Gestational age
	22 weeks n=10	23 weeks n=90	24 weeks n=268	25 weeks n=346	Total n=714
Male gender (%)	7 (70)	52 (57.8)	137 (51.1)	193 (55.8)	389 (54.5)
Birth weight, gms (%)
<500	3 (30)	7 (7.8)	12 (4.5)	11 (3.2)	33 (4.6)
501–700	7 (70)	77 (85.6)	160 (59.7)	82 (23.7)	326 (45.7)
701–1000	0 (0)	6 (6.7)	96 (35.8)	253 (73.1)	355 (49.7)
Mean birth weight, gm (SD)	542.8 (60.1)	598.8 (67.5)	671.9 (92.0)	773.6 (121.7)	710.2 (124.2)
Indigenous status (%)	0 (0)	3 (3.3)	9 (3.4)	9 (2.6)	21 (2.9)
Vaginal delivery (%)	10 (100)	73 (81.1)	165 (61.6)	160 (46.2)	408 (57.1)
Intubated at delivery (%)	9 (90)	84 (93.3)	236 (88.1)	276 (79.8)	605 (84.7)
Antenatal corticosteroids (%)	4 (40)	72 (80)	228 (85.1)	259 (74.9)	563 (78.9)
Singleton (%)	9 (90)	65 (72.2)	195 (72.8)	260 (75.1)	529 (74.1)
Mean CRIB2 Score (SD)	19.3 (1.3)	16.1 (1.7)	14.5 (1.5)	12.8 (1.6)	13.9 (2.0)
Any major morbidity (%)	6 (60)	66 (73.3)	173 (64.6)	213 (61.6)	458 (64.1)
Moderate to severe neurodevelopmental impairment (%)	1 (10)	13 (14.4)	28 (10.4)	33 (9.5)	75 (10.5)
Death
Overall (%)	7 (70)	47 (52.2)	130 (48.5)	104 (30.1)	288 (40.3)
Predischarge death (%)	7 (70)	46 (51.1)	128 (47.8)	97 (28)	278 (38.9)
Death/major morbidity (%)	10 (100)	81 (90)	233 (86.9)	254 (73.4)	578 (81.0)
Death/moderate to severe neurodevelopmental impairment (%)	8 (80)	60 (66.7)	158 (59)	137 (39.6)	363 (50.8)

Note: Overall neurodevelopmental assessment follow-up rate was 78%. CRIB: Clinical Risk Index for Babies.

A total of 458 infants (64.1%) were noted to have at least one of the major neonatal morbidities and 75 (10.5%) were noted to have neurodevelopmental impairment at 2–3 years corrected age. The overall number of deaths in this cohort was 288 (40.3%), of which 278 (38.9%) occurred prior to discharge. The composite outcome of death and/or any major morbidity was noted in 578 (81.0%) infants and death or neurodevelopmental impairment in 363 (50.8%) infants.

The precision of NICHD calculator was low for prediction of death in our cohort, with an overall AUC of 0.65 (0.61–0.69) (figure 2). The accuracy of the calculator for predicting death did not change substantially when stratifying by inborn or outborn status, (AUC 0.65 (0.60–0.70) compared with 0.65 (0.53–0.77), respectively) (supplemental figures 1 and 2). When stratified by gestational age and compared with the outcome of death, the AUC ranged from 0.56 to 0.61 (figure 2). For the composite outcome of death or neurodevelopmental impairment, the accuracy of the NICHD for predicting this outcome was similarly low with an AUC of 0.65 (0.61–0.69) (figure 3). This also did not change substantially when stratified by inborn or outborn status, (AUC 0.66 (0.61–0.70) compared with 0.66 (0.54–0.77), respectively) (supplemental figures 1 and 2). When compared with actual stratified gestational age outcomes, the AUC ranged from 0.57 to 0.62 (figure 3).

Figure 2

ROC of predicted death compared with actual death as stratified by gestational age (23–25 weeks). Note: ROC for 22 weeks gestation age not included due to low numbers. AUC, area under the curve; ROC, receiver operative characteristics.

Figure 3

ROC for predicted combined death or neurodevelopmental impairment compared with actual death or neurodevelopmental impairment as stratified by gestational age (23–25 weeks). Note: ROC for 22 weeks gestation age not included due to low numbers. AUC, area under the curve; ROC, receiver operative characteristics.

Table 2 illustrates the comparison of the predicted death according to the NICHD calculator and the actual deaths in our cohort as stratified by gestational age. For the overall cohort, the actual rate of death of 27.5% exceeded the predicted probability for the group with the lowest estimated probability of death (<20%). The actual rate of death for this group with a predicted probability of <20% ranged between 25% and 100% for infants 23–25 weeks gestation. Comparison of the predicted and actual composite outcomes of death or neurodevelopmental impairment showed similar inaccuracy of the calculator for those with the highest and lowest predicted probabilities of this outcome (table 3). The actual rate of death or neurodevelopmental impairment for the overall cohort was 33.0% for the lowest predicted group of <20% and was 69.4% for those in the highest predicted group of >80%.

Table 2

Comparison of predicted death probability versus actual death

Gestational age	Actual death	Predicted death probability, n (%)
		<20%	20%–39.9%	40%–59.9%	60%–79.9%	≥80%	Total
22 weeks	Yes	NA	NA	NA	1 (25)	6 (100)	7 (70)
	No	NA	NA	NA	3 (75)	NA	3 (30)
	Total	NA	NA	NA	4 (100)	6 (100)	10(100)
23 weeks	Yes	1 (100)	1 (50)	19 (46.3)	23 (53.5)	3 (100)	47 (52.2)
	No	NA	1 (50)	22 (53.7)	20 (46.5)	NA	43 (47.8)
	Total	1 (100)	2 (100)	41(100)	43(100)	3 (100)	90(100)
24 weeks	Yes	8 (57.1)	58 (40.6)	55 (56.1)	9 (69.2)	NA	130 (48.5)
	No	6 (42.9)	85 (59.4)	43 (43.9)	4 (30.8)	NA	138 (51.5)
	Total	14(100)	143(100)	98(100)	13(100)	NA	268(100)
25 weeks	Yes	46 (24.9)	45 (32.8)	12 (52.2)	1 (100)	NA	104 (30.1)
	No	139 (75.1)	92 (67.2)	11 (47.8)	NA	NA	242 (69.9)
	Total	185(100)	137(100)	23(100)	1 (100)	NA	346(100)
22–25 weeks	Yes	55 (27.5)	104 (36.9)	86 (53.1)	34 (55.7)	9 (100)	288 (40.3)
	No	145 (72.5)	178 (63.1)	76 (46.9)	27 (44.3)	NA	426 (59.7)
	Total	200(100)	282(100)	162(100)	61(100)	9 (100)	714(100)

Table 3

Comparison of predicted versus actual deaths or moderate to severe neurodevelopmental impairment

Gestational age	Actual death	Predicted death or NDI probability, (%)
		<20%	20%–39.9%	40%–59.9%	60%–79.9%	≥80%	Total
22 weeks	Yes	NA	NA	NA	NA	8 (80)	8 (80)
	No	NA	NA	NA	NA	2 (20)	2 (20)
	Total	NA	NA	NA	NA	10(100)	10(100)
23 weeks	Yes	1 (100)	NA	11 (68.8)	NA	48 (66.7)	60 (66.7)
	No	NA	1 (100)	5 (31.3)	NA	24 (33.3)	30 (33.3)
	Total	1 (100)	1 (100)	16(100)	NA	72(100)	90(100)
24 weeks	Yes	1 (50)	2 (100)	22 (41.5)	98 (60.9)	35 (70)	158 (59)
	No	1 (50)	NA	31 (58.5)	63 (39.1)	15 (30)	110 (41)
	Total	2 (100)	2 (100)	53(100)	161(100)	50(100)	268(100)
25 weeks	Yes	2 (22.2)	25 (32.9)	58 (39.7)	43 (41.7)	9 (75)	137 (39.6)
	No	7 (77.8)	51 (67.1)	88 (60.3)	60 (58.3)	3 (25)	209 (60.4)
	Total	9 (100)	76(100)	146(100)	103(100)	12(100)	346(100)
22–25 weeks	Yes	4 (33.3)	27 (34.6)	80 (40)	152 (54.3)	100 (69.4)	363 (50.8)
	No	8 (66.7)	51 (65.4)	120 (60)	128 (45.7)	44 (30.6)	351 (49.2)
	Total	12(100)	78(100)	200(100)	280(100)	144(100)	714(100)

NDI, neurodevelopmental impairment.

Discussion

In this study, we attempted to determine the validity of the 2009 NICHD calculator for the prediction of outcomes in extremely preterm infants in our contemporary Australia and New Zealand population 2009–2010 cohort. We found the calculator to have underperforming predictive ability in determining the rates of death and the composite outcome of death or moderate to severe neurodevelopmental impairment. This predictive capability did not improve when stratified by birth in a tertiary centre or outside a tertiary centre. Specifically, the calculator was less accurate with predictions at the extreme categories of predicted outcomes. When stratified and compared with actual gestational age outcomes, we found the calculator to have marginally better predictive ability for both outcomes.

The predictive capability of the calculator in our cohort differed somewhat to previous reports of comparisons in the USA and Australia. In an attempt to validate the accuracy of the NICHD calculator in a cohort from California, Lee et al concluded that the model improved the prediction of outcomes compared with gestational age based outcomes in their setting of mostly tertiary level academic and community NICUs.10 In their study the authors determined that the NICHD model resulted in a higher true-positive rate and a lower false-positive rate in this cohort. However, this study was limited to predischarge outcomes and did not assess or compare neurodevelopment. In another cohort study of 114 infants from Victoria, Australia, Boland et al demonstrated that the NICHD calculator was good at predicting the outcomes of death in inborn infants (AUC 0.8 (0.7–0.9)) but was not as precise for outborns (AUC 0.6 (0.3–0.9)).11

Our study expands on the earlier study by Boland et al with the inclusion of more infants (714 infants) from all tertiary level NICUs from Australia and New Zealand. This allows for better generalisability of the results for the population. Our results indicate that the NICHD calculator does not seem to accurately predict outcomes in our contemporary cohort.15 Specifically, overestimation of the probability of death or neurodevelopmental impairment at the highest risk group may affect decisions surrounding initiation and continuation of intensive care. While it is clear from many studies that survival and outcomes cannot be accurately predicted by gestational age alone, there is a need to validate the accuracy of available models and to potentially develop a prediction model that is based on local data for the population. The provision of accurate and current data on outcomes and prognosis is vital to inform decisions made by parents and the medical staff.

The strength of our study is that it includes all the tertiary level NICUs within Australia and New Zealand, making this a good representation of the extremely preterm population. The standardised definitions and rigorous data crosschecking by ANZNN ensures completeness and validity of the data obtained. The NICHD calculator was developed with the inclusion of only mechanically ventilated infants as an indicator of active provision of intensive care. However, with the high usage of early CPAP for many units in our cohort,16 we felt that inclusion of such infants is justified as it would constitute the provision of intensive care for which the calculator is currently being used. There were no babies who died without mechanical ventilation within the first day of life in our study, which likely indicates that there was no active palliation in our study cohort. Even so, it is likely that our inclusion of infants who survive without mechanical ventilation would increase the number of infants with favourable outcome than would be predicted.

While blindness, deafness requiring amplification, GMFCS levels 2 to 5 cerebral palsy were similarly included in the definition of moderate to severe neurodevelopmental impairment for the NICHD calculator, there were some notable differences in the classification of neurodevelopmental impairment in our study and that of the initial NICHD study. For the outcomes calculator, neurodevelopmental impairment was defined as a score of ≤70 (≥2 SD below the mean) on either the Psychomotor Developmental Index or Mental Developmental index of the BSID-Second Edition (BSID-II). The ANZNN uses the definition of >2 SD below the mean on either the cognitive, language or motor scales of BSID-III. Several studies have highlighted the difference between the two tests, where BSID-III tends to underestimate the prevalence of neurodevelopmental impairment by 2%–7%.17–19 There have also been reported difficulties in trying to reconcile the cut-offs between the two tests for the individual components—cognitive and language.20 This could account for the suboptimal accuracy of the NICHD calculator in predicting neurodevelopmental impairment as the ANZNN cut-off may be an underestimate of the incidence when compared with the calculator. Due to the differences in the tests between the study cohorts, the composite outcome of death or profound impairment was also not compared between the groups. The timing of the developmental assessment was at an older age for the ANZNN cohort (≥18 months corrected age with the majority or assessments performed at 2–3 years corrected age) where the NICHD cohort screened infants at 18–22 months. Nonetheless, these early assessments at these time points should similarly provide an important indication of any potential developmental issues that require early intervention.

Conclusion

The NICHD outcomes model does not provide an accurate prediction of outcomes in the extremely preterm infants in the ANZNN cohort. There is a need to validate available models for use in different population cohorts and for the development of neonatal outcomes prediction models that are based on local geographical data, such as that for infants born in Australia and New Zealand.