Identification of practices and morbidities affecting the mortality of very low birth weight infants using a multilevel logistic analysis: clinical trial or standardisation?

OBJECTIVES: To determine the feasibility of clinical trials of newly developed treatments or standardisation of existing practices to further improve outcomes among very low birth weight (VLBW) infants, a nationwide database was analysed with a two-dimensional approach using two multivariate logistic models.
DESIGN: Retrospective observational analysis.
SETTING: Level III perinatal centres in Japan. PARTICIPANTS: 15 920 VLBW infants admitted at 38 participating centres from 2003 through 2010. OUTCOME MEASURES: Clinical information for the infants was collected until discharge from the centres. A multivariate logistic model identified practices and morbidities associated with mortality. Then, those which were significantly associated with mortality were analysed using a multilevel logistic model. The residues calculated by the multilevel analysis were used as an indicator of centre variation.
RESULTS: Among practices, antenatal steroids and intubation at birth showed relatively high centre variations (0.9 and 0.8) and favourable ORs (0.7 and 0.5) for mortality, while caesarean section showed a low centre variation (0.4) and a favourable OR (0.8). Sepsis and air leak showed high centre variations (0.4 and 0.4) and high ORs (3.8 and 3.4) among morbidities. Pulmonary haemorrhage, persistent pulmonary hypertension of the newborn, and intraventricular haemorrhage showed moderate variations (0.2, 0.3 and 0.2, respectively) and high ORs (5.6, 4.1 and 2.9, respectively). In contrast, necrotising enterocolitis showed the lowest variation (0.1) and a high OR (4.9).
CONCLUSIONS: The two-dimensional approach has clearly demonstrated the importance of clinical trial or standardisation. The practices and morbidities with low centre variations and high ORs for mortality must be improved through clinical trials of newly introduced techniques, while standardisation must be considered for practices and morbidities with a high centre variation. TRIAL REGISTRATION: The database was registered as UMIN000006961.

There exists a centre variation in practices and incidences of morbidities among high-risk infants.

If the centre variation is wide, the standardisation of established treatment is more important than the introduction of a new practice by clinical trial.

An analysis of network database may provide the necessity of clinical trial or standardisation.

Risk-adjusted centre variations of interventions and morbidities among high-risk infants were calculated using a multilevel analysis.

The practices and morbidities with low centre variations and high ORs for mortality must be improved through clinical trials of newly introduced techniques.

In contrast, standardisation must be considered for practices and morbidities with a high centre variation and high ORs for mortality.

All analysis was derived from a large database.

Limited number of hospitals participated in the study.

Introduction

Although there have been considerable advances in neonatal care, there is still significant room to improve outcomes in very low birth weight (VLBW) infants.1–6 If some interventions or morbidities are strongly associated with poor outcomes, they should be improved through newly introduced treatments. However, it is also true that there is centre variation in interventions and the incidences of morbidities.7 Routine practices may vary even among level III neonatal intensive care units (NICUs). If this centre variation is associated with increased mortality, the standardisation of these practices is more pressing than the introduction of a new treatment for the improvement of outcomes of VLBW infants. Because the practices and morbidities in hospitals can be affected by both the relevance of risk and centre variation, a two-dimensional approach using two multivariate logistic models was used in this study. The first dimension estimated the risk of an individual practice or morbidity in association with mortality using a linear logistic model by controlling for background risk factors. The second dimension evaluated the centre variation in practices and morbidities using a multilevel logistic analysis including individual hospital as an independent variable. The practices and morbidities with low centre variations and high ORs for mortality must be improved through clinical trials of newly introduced techniques. In contrast, if the centre variation is high and the OR of the intervention indicates a decrease in mortality, the standardisation of established treatments among hospitals through the implementation of guidelines is more important than a newly introduced clinical trial. Thus, we hypothesised that this type of approach of two-dimensional plotting is useful to distinguish between clinical trials and standardisation to further improve outcomes among VLBW infants.

Subjects and methods

Study design

The study is an observational analysis of the neonatal database. All data were retrospectively analysed.

Patient selection

A neonatal research network database in Japan was used in the current study. The database included infants with birth weights at or less than 1500 g who were treated in participating neonatal centres. To characterise the risk of each practice or morbidity with mortality and their centre variation among hospitals, 17 156 infants born from 2003 through 2010 at 38 hospitals that participated in the network throughout the 8 years were analysed. Among all the infants, 33 infants died in the delivery room, and 1168 infants with major congenital anomalies were excluded from the study because mortality in those infants was beyond the quality of NICU care. Furthermore, 35 infants were also excluded due to incomplete data registration. Thus, 15 920 infants were included in the study (figure 1). All 38 hospitals were designated as level III perinatal centres. The definitions of the collected variables were as previously reported, and are available on the web (http://plaza.umin.ac.jp/nrndata/).7

Figure 1

Flow chart of registration and evaluation. Total 17 156 infants whose birth weight at or less than 1500 g were registered on the database. Among them, 33 infants with delivery room death regardless of vigorous resuscitation, 1168 infants with major congenital anomalies and 35 infants with incomplete registration were excluded from the study. Thus, the number of infants evaluated was 15 920, which were reported from 38 hospitals during the study year 2003 through 2010.

Statistics

Identifying risk factors at birth for mortality

To identify risk factors at birth for mortality among VLBW infants, a linear logistic model was introduced using dead in the NICU as a dependent factor. The risk factors tested were maternal age, primipara, multiple pregnancy, pregnancy-induced hypertension (PIH), diabetes mellitus, clinically diagnosed chorioamnionitis, fetal heart rate abnormalities (NRFS, non-reassuring fetal status), delivery presentation, mode of delivery, gestational age, birth weight, gender and 1 min Apgar score. Independent variables of mortality were used in this model to adjust background risks of the infants for the following analyses.

Calculating ORs of practices and morbidities for mortality

To calculate the odds for mortality, another linear logistic regression model was established. In this model, all of the above variables that were independent risk factors for mortality were included. Furthermore, each practice or morbidity was included as an independent variable in the model. To evaluate positive risks for mortality, each practice and morbidity was converted to produce ORs more than 1.0, if necessary.

The hospital practices analysed for association with mortality in the infants were antenatal steroids (ANSs), caesarean section (C/S), neonatal transport, cord blood transfusion, oxygen use at birth, intubation at birth, continuous positive airway pressure, mechanical ventilation, high-frequency oscillatory ventilation, pulmonary surfactant, inhaled nitric oxide, indomethacin, patent ductus arteriosus (PDA) ligation, glucocorticoid for chronic lung disease (CLD) and intravenous alimentation.

The morbidities analysed among the infants were respiratory distress syndrome (RDS), air leak syndrome, pulmonary haemorrhage, persistent pulmonary hypertension of the newborn (PPHN), CLD at 28 days after birth, CLD at 36 weeks of corrected age, symptomatic PDA, late-onset adrenal insufficiency of prematurity, intraventricular haemorrhage (IVH), IVH grade III/IV, periventricular leukomalacia, sepsis and necrotising enterocolitis (NEC)/intestinal perforation. All these variables were tested for their independent effect on mortality using a stepwise logistic analysis.

Calculating centre variations

Another logistic model with a multilevel analysis was applied.8 The influences of hospital policy towards interventions and patient clustering effects were analysed using hierarchical structures. The infants were set at the first level variable and the hospitals at the second level, and each practice and morbidity was included as a dependent variable in the multilevel regression model. The residues calculated by the multilevel analysis, which could not be explained with patient clustering, indicate the centre variations in practice or the incidence of morbidities among the centres. Unlike a fixed effect model calculated by analysis of variance, the residues estimated by the multilevel analysis are normally distributed variables with mean zero. Therefore, the mean and SD of residues were used as a useful indicator of centre variation. For example, a residual value of 0.1 indicates a small centre variation, whereas a value of 1.0 indicates a relatively large centre variation.

Statistical methods

All statistical analyses were performed using MLwiN V.2.2 (Center for Multilevel Modeling, University of Bristol, UK).

All information about the infants was collected anonymously, and the stored data were unlinked from individual data. The protocol of this study was approved by the Central Internal Review Board at Tokyo Women's Medical University, where all data were collected and stored. The database was registered as UMIN000006961.

Results

Risk factors at birth for mortality

The multivariate logistic model showed that multiple pregnancy, PIH, CAM, NRFS, presentation of the fetus, mode of delivery, gestational age more than 37 or less than 24 weeks, birth weight, gender and Apgar scores <4 at 1 min were considered significant independent variables associated with NICU mortality. These variables were used for adjusting the background risks of the infants for further analyses.

Risk-adjusted ORs of practices and morbidities and hospital variation

ANSs, C/S and intubation at birth were practices that were significantly associated with mortality, whereas RDS, air leak, pulmonary haemorrhage, PPHN, IVH, sepsis and NEC were morbidities significantly associated with mortality. Centre variations were calculated for these practices and morbidities. Table 1 shows the ORs and centre variations of each practice or morbidity with 95% CI.

Table 1

ORs and centre variation among practices or morbidities

Practice/morbidity	OR (95% CI)	Centre variation (95% CI)
Practice
ANS	0.7 (0.6 to 0.9)	0.9 (0.5 to 1.3)
C/S	0.8 (0.7 to 0.9)	0.4 (0.2 to 0.6)
Intubation	0.5 (0.4 to 0.7)	0.8 (0.5 to 1.2)
Morbidity
RDS	1.2 (1.0 to 1.4)	0.3 (0.2 to 0.5)
Air leak	3.4 (2.7 to 4.5)	0.4 (0.2 to 0.6)
Pulmonary haemorrhage	5.6 (4.4 to 6.9)	0.2 (0.1 to 0.4)
PPHN	4.1 (3.4 to 5.0)	0.3 (0.1 to 1.4)
IVH	2.9 (2.5 to 3.3)	0.2 (0.1 to 0.3)
Sepsis	3.8 (3.2 to 4.4)	0.4 (0.2 to 0.6)
NEC	4.9 (3.9 to 6.2)	0.1 (0.0 to 0.2)

ANS, antenatal steroid; C/S, caesarean section; Intubation, resuscitation with intubation at birth; RDS, respiratory distress syndrome; PPHN, persistent pulmonary hypertension of the newborn; IVH, intraventricular haemorrhage; NEC, necrotising enterocolitis.

Two-dimensional plotting

Figures 2 and 3 show the two-dimensional distribution of ORs for mortality and centre variations. Among practices, ANSs and intubation at birth showed relatively high centre variations and favourable ORs for mortality, whereas C/S showed a low centre variation and the same favourable OR. Sepsis and air leak showed high centre variations and high ORs for mortality among morbidities. Pulmonary haemorrhage, PPHN and IVH showed moderate variations and high ORs. In contrast, NEC showed the lowest variation, with a high OR.

Figure 2

Distribution of ORs for mortality and centre variations in practices. The x axis shows risk-adjusted ORs of each practice for mortality among VLBW infants. The y axis shows the risk-adjusted centre variation of each practice among 38 neonatal intensive care units. Vertical and horizontal bars represent 95% CIs. ANS, antenatal steroid; intubation, resuscitation with intubation at birth; C/S, caesarean section; VLBW, very low birth weight.

Figure 3

Distribution of ORs for mortality and centre variations in morbidities. The x axis shows risk-adjusted ORs of each morbidity for mortality among VLBW infants. The y axis shows the risk-adjusted centre variation of each morbidity among 38 NICUs. Vertical and horizontal bars represent 95% CIs. IVH, intraventricular haemorrhage; NEC, necrotising enterocolitis; NICU, neonatal intensive care unit; PPHN, persistent pulmonary hypertension of the newborn; RDS, respiratory distress syndrome; VLBW, very low birth weight.

Discussion

The two-dimensional approach described here clearly distinguished between the standardisation of established treatments and the introduction of new treatment for the further improvement of outcomes among VLBW infants as we hypothesised. If there is a wide centre variation in practices or morbidities, standardising current practices or preventing morbidities must be considered first. In contrast, if the centre variation is small, a new intervention for improvement needs to be tested.

ANSs and intubation at birth were among practices that had a less than 1 OR for mortality and a high centre variation. For these practices, standardisation should be introduced for improvement. Specifically, the benefit of ANSs is already well proved. Thus, the standardisation of this practice would not be difficult. Intubation at birth seems to be favourable for saving VLBW infants. However, the beneficial effect on morbidities, such as CLD and retinopathy of prematurity, must be considered from a different point of view. C/S showed an OR less than 1 and low centre variation. Thus, a clinical trial to demonstrate the efficacy of C/S when delivering VLBW infants is necessary before it is used as a routine practice.

Among morbidities, sepsis and air leak showed high centre variations and high ORs for mortality. It would be difficult to introduce a new intervention to reduce these morbidities before standardising daily practices in NICUs. If some NICUs with these high morbidities can change their routine practices to reduce their incidence, it may be more effective rather than develop new treatments. IVH, PPHN and pulmonary haemorrhage had high ORs for mortality. However, their centre variations were moderate. For these morbidities, standardisation and the development of new treatment will be essential. In contrast, the OR of NEC was high, while its centre variation was the smallest among the morbidities. Although the incidence of NEC is very low in Japan, the mortality rate of the infants with NEC is still high. This result indicates that new treatment is necessary to reduce this morbidity.

We have often experienced that the introduction of a newly invented intervention with a high expectancy failed to prove its efficacy in a clinical trial. In this case, wide centre variation might compromise the benefit of the intervention. The importance of surveying centre variation was previously reported.9 Furthermore, centre variation actually impaired several important clinical trials.10 11 We believe that our analysis can answer the question of which comes first, clinical trial or standardiation.

The limitation of this study is that the analysis was performed only among 38 hospitals, and therefore, the trend shown in the study does not reflect a nationwide phenomenon. However, these hospitals are leading NICUs, and covering 30% of the total VLBW infants born in Japan. Thus, the centre variations among these NICUs are hypothesised to be the smallest in Japan. If we included all NICUs in the country, we would have observed wider centre variations. We believe that the analysis of a limited number of hospitals is appropriate for this type of study. Additionally, the database does not include information about the timing of morbidities. Thus, a separate time-to-event analysis on each morbidity is necessary. Furthermore, the direct relationship between interventions and mortality was not evaluated in this study. Thus, the recommendation of C/S and intubation at birth could not be warranted to all VLBW infants.

In conclusion, the simultaneous evaluation of the risks and the centre variations in practices or morbidities are useful to find the new strategy for the further improvement of outcomes in VLBW infants. This kind of approach is also reasonable and important in another field of medicine, if there is the probability of centre variations in practices and morbidities.