Measuring quality of care in moderate and late preterm infants.

OBJECTIVE: To examine quality measures for moderate and late preterm (MLP) infants. STUDY
DESIGN: By prospectively analyzing Vermont Oxford Network's all NICU admissions database, we adapted Baby-MONITOR, a composite quality measure for extremely/very preterm infants, for MLP infants. We examined correlations between the adapted MLP quality measure (MLP-QM) in MLP infants and Baby-MONITOR in extremely and very preterm infants. RESULT: We studied 376,219 MLP (30-36 weeks GA) and 57,595 extremely/very preterm (25-29 weeks GA) infants from 465 U.S. hospitals born from 2016 to 2020. MLP-QM summary scores in MLP infants had weak correlation with Baby-MONITOR scores in extremely and very preterm infants (r = 0.47). There was weak correlation among survival (r = 0.19), no pneumothorax (r = 0.35), and no infection after 3 days (r = 0.45), but strong correlation among human milk at discharge (r = 0.79) and no hypothermia (r = 0.76).
CONCLUSION: Modest correlation among hospital care measures in two preterm populations suggests the need for MLP-specific care measures.

Introduction

Moderate and late preterm (MLP) infants (born between 32 and 366/7 weeks’ gestation) comprise 84% of all preterm births in the United States (U.S.). (1) Compared to full-term infants (≥39 weeks’ gestation), MLP infants are at increased risk for prematurity-associated complications including respiratory distress syndrome (RDS), intraventricular hemorrhage (IVH), adverse neurodevelopmental outcomes, and death. (2,3) The mean total newborn hospitalization costs for the MLP population are nine times that of full-term infants. (4) Additionally, there is significant variation in MLP infant care between hospitals, such as the frequency of mechanical ventilation and provision of parenteral nutrition. (5)

Despite such variation in outcomes and care between hospitals, there are no population specific quality measures of care for the hospitalized MLP population, the majority of NICU patients. For the most preterm population, Baby-MONITOR (Measure Of Neonatal InTensive care Outcomes Research), a composite indicator of care quality for extremely and very preterm infants, has been developed. (6,7) Associated studies using Baby-MONITOR highlight variation in quality across hospitals with the same neonatal care level (8) and in care within and between hospitals by race and ethnicity. (9–11) Similar variation in care has been suggested in the early term and term population as assessed by the “Unexpected Complications among Term Newborns” quality measure, although much of this variation is in admission to neonatal intensive care units (NICUs) and hospital transfers. (12,13) Neither measure has been adapted for the MLP population and thus, similar insights into this large and resource intensive population are undefined.

To improve the care of the MLP population, we need to assess the quality of care provided in the NICU. Although previously validated quality measures, such as Baby-MONITOR, offer a starting point, many gestational age (GA) specific components in Baby-MONITOR or the “Unexpected Complications among Term Newborns” do not apply to the MLP population. Additional measures will likely be necessary to fully capture the quality of care provided to this population. The objective of this study was to examine how Baby-MONITOR and other potential measures of care quality relevant to the MLP population perform for assessing variation in the quality of care as measured in a large, national representative database of NICU admissions. To explore how quality of care for extremely and very preterm versus MLP infants may be correlated, we compared Baby-MONITOR scores to an adapted MLP measure (MLP-QM) within hospitals.

Materials/Subjects and Methods

Population

We conducted a prospective analysis of measures of care quality using data from the all NICU admissions database maintained by Vermont Oxford Network (VON), a voluntary worldwide community dedicated to improving the quality, safety, and value of perinatal and neonatal care. (14) The database includes: 1) VLBW (≤1500g) infants who are admitted anywhere in a hospital or die anywhere in a hospital within 28 days of birth; and 2) infants with a birth weight >1500g admitted to a NICU within 28 days of birth, where a NICU is defined as a location within a hospital where continuous positive airway pressure (CPAP) or intermittent mechanical ventilation (MV) can be provided to infants (not including the delivery room or another location where respiratory interventions are provided briefly for stabilization) or infants with a birth weight >1500g who die anywhere in the hospital within 28 days of birth. (15)

To identify our analytic cohort, we excluded: infants with serious congenital anomalies (n=24,825); deaths in the delivery room or within 12 hours of admission to the NICU (n=1,213); infants transferred more than once (n=798); and infants with implausible values for birth weight (n=1,168). Implausible values for birthweight were defined as birth weight less than 201g or greater than four standard deviations from the mean for GA and sex. Infants were then divided into two groups by GA: MLP infants of 300/7 to 366/7 weeks GA and extremely and very preterm infants from 250/7 to 296/7 weeks GA. We examined infants from 300/7 to 366/7 weeks GA as these infants are not captured in existing quality measures. (7,16)

The institutional review board of the University of Vermont determined the use of the VON research repository for this analysis was not human subjects research.

Quality Measures

Baby-MONITOR consists of both infant-level process measures (any human milk at discharge, no admission hypothermia, no health care-associated infection, antenatal steroid exposure, timely retinal examination) and outcome measures (survival to hospital discharge, no pneumothorax, no chronic lung disease, greater than median growth velocity). (6) We examined the distribution of the Baby-MONITOR summary scores and the components in both the MLP and the extremely and very preterm population at the infant level. The adapted MLP-QM kept measures relevant to all preterm infants: any human milk at discharge, no admission hypothermia, no health care-associated infection, survival to hospital discharge, and no pneumothorax. Baby-MONITOR components specific to the most preterm infants were replaced with new MLP relevant measures after examination of all proposed measures in the creation of Baby-MONITOR. (6) Antenatal steroid exposure and timely retinal examination are less applicable to the MLP population (Supplemental Table 1). Chronic lung disease (CLD) is defined among infants born before 33 weeks because the traditional definition of oxygen use at 36 weeks corrected GA may lead to overestimation of lung disease in older infants (Supplemental Table 1). (17) As the “no CLD” measure is not applicable to all MLP infants, it was replaced with no oxygen at 28 days or no oxygen at discharge if discharged earlier than 28 days. (5,17) Although the Baby-MONITOR growth velocity measure has only been validated in the VLBW population, growth remains a critical aspect in MLP infant care. (18) Change in weight z-score was included in the adapted MLP-QM as it has been validated in populations with wide ranges of gestational ages to capture significant weight loss regardless of birth weight. (19) Additionally, we included extreme length of stay as several studies have demonstrated variation in the postmenstrual age at discharge in the MLP population. (20–22) Extreme length of stay was defined as total hospital stay greater than the 95th percentile for the predicted value, based on a multivariable risk adjustment model including birth weight, ventilation status, respiratory distress syndrome, surgery (other than for retinopathy of prematurity [ROP]), 1 minute APGAR score, small for gestational age, reason for transfer, vaginal delivery, inborn/outborn, sex, prenatal care, and major birth defect. (23–27)

Measures in both the Baby-MONITOR and MLP-QM score were adjusted for relevant infant characteristics (GA, sex, 5-minute Apgar score, whether the mother received prenatal care, whether the infant was inborn or outborn, small for GA, part of a multiple birth, or born by cesarean delivery) as in prior work (Supplemental Table 2). (6,10,28) Oxygen use at 28 days of life and CLD were adjusted for the center’s elevation. (29) Measures were calculated to appropriately attribute events for infants transferred between hospitals after birth by excluding infants if they were either transferred out by day 3 from the growth, infection, and human milk measures or if they were admitted after day 3 for the oxygen measures. Consistent with previous studies of Baby-MONITOR, measures were standardized relative to other NICUs in the dataset. (28,30) For component scores, the estimate was calculated by observed minus expected percentage for the hospital, divided by the standard error, and scaled so the standard deviation across hospitals equals one. The standardized scores for each component of the composite measure were equally weighted after placing them on a common scale and averaged to derive the summary Baby-MONITOR and MLP-QM scores. A higher score indicates higher quality of care. Further details regarding the quality score methodology used in this analysis are described in prior work. (6,7,31)

Statistical Analysis

For each hospital, we derived scores for Baby-MONITOR and each of its components based on extremely and very preterm infants, and scores for MLP-QM and each of its components based on MLP infants. To receive scores, hospitals needed data on at least one infant for each measure. We examined the distributions of MLP-QM summary scores and components overall and by NICU level. We used Pearson correlation coefficients to determine the correlation of the summary MLP-QM score in MLP infants with the Baby-MONITOR score in extremely and very preterm infants cared for at the same hospital. We also compared the individual components of MLP-QM between MLP and extremely and very preterm infants within hospitals. Correlations were rated as weak (<0.5), moderate (0.5 – 0.75), strong (0.75 – 0.9), very strong (0.9 – 1), or perfect (1). (32) R version 4.0.2 was used for all data analyses.

Results

We included infants at 465 U.S. hospitals born at 25–36 completed weeks’ gestation between January 2016 and December 2020. After applying exclusion criteria, 433,814 infants were eligible for analysis: 376,219 (87%) moderate and late preterm (30–36 weeks GA) and 57,595 (13%) extremely and very preterm (25–29 weeks GA).

Table 1 compares population demographics in the MLP population with the extremely and very preterm population at the infant and NICU level. Infants in both groups had similar rates of multiple gestation and prenatal care as well as similar 5-minute Apgar scores. Extremely and very preterm infants were more likely to be delivered via cesarean delivery.

Table 1.

Characteristics of 433,814 Infants 25–36 Weeks’ Gestational Age Reported by 465 U.S. Hospitals

Characteristics	30 – 36 Weeks Gestational Age (Moderate and Late Preterm)			25 – 29 Weeks Gestational Age (Extremely and Very Preterm)
	Infant Level		NICU Level	Infant Level		NICU Level
	N	No. (%) or Mean (SD)	Median [IQR]	N	No. (%) or Mean (SD)	Median [IQR]
Gestational age at birth, wks	376219	34.2 (1.7)	34.3 [34.1,34.5]	57595	27.7 (1.4)	27.8 [27.6,28.0]
Birth weight, g	376168	2225 (575)	2248 [2183,2301]	57591	1036 (275)	1052 [1020,1097]
Small for gestational age	376147	41900 (11.1)	11.0 [9.2,12.7]	57582	4976 (8.6)	7.2 [4.0,9.9]
Male	376198	204181 (54.3)	54.4 [52.7,56.0]	57586	30173 (52.4)	52.8 [49.2,56.6]
Apgar score at 5 min	373795	8.3 (1.2)	8.3 [8.2,8.5]	56985	7.1 (1.8)	7.2 [6.9,7.5]
Cesarean delivery	376125	227884 (60.6)	59.7 [55.4,64.9]	57584	42210 (73.3)	72.7 [67.9,77.7]
Multiple gestation	376202	90892 (24.2)	23.1 [20.0,26.3]	57590	13637 (23.7)	22.4 [16.9,27.5]
Any prenatal care	375218	363520 (96.9)	97.2 [95.6,98.4]	57394	54991 (95.8)	96.4 [93.3,98.4]
Admissions			644 [360,1138]			83 [36,179]
Inborn	376219	334704 (89.0)	94.4 [87.0,98.5]	57595	48987 (85.1)	92.6 [84.3,98.5]
Baby-MONITOR and MLP-QM measures
Any human milk at discharge	372077	249536 (67.1)	68.2 [58.2,76.5]	52624	25015 (47.5)	48.2 [38.7,60.4]
Hypothermia on admission	370493	19971 (5.4)	2.1 [0.9,6.9]	56418	3465 (6.1)	3.4 [0.6,7.9]
Any infection after day 3	365397	1752 (0.5)	0.4 [0.1,0.7]	56642	4874 (8.6)	7.2 [4.2,10.3]
In hospital mortality	375788	1445 (0.4)	0.3 [0.1,0.6]	57251	3600 (6.3)	6 [3.8,8.1]
Pneumothorax	376153	5879 (1.6)	1.5 [0.9,2.1]	57556	2359 (4.1)	3 [0,5.2]
Original Baby-MONITOR measures
Chronic lung disease	82200	5056 (6.2)	4.2 [2,7]	53288	16589 (31.1)	25.7 [15.5,36.7]
Greater than median growth velocity	28973	14785 (51)	51 [42.3,60.7]	48016	24508 (51)	50.7 [41.9,60]
Any antenatal Steroids	134546	115223 (85.6)	86.1 [79.7,90]	57337	50047 (87.3)	86.7 [79.2,91.5]
Timely eye examination	18897	14648 (77.5)	77.4 [63.3,90.4]	53803	50472 (93.8)	95 [88.6,97.8]
New MLP-QM measures
Extreme length of stay	373022	17438 (4.7)	3.7 [2.1,5.5]	53551	3196 (6)	4.7 [1.9,7.7]
Oxygen on day 28 or earlier discharge	359339	9538 (2.7)	1.5 [0.7,2.7]	51882	28266 (54.5)	53.8 [39.3,64.2]
Greater than median weight z-score change	371074	185349 (49.9)	49.9 [43.5,58.2]	53431	26714 (50)	50 [38.5,59.5]

Infants that were greater than 30 weeks gestational age but less than 1500g were classified according to their gestational age. Please see the methods for additional details.

Table 1 also compares Baby-MONITOR and MLP-QM components in the MLP population with the extremely and very preterm population. Of the components common to Baby-MONITOR and MLP-QM, MLP infants were more likely to be discharged home on human milk (Median 68.2% [IQR 58.2, 76.5] vs. Median 48.2% [IQR 38.7, 60.4]), have lower in-hospital mortality (Median 0.3% [IQR 0.1, 0.6] vs. Median 6% [IQR 3.8, 8.1], and have lower infection after day of life 3 (Median 0.4% [IQR 0.1, 0.7] vs. Median 7.2% [4.2,10.3]). Of the adapted new MLP-QM measures, extreme length of stay and greater than median weight z-score change were similar in the MLP and extremely and very preterm populations. All MLP-QM components demonstrated variability at the hospital level, with the majority having normal distributions (Supplemental Figure 1).

Figure 1 displays a hospital level comparison of the summary Baby-MONITOR score in extremely and very preterm infants with the MLP-QM score in MLP infants (Pearson correlation coefficient r = 0.47). The weak correlation between scores suggests that hospitals that have high scores in the MLP-QM for MLP infants may not have corresponding high scores in Baby-MONITOR for the extremely and very preterm infants.

Figure 1:

Correlation of Summary Adapted MLP Quality Measure Scores in MLP Infants with Baby-MONITOR Scores in Extremely and Very Preterm Infants

Figure 2 depicts the correlation between the individual components of the adapted MLP-QM scores in the MLP population with the extremely and very preterm population. There was weak correlation among survival (r = 0.19), no pneumothorax (r = 0.35), and no infection after 3 days (r = 0.45). There was strong correlation among human milk at discharge (r = 0.79) and no hypothermia (r = 0.76). There was moderate correlation for no extreme length of stay (r = 0.59), no oxygen at 28 days (r = 0.52), change in weight z score greater than median (r = 0.54), and overall MLP monitor score (r = 0.59).

Figure 2:

Correlation of Adapted MLP Quality Measure Components in MLP Infants and Extremely and Very Preterm Infants

Discussion

We adapted Baby-MONITOR, an existing measure of care quality designed for the extremely and preterm infant population, for the MLP population to create an adapted MLP quality measure, MLP-QM. We found weak correlation between MLP-QM summary scores in MLP infants and Baby-MONITOR summary scores in extremely and very preterm infants at the hospital level. While some components, such as human milk at discharge and no hypothermia, were highly correlated, there was weak correlation of survival, no pneumothorax, and no infection after 3 days between MLP and extremely and very preterm infants cared for at the same hospital. These inconsistent correlations suggest that while hospital care practices addressing some areas, such as human milk feeding or thermogregulation, are equally robust in MLP and extremely and very preterm infants, other hospital care practices may vary greatly in the two populations.

We aimed to assess whether an established VLBW quality measure, Baby-MONITOR, could be adapted to potentially evaluate the quality of care in the MLP population. (7) Given the high prevalence of MLP infants in the NICU, assessing the quality of care provided to these infants is critical to improving NICU care. (1) However, Baby-MONITOR contains several components that are not applicable for the MLP population: antenatal steroids, timely ROP exam, and no CLD. (6) Adapting Baby-MONITOR to include oxygen use at 28 days, extreme length of stay, and greater than median z-score gives MLP-QM improved face validity as these outcomes are relevant and common in the MLP population. Additionally, several studies have demonstrated variation in the postmenstrual age at discharge and degree of respiratory support provided in the MLP population, suggesting that length of stay and oxygen use could be used as quality measures in this population. (20,22,33) The observed strong correlation between human milk at discharge and hypothermia in Baby-MONITOR and MLP-QM suggests that these components may be appropriate for an MLP-specific quality measure. The added measures, including oxygen use at 28 days, extreme length of stay, and greater than median z-score, as well as human milk at discharge and hypothermia may provide a starting point for formal development of an MLP specific quality measure through a Delphi process with established experts or through a statistical approach. (34) Formal measure development for MLP infants is essential to create a definition of quality of care for this population.

Poor correlation in hospital performance in Baby-MONITOR in extremely and very preterm infants with the adapted MLP-QM in MLP infants cautions against applying definitions of quality validated in one, specific population to other populations and associated assumptions about the quality of care. Studies in adults have demonstrated poor correlation in hospital mortality rates for individual diagnoses at a single hospital, implying that providing high quality care for one population or disease may not translate into high quality care in other populations or diseases. (35) The National Quality Forum (NQF) has not endorsed any disease-specific quality measures for adult intensive care, as they do not adequately assess the overall quality of care within a specific ICU. (36–38) Similarly, careful consideration must be taken when developing quality measures to evaluate NICU care given population heterogeneity with respect to the wide range of GAs and their associated risk profiles, resource utilization, and outcome likelihood. While unexpected complications in term newborns is an NQF endorsed measure, this measure does not capture the quality of care for infants admitted to the NICU, such as MLP infants. (39) To address this complex problem of assessing quality in a heterogenous population, Silber et al. proposed using a specific template of patients standardized for diagnoses and patient characteristics and diagnoses to compare quality across hospitals. (40,41) While the heterogeneity of the NICU population may not be merit this approach to quality assessment, composite quality measures for overall NICU care must accurately weight process and outcome measures for infants of all GAs. Validated quality measures are certainly needed for the MLP population given their prevalent NICU admissions. (42,43)

Poor correlation between adapted MLP quality score and components in MLP infants and the Baby-MONITOR score in extreme and very preterm population suggests that at a single hospital, quality of care for extremely and very preterm infants may not correlate with quality of care for MLP infants. One potential reason for this finding is that processes implemented to improve care for VLBW infants may not apply to other subpopulations, such as MLP infants. (34) For example, discordance in infection rates in extremely and very preterm and MLP infants may be secondary to GA specific differences in vascular access management. (44) As higher rates of intubation and surfactant administration are seen in extremely and very preterm infants compared to MLP infants, discordance in pneumothorax could be secondary to GA specific processes of respiratory management. Centers with low levels of pneumothorax in extremely and very preterm infants but higher levels in MLP infants who have lower risk of pneumothorax may be providing lower quality care. (45) Poor correlation between survival in extremely and very preterm infants and MLP infants has not previously been reported. Both process-level factors (e.g., unit use of standardized care guidelines) and hospital-related factors (e.g. unit size, unit level of care, staffing models) may contribute to this observation. Further research on how process, hospital, and other factors contribute to these survival differences is necessary.

This study has limitations. First, while we efficiently adapted Baby-MONITOR for MLP infants by selecting evidence-based measures previously evaluated by experts, the current evaluative standard for formal measure development is measure selection through expert consensus or statistical assessment of validity. (6,17,19–22) Second, other potential quality measures reflecting relevant aspects of MLP care, such as time to initiate enteral feeds or time to reach full feeds, are not included in the adapted MLP-QM because such measures were not available in the data. Third, we examined infants from 30–36 weeks GA together to capture the quality of care for infants not measured by Baby-MONITOR or term infant measures; however, in doing so, we may not capture differences in quality between moderate and late preterm infants. Future quality measure development should consider potential differences within smaller GA subgroups. Fourth, similar to established NICU quality measures, the MLP-QM does not incorporate components of patient-centered care, such as family involvement at bedside or education. These measures are likely relevant to the MLP population, whose NICU care may be more amendable to family involvement than the extremely preterm population, and merit consideration and further study. Finally, we used a convenience-based sample of hospitals participating in the VON all-admission database. Formal measure development for MLP infants could address these limitations, creating a valid MLP-specific measure.

Despite these limitations, this study uses a large, contemporary dataset of NICU admissions across many hospital types to demonstrate the need for a quality measure in the prevalent MLP population. The robust dataset facilitated study of new candidate MLP quality measures and their associated variability as well as examination of correlation of the MLP-QM with previously described VLBW quality measures. This work is the first endeavor to develop a quality measure for this important population, comprising the majority of NICU patients.

Conclusion

We developed an adapted quality measure to assess quality of care in MLP infants that demonstrated poor correlation between overall and component quality scores in extremely and very preterm infants cared for at the same hospital. Additional research is needed to develop a validated quality measure for the MLP population. Weak correlation between summary and component scores suggests that different processes may mediate positive outcomes in MLP infants compared to extremely and very preterm infants. Further study is needed to elucidate how process and hospital-level factors influence MLP quality of care.