Development and validation of a nomogram for predicting late-onset sepsis in preterm infants on the basis of thyroid function and other risk factors: Mixed retrospective and prospective cohort study.

Preterm birth and infection are common causes of neonatal death. In this study, we aimed to develop a nomogram for assessing the individual prior probability of late-onset sepsis on the basis of risk factors in preterm infants. This study is a mixed retrospective and prospective cohort study conducted in three centers. Data from January 2014 to December 2017 was used for the development cohort, and data from January 2018 to December 2018 was used for the validation cohort. In the development cohort, we identified the predicting variables of late-onset sepsis in preterm infants, from which a nomogram was obtained. Then we built nomograms, for with and without thyroid function, to predict late-onset sepsis. The nomogram was validated in the validation cohort concerning discrimination and calibration. A total of 1256 and 452 preterm infants were included in the development and validation cohort, respectively. We found thyroid hypofunction in preterm infants could increase the incidence of late-onset infection. The prediction model incorporated thyroid function, birth weight, use of endotracheal intubation, and duration of umbilical venous catheters was validated and developed as a nomogram for predicting late-onset sepsis in preterm infants. Nomogram in this study may contribute to clinical assessment and treatment decisions.

Introduction

Worldwide, 7.6 million children under 5 years old die each year. Neonatal mortality (defined as death within the first 4 weeks of life) accounts for 41% of deaths of children under 5 years of age. Preterm birth and infection are the leading causes of neonatal death, accounting for 29% each [1]. Therefore, an effective assessment for the risk of infection in premature infants will enable clinicians to make early clinical decisions and reduce mortality in premature infants. Pediatric doctors in Boston Children’s Hospital of Harvard University developed a quantitative model to estimate the probability of neonatal early-onset bacterial infection on the basis of maternal intrapartum risk factors [2]. However, individualized assessment risk of late-onset bacterial sepsis in preterm infants is lacking.

Late-onset bacterial sepsis in preterm infants has multiple causes. The National Institute of Child Health and Human Development has indicated that the incidence of infection in premature infants increases with decreasing birth weight, reporting rates of 43% for infants weighing 401–750 g, 28% for those weighing 751–1000 g, 15% for those weighing 1001–1250 g, and 7% for infants weighing 1251–1500 g [3]. There are two possible explanations for the high susceptibility of preterm infants to infection. Firstly, premature infants have documented immune dysfunction [4]. Secondly, premature infants often require prolonged intravenous access, endotracheal intubation, or other invasive procedures [5].

During the first 3 months of life, innate immune cells provide defense against pathogens [4]. Decreased function of innate immune cells will increase the susceptibility of preterm infants to invasive infection. Studies show that thyroid hormones (THs) play important roles in regulating the innate immune system [6], [7], [8]. Infants generally have an abnormally regulated hypothalamic–pituitary–thyroid (HPT) axis when they are critically ill, with low 3,5,3′-triiodo-L-thyronine (T3) and thyroxine (T4) levels being high risk factors of poor prognosis for neonatal sepsis [9]. Another study analyzing 148 preterm infants with a gestational age (GA) lower than 33 weeks showed that the serum T3 concentration in preterm infants negatively correlates with interleukin-6 (IL-6) and C-reactive protein (CRP) levels [10]. It is known that sicker babies have lower (F)T4 and (F)T3 levels, and thus the relationship between thyroid hypofunction and late-onset infection in premature babies is not surprising. We argue that low TH levels are the result of disease, metabolic state, and medications, and are a sign of a weaker baby that may be susceptible to late-onset bacterial sepsis.

However, birth weight, invasive procedures, and thyroid hypofunction may be only one of the risk factors for late-onset sepsis in preterm infants. Clinicians need a comprehensive mathematical tool that can combine the risk factors to predict susceptibility for late-onset bacterial sepsis. Nomograms are graphical depictions of predictive statistical models for individual patients [11]. Use of nomograms has the advantage of combining risk factors for late-onset bacterial sepsis, to predict probability of bacterial late-onset sepsis in individual preterm infants. Hence, the present study is aimed at developing a practical clinical tool by combining birth weight, thyroid function and other risk factors into a nomogram. We also tested whether this model provides a more accurate prediction of bacterial late-onset sepsis in preterm infants when compared with nomograms without thyroid function.

Patients and methods

Setting and participants

This study is a mixed retrospective and prospective cohort study. It was conducted in three neonatal critical care centers in Guangdong province of China, including the Second Affiliated Hospital of Shantou University Medical College, Women and Children Hospital of Guangdong Province, and Affiliated Xiaolan Hospital of Southern Medical University. The study protocol was approved by each research institute’s committee of human research in the participating centers (NO.2016027), and abided by the standards of the Declaration of Helsinki.

We collected preterm infant data in the participating centers, from January 2014 to December 2017, to serve as the development cohort for retrospective analysis. The data were anonymized in the retrospective study. Inclusion criteria of preterm infants were as follows: (1) gestational age <37 weeks, and (2) born in one of the above three hospitals and admitted to the neonatal department within 24 h after birth. Preterm infants were excluded from this study if they met the following criteria: (1) mothers had thyroid, liver, kidney, lung or heart disease before pregnancy, (2) infants had a congenital malformation, and (3) hospital stay of the preterm infant was less than 7 days. Similarly, all preterm infants in the participating centers who fulfilled the inclusion criteria were consecutively and prospectively collected from January 2018 to December 2018 for the validation cohort. All cases were enrolled after obtaining informed consent in the prospective study.

Data collection

Variables included age of mother, maternal antenatal glucocorticoid (GC) treatment, premature rupture of membranes (PROM), antibiotic treatment before delivery, pregnancy diabetes, pregnancy hypertension, delivery season, method of delivery, multiple pregnancy, gestational age (GA), birth weight, gender, asphyxia, use of dopamine, use of albumin, use of antibiotics, start day of enteral nutrition (EN) initiation, endotracheal intubation (EI), mechanical ventilation (MV), peripheral insertion of a central catheter (PICC), and umbilical venous catheterization (UVC). The data for dopamine, albumin, and antibiotic treatment, all obtained before detecting thyroid function, were used for analysis. The data for EI, MV, PICC, and UVC, all obtained before diagnosis of late-onset sepsis, were used for analysis.

Detection and assessment of thyroid function

To determine thyroid function, blood samples of preterm infants were taken in the morning between 4 and 7 days after birth. After blood samples were collected, thyroid function was measured with a chemiluminescence kit (Beckman Coulter, Prague, Czech Republic), quantifying T3, T4, FT3, FT4, and TSH levels. A TSH >10 mIU/L was defined as high TSH [12]. In order to compensate for the change in TH levels due to gestational age, T3, T4, FT3, and FT4 were corrected by using either the 10th percentile (P10) or −1 SD as the cut-off value [13], [14]. Thyroid hypofunction included transient hypothyroxinemia of prematurity (THOP: low T4 with normal TSH), congenital hypothyroidism (CH: low T4 with elevated TSH), low serum T3, and high TSH.

Diagnosis of late-onset bacterial sepsis

The outcome in this cohort study was late-onset bacterial sepsis. Diagnosis of late-onset bacterial sepsis was made according to symptoms and laboratory evidence [15]. Symptoms of bacterial sepsis in preterm infants included temperature instability, hypotension, poor perfusion with pallor, tachycardia, bradycardia, apnea, cyanosis, irritability, lethargy, seizures, abdominal distention, jaundice, petechiae [16]. Blood count, C-reactive protein (CRP), procalcitonin (PCT), and the cultures from blood and other sterile sites were obtained to identify bacterial infection. Culture-based diagnostics for late-onset bacterial sepsis was that the infant manifested signs and symptoms of infection after 7 days of age and in line with any of the following: (1) a blood or cerebrospinal fluid (CSF) culture positive for a pathogenic bacterial species, and (2) When the blood culture result was conditional pathogenic bacteria (e.g., coagulase-negative staphylococci), the treating physician considered the infant infected by combination with other laboratory parameters of infection, such as blood count, CRP, and PCT [15], [16].

Statistical analysis

For continuous variables, the Shapiro-Wilk test was used to determine the normal distribution of the continuous variables, and the Wilcoxon-Mann-Whitney U test was conducted for skewed distributions (presented as the median and the Min-Max range). Descriptive statistics for categorical variables were reported as frequency (percentage) and were compared using the Pearson chi-square test or Fisher's exact test, as appropriate. Collinearity among all covariates was assessed using the Spearman correlation and Belsley collinearity tests (13). Preterm infant data in the participating centers from January 2014 to December 2017 was used as the development cohort, and data from January 2018 to December 2018 was used as the validation cohort. Statistical analyses were performed using SPSS 24.0 (SPSS, Chicago, IL), SAS software (SAS v9.4; SAS Institute, NC, USA) and R v3.3.3 (R Foundation for Statistical Computing, Vienna, Austria). P-values of less than 0.05 were considered to be statistically significant.

To develop a predictive nomogram for late-onset bacterial sepsis in individual preterm infants, a logistic regression analysis was initially performed by a backward stepwise method to identify the reduced model in the development cohort. Estimated relative risk (RR) and 95% confidence intervals (CI) were obtained. Selection of the prediction model was performed using the Akaike Information Criterion (AIC), which can find the model that best interprets the data but contains the fewest parameters [17].

Before making the nomogram, it was necessary to verify the predictive power of the model. Common validation methods include internal validation and external validation, with external validation being superior to internal validation. External verification uses data of another group of research objects to verify the prediction accuracy of the model. Data sources of external validation include data of the same centers in the same period, data of the same center in different periods, and data of different centers [11]. Because our data is from three centers, and the number of cases including in this study of the three centers was inconsistent, we can exclude other possible factors related to specific treatment protocols in the research centers. Therefore, we used the external validation method, and the data source was from these three centers in different periods. There are two indices for assessing the nomogram. Firstly, we drew a calibration curve that compared the predicted probability of the alignment chart with the actual event occurrence rate. The closer the curve is to the reference line, the better the calibration degree will be. Secondly, the area under the curve (AUROC) was derived from conventional receiver operating characteristic (ROC) curves. The larger the value of AUROC is the better the predictive ability of the nomogram. Performance of the nomogram was evaluated by the concordance index (C-index). The C-index as a measure of classification accuracy was further compared using the nonparametric approach of DeLong [18]. The performance of the model was measured by accuracy, sensitivity, specificity, positive and negative predictive values (PPV and NPV), and the percentage of correctly classified cases [PC, PC = (true positive + true negative)/total samples].

Results

Occurrence of late-onset bacterial sepsis in the study population

In total, 1,708 preterm infants were eligible for analysis. There were 130 of 1708 (7.61%) preterm infants with late-onset bacterial sepsis. The mean age of onset for the first episode of late-onset bacterial sepsis was 21.43 ± 14.62 days old. Among the infection cases, 81/130 (62.3%) occurred from 1 to 3 weeks after birth. The incidence of late-onset bacterial sepsis in preterm infants was 24/50 (48%) for a GA of below than 28 weeks, 58/251 (23.1%) for a GA of 28 to 32 weeks, and 48/1407 (3.41%) for a GA of 32 to 37 weeks. Of these infections, 53.66% were attributed to Gram-negative organisms, and 42.71% were attributed to Gram-positive organisms. The top five most frequent organisms were Escherichia coli, Klebsiella pneumoniae, Staphylococcus haemolyticus, Staphylococcus aureus, and Staphylococcus epidermidis.

Distribution of thyroid hypofunction in the study population

Fifty of the 1708 (2.93%) infants were below a GA of 28 weeks, 251 of 1708 (14.7%) infants had a GA of 28 to 32 weeks, and 1407 of 1708 (82.37%) infants had a GA of 32 to 37 weeks. The incidence of thyroid hypofunction in preterm infants was 26/50 (52%) for a GA of less than 28 weeks, 62/251 (24.7%) for a GA of 28 to 32 weeks, and 223/1407 (15.85%) for a GA of 32 to 37 weeks. Overall, 311 of 1,708 preterm infants (18.21%) had thyroid hypofunction, with thyroid hypofunction cases consisting of CH (8/311), THOP (151/311), low T3 (139/311), and high TSH (13/311).

Risk factors of thyroid hypofunction in the study population

In order to analyze the effect of clinical variables on the thyroid function of preterm infants, we examined the association between patient characteristics and clinical variables in preterm infants with normal thyroid function and thyroid hypofunction (see Table 1). The preterm infants with low GA, low birth weight, asphyxia, delayed start of EN, use of dopamine, use of albumin, use of antibiotics, and whose mothers did not receive antenatal GC treatment, had a high incidence of thyroid hypofunction. According to the logistic regression analysis, the risk factors of thyroid hypofunction in preterm infants were low birth weight (RR = 0.573, 95%CI = 0.358–0.917), treatment with dopamine (RR = 1.652, 95%CI = 1.073–2.542), albumin (RR = 2.156, 95%CI = 1.441–3.227), or antibiotics (RR = 1.766, 95%CI = 1.205–2.59), and lack of maternal antenatal treatment with GC (RR = 0.453, 95%CI = 0.285–0.932). The incidence of thyroid hypofunction was 23% in the preterm infants whose mothers were not antenatal treated with GC, which was decreased to 15.19% in those infants whose mothers received antenatal GC treatment. There were 17.01% preterm infants with no asphyxia showing thyroid hypofunction, but the incidence of thyroid hypofunction rose to 27.41% in preterm infants with asphyxia. The incidence of thyroid hypofunction was 15.66%, 14.82%, and 11.02% in preterm infants who were not treated with dopamine, albumin, and antibiotics, respectively, compared to 34.33%, 36.74%, and 21.92% in those infants correspondingly treated with the above three medicines (see Table 1).

Table 1

Patient demographics and clinical characteristics between thyroid hypofunction and normal thyroid function in preterm infants.

Characteristic	All	Normalthyroid function	Thyroid hypofunction	Statistic	P
N	1708	1397	311
Mother
Age (y)	28 (16–51)	29 (16–44)	28 (17–51)	−1.251	0.211
Hypertension				0.57	0.445
Yes	209	167 (79.9%)	42 (20.1%)
No	1499	1230 (82.05%)	269 (17.95%)
Diabetes				3.356	0.074
Yes	128	97 (75.78%)	31 (24.22%)
No	1580	1300 (82.28%)	280 (17.72%)
Multiple pregnancy				3.482	0.067
Yes	205	158 (77.07%)	47 (22.93%)
No	1503	1239 (82.44%)	264 (17.56%)
Maternal antenatal GC				16.592	＜0.001
Yes	1047	888 (84.81%)	159 (15.19%)
No	661	509 (77%)	152 (23%)
Antenatal antibiotic				3.92	0.05
Yes	616	519 (84.25%)	97 (15.75%)
No	1092	878 (80.4%)	214 (19.6%)
Season of delivery				0.277	0.964
Spring	378	308 (81.48%)	70 (18.52%)
Summer	484	395 (81.61%)	89 (18.39%)
Autumn	498	406 (81.53%)	92 (18.47%)
Winter	348	288 (82.76%)	60 (17.24%)
Method of delivery				0.308	0.612
Vaginal delivery	981	798 (81.35%)	183 (18.65%)
Cesarean delivery	727	599 (82.39%)	128 (17.61%)
Preterm infants
Gender of infants				1.147	0.312
Female	744	617 (82.93%)	127 (17.07%)
Male	964	780 (80.91%)	184 (19.09%)
GA (w)				50.699	＜0.001
24–28	50	24 (48%)	26 (52%)
28–32	251	189 (75.3%)	62 (24.7%)
32–37	1407	1184 (84.15%)	223 (15.85%)
Birth weight (kg)	2 (0.69–3.84)	2.05 (0.9–3.84)	1.9 (0.69–3.2)	−3.888	＜0.001
Asphyxia				12.663	0.001
Yes	197	143 (72.59%)	54 (27.41%)
No	1511	1254 (82.99%)	257 (17.01%)
Start day of EN (d)	1.02 ± 1.54	0.95 ± 1.4	1.36 ± 2.07	−2.254	0.024
Use of dopamine				47.113	＜0.001
Yes	233	153 (65.67%)	80 (34.33%)
No	1475	1244 (84.34%)	231 (15.66%)
Use of albumin				72.025	＜0.001
Yes	264	167 (63.26%)	97 (36.74%)
No	1444	1230 (85.18%)	214 (14.82%)
Use of antibiotics				30.589	＜0.001
Yes	1127	880 (78.08%)	247 (21.92%)
No	581	517 (88.98%)	64 (11.02%)

Abbreviation: EN: enteral nutrition; GA: gestational age; GC: glucocorticoid.

Results are shown as the median (min,max)] or n (%).

*Mann-Whitney test or Kruskal-Wallis test or CMH χ2 test or Fisher exact test when appropriate.

Thyroid hypofunction and late-onset bacterial sepsis occur in various disease condition

In this study, the top five diseases in preterm infants were intrauterine infection, patent ductus arteriosus (PDA), neonatal hyperbilirubinemia, hyaline membrane disease (HMD), hypoxic ischemic encephalopathy (HIE). Thyroid hypofunction and late-onset bacterial sepsis occured in these five diseases condition is shown in Table 2. The proportion of thyroid hypofunction and late-onset sepsis in preterm infants without any disease was significantly lower than that in preterm infants with combined diseases, suggesting that both thyroid hypofunction and late-onset sepsis tend to occur in preterm infants with combined diseases.

Table 2

Thyroid hypofunction and late-onset bacterial sepsis occur in various disease condition of preterm infants.

Diseases	N	Normalthyroid function	Thyroid hypofunction	No LOS	LOS
N	1708	1397 (81.78%)	311 (18.21%)	1578 (92.39%)	130 (7.61%)
Intrauterine infection	636	537 (84.44%)	99 (15.56%)	604 (94.97%)	32 (5.03%)
Patent ductus arteriosus	254	158 (62.2%)	96 (37.8%)	237 (93.31%)	17 (6.69%)
Hyperbilirubinemia	233	147 (63.09%)	86 (36.91%)	223 (95.71%)	10 (4.29%)
Hyaline membrane disease	217	114 (52.53%)	103 (47.47%)	190 (87.56%)	27 (12.44%)
HIE	164	119 (72.56%)	45 (27.44%)	152 (92.68%)	12 (7.32%)
Apnea	146	86 (58.9%)	60 (41.1%)	125 (85.62%)	21 (14.38%)
Pneumonia	61	39 (63.93%)	22 (36.07%)	59 (96.72%)	2 (3.28%)
Intracranial hemorrhage	33	19 (57.58%)	14 (42.42%)	26 (78.29%)	7 (21.21%)
Congenital heart disease	29	17 (58.62%)	12 (41.38%)	29 (100%)	0 (0%)
No complications	355	321 (90.42%)	34 (9.58%)	355 (100%)	0 (0%)

HIE: hypoxic ischemic encephalopathy.

Cohort description

There were 1256 preterm infants in the development cohort and 452 preterm infants in the validation cohort. Association of patient characteristics with clinical variables in preterm infants with and without late-onset bacterial sepsis is shown in Table 3.

Table 3

Characteristics of patients in the development and validation cohorts.

Variables	Development Cohorts		P	Validation Cohorts		P
	No LOS(N = 1160)	LOS(N = 96)		No LOS(N = 418)	LOS(N = 34)
Mother
Age (y)	29 (21–51)	28 (16–48)	0.049	28 (18–53)	29 (19–52)	0.156
GC			0.977			0.428
Yes	709 (92.08%)	61 (7.92%)		254 (91.7%)	23 (8.3%)
No	451 (92.8%)	35 (7.2%)		164 (93.71%)	11 (6.29%)
PROM			0.364			0.482
Yes	416 (93.27%)	30 (6.73%)		147 (91.3%)	14 (8.7%)
No	744 (91.85%)	66 (8.15%)		271 (93.13%)	20 (6.87%)
Delivery			0.002			0.471
Vaginal	681 (94.45%)	40 (5.87%)		238 (91.15%)	22 (8.85%)
Cesarean	479 (89.53%)	56 (10.47%)		180 (94.27%)	12 (5.73%)
Infant
Gender			0.369			0.311
Female	501 (91.59%)	46 (8.41%)		185 (93.91%)	12 (6.09%)
Male	659 (92.95%)	50 (7.05%)		233 (91.37%)	22 (8.63%)
GA (w)			＜0.001			＜0.001
24–28	4 (33.33%)	8 (66.67%)		22 (57.89%)	16 (42.11%)
28–32	161 (77.03%)	48 (22.97%)		32 (76.19%)	10 (23.81%)
32–37	995 (96.14%)	40 (3.86%)		364 (97.85%)	8 (2.15%)
BW (kg)	2.05(0.95–3.84)	1.45(0.69–2.78)	＜0.001	2.0(0.73–3.6)	1.42(0.6–3.1)	＜0.001
Asphyxia			0.022			0.094
Yes	127 (87.59%)	18 (12.41%)		45 (86.54%)	7 (13.46%)
No	1033 (92.98%)	78 (7.02%)		373 (93.25%)	27 (6.75%)
EI			＜0.001			＜0.001
Yes	221 (80.36%)	54 (19.64%)		78 (78.79%)	21 (21.21%)
No	939 (95.72%)	42 (4.28%)		340 (96.32%)	13 (3.68%)
UVC (d)	0 (0–9)	0 (0–10)	＜0.001	0 (0–8)	0 (0–10)	＜0.001
PICC (d)	0 (0–61)	0 (0–64)	＜0.001	0 (0–53)	0 (0–60)	＜0.001
MV (d)	0 (0–33)	0 (0–23)	＜0.001	0 (0–24)	0 (0–31)	＜0.001
TF			＜0.001			＜0.001
Yes	172 (81.52%)	39 (18.48%)		80 (80%)	20 (20%)
No	988 (94.55%)	57 (5.45%)		338 (96.02%)	14 (3.98%)

Abbreviation: EI: endotracheal intubation; GA: gestational age; GC: glucocorticoid; LOS: late-onset sepsis; MV: mechanical ventilation; PICC: peripherally inserted central catheter; PROM: premature rupture of membranes; UVC: umbilical venous catheter; TF: thyroid hypofunction.

Results are shown as the median (min, max)] or n (%).

*Mann-Whitney test or Kruskal-Wallis test or CMH χ2 test or Fisher exact test when appropriate

Risk factors for late-onset bacterial sepsis of preterm infants in the development cohort

According to the results of logistic regression analysis in the development cohort, birth weight, EI, UVC, and thyroid hypofunction were identified as predictors for late-onset bacterial sepsis in preterm infants. Preterm infants of low birth weight (RR = 0.136, 95%CI = 0.051–0.361), required EI (RR = 5.195, 95%CI = 1.797–15.016) or UVC (RR = 1.346, 95%CI = 1.194–1.519), and had thyroid hypofunction (RR = 4.084, 95%CI = 2.036–6.262) had a high incidence of late-onset bacterial sepsis (Table 4). The median birth weight in preterm infants with late-onset bacterial sepsis was 1.45 kg, while the median birth weight in those without infection was 2.05 kg. Among preterm infants who did not require EI, 4.28% suffered late-onset bacterial sepsis, but the incidence of late-onset bacterial sepsis was up to 19.64% in infants requiring EI. The incidence of late-onset bacterial sepsis in preterm infants with normal thyroid function was 5.45%, but was up to 18.48% in those with thyroid hypofunction.

Table 4

Logistic regression of the risk factors for bacterial LOS in the development cohort.

Characteristic	RR	95%CI	P
Age of mother	1.024	0.973–1.077	0.369
Delivery	1.029	0.858–1.093	0.136
GA	0.867	0.729–1.032	0.109
Birth weight	0.136	0.051–0.361	＜0.001
Asphyxia	1.054	0.894–1.047	0.257
UVC	1.346	1.194–1.519	＜0.001
PICC	0.968	0.943–1.004	0.071
MV	1.038	0.962–1.120	0.132
EI	5.195	1.797–15.016	0.002
TF	4.084	2.036–6.262	＜0.001

Abbreviation: EI: endotracheal intubation; GA: gestational age; LOS: late-onset sepsis; MV: mechanical ventilation; PICC: peripherally inserted central catheter; UVC: umbilical venous catheters; TF: thyroid hypofunction.

Development, validation, and comparison of nomograms with and without thyroid function for predicting individual prior probability of late-onset bacterial sepsis in preterm infants

We used the development cohort to develop nomograms for late-onset bacterial sepsis in preterm infants, and we used the validation cohort to test the nomograms. Variables including age of mother, delivery method, GA, asphyxia, PICC, and MV were eliminated from the logistic regression due to their p-values exceeding 0.05. Birth weight, EI, UVC, and thyroid function were identified as the independent predictors in the logistic regression analysis. In order to further verify the predictive effect of thyroid function on late-onset bacterial sepsis in preterm infants, we constructed Nomogram A and Nomogram B to predict late-onset bacterial infection with and without thyroid function, respectively. Nomogram A used birth weight, EI, UVC, and thyroid function as variables, and Nomogram B used birth weight, EI, and UVC as variables. We drew a calibration curve to assess the accuracy of the nomograms. The calibration plot for the probability of late-onset bacterial sepsis in preterm infants showed optimal agreement between the prediction by Nomogram A and actual observation [development: 0.970 (0.960–0.982); validation: 0.963 (0.950–0.980)] (Fig. 1A). The calibration plot also showed agreement between the prediction by Nomogram B and actual observation [development: 0.816 (0.768–0.863); validation: 0.808 (0.760–0.856)] (Fig. 1B). The predictive power, between Nomogram A and Nomogram B, for the incidence of late-onset bacterial sepsis in preterm infants was compared. The AUROC value of Nomogram A [development: C-index = 0.855 (0.802–0.907); validation: C-index = 0.834 (0.775–0.894)] is larger than that for Nomogram B [development: C-index = 0.793 (0.693–0.893); validation: C-index = 0.765 (0.660–0.870)] [development: P = 0.028; validation: P = 0.0264] (Fig. 2A and 2B). These results indicate that the nomogram incorporating thyroid function displays better predictive power in predicting the probability of late-onset bacterial sepsis compared with the nomogram without thyroid function (Table 5).

Fig. 1

Calibration plots for predicted models of individual prior probability of bacterial LOS with (A) and without (B) thyroid function. The 45 °C dashed line represents ideal predictions (Ideal line), the plot illustrates the accuracy of the best-fit model (Apparent) and the bootstrap model (Bias-corrected) for predicting individual prior probability of bacterial LOS in preterm infants. LOS: late-onset sepsis.

Fig. 2

Receiver operating characteristic (ROC) curve analyses of predicted models for individual prior probability of bacterial LOS (A) with and (B) without thyroid function. LOS: late-onset sepsis.

Table 5

C-index of nomogram models with and without thyroid function for predicting bacterial LOS in the development cohort and validation cohort.

Factor	Development cohorts (N = 1256)		Validation cohorts (N = 452)
	C-index (95%CI)	P	C-index (95%CI)	P
Nomogram A (with TF)	0.855 (0.802–0.907)		0.834 (0.775–0.894)
Nomogram B (without TF)	0.793 (0.693–0.893)		0.765 (0.660–0.870)
Nomogram A vs Nomogram B		0.027		0.028

LOS: late-onset sepsis; TF: thyroid function; p-values were calculated using the Delong method [18].

Lastly, the prediction model (Nomogram A) that incorporated thyroid function, birth weight, EI, and UVC was validated and presented as the nomogram for predicting individual prior probability of late-onset bacterial sepsis in preterm infants (Fig. 3). The points of each predictor in the nomogram were first determined by drawing a vertical line from the factor to the point axis. Then, the sum of all the points from all predictors was used to generate the total points. By drawing a vertical line from the total point axis to the risk of late-onset bacterial sepsis axis, the estimated probability of late-onset bacterial sepsis could be obtained. The area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, NPV, PPV, and PC with 95% CI (%) for the nomogram of late-onset bacterial sepsis are shown in Table 6.

Fig. 3

Nomogram for predicting prior probability of bacterial LOS in a preterm infant. To obtain the nomogram-predicted probability, locate patient values on each axis. Draw a vertical line to the point axis to determine how many points are attributed for each variable value. Sum the points for all variables. Locate the sum on the total point line to assess the individual probability of bacterial late onset sepsis in the preterm infants. EI: endotracheal intubation; LOS: late-onset sepsis; UVC: umbilical venous catheters.

Table 6

The performance of the nomogram for prediction of bacterial LOS in preterm infants.

Cohort	AUC(95%CI)	Accuracy(95%CI)	Sensitivity(95%CI)	Specificity(95%CI)	NPV(95%CI, %)	PPV(95%CI, %)	PC(95%CI, %)
Development	0.855(0.802, 0.907)	0.970(0.960, 0.982)	0.500(0.377, 0.622)	0.918(0.880, 0.957)	0.850(0.802, 0.898)	0.667(0.553, 0.800)	0.818(0.784, 0.885)
Validation	0.834(0.775, 0.894)	0.963 (0.950–0.980)	0.453(0.331, 0.575)	0.923(0.886, 0.961)	0.839(0.789, 0.887)	0.660(0.519, 0.800)	0.810(0.747, 0.869)

LOS: late-onset sepsis; AUC: area under the curve; NPV: negative predictive value; PPV: positive predictive value; PC: the percentage of correctly classified cases.

Discussion

In this study, we found no maternal antenatal GC, low birth weight, and treatment of preterm infants with use of dopamine, albumin, and antibiotics results in high incidence of thyroid hypofunction, and subsequently an increase in the incidence of late-onset bacterial sepsis in preterm infants. This study demonstrates that low birth weight, use of EI and UVC, and thyroid hypofunction are risk factors of late-onset sepsis in preterm infants. Using these findings, we established a prognostic nomogram for predicting individual prior probability of late-onset bacterial sepsis in preterm infants.

Risk factors of thyroid hypofunction in this study show that thyroid hypofunction tends to occur in critically ill preterm infants

Environmental stimulation causes drastic changes in hypothalamic–pituitary–thyroid (HPT) function in preterm infants after birth. TH levels become relatively stable after 3 days of birth, but are prone to influences from both the external environment and disease [19]. One study analyzing thyroid function and 20 perinatal factors, from 932 preterm infants with birth weights lower than 1500 g, at 5 days after birth showed that for a GA of less than 27 weeks, administration of dopamine, and MV are risk factors of thyroid hypofunction [20]. A prior study showed that maternal antenatal GCs could increase T4 levels in the 1st week after birth in 23- to 28-week GA infants [21]. One study from China, which detected THs at 12–16 days after birth, showed that being male and having been administered albumin and dopamine were risk factors for THOP, and being male and having a GA lower than 28 weeks were risk factors for low serum T3 [14]. In this study, we collected relevant factors to perform logistic regression analyses, which suggested that low birth weight, lack of maternal antenatal treatment with GC, and administration of dopamine, albumin, and antibiotics could reduce serum TH levels in preterm infants. The proportion of thyroid hypofunction in preterm infants without any disease was lower than that with combined diseases. These results are consistent with those in other studies mentioned above, indicating our subjects are representative of the population, and we further found that thyroid hypofunction tends to occur in critically ill preterm infants.

Occurrence of late-onset bacterial sepsis in the study population is feasible

Late-onset bacterial sepsis is one of the most common and fatal complications in preterm infants. With lower birth weight, the incidence of bacterial LOS is higher [3]. Among the preterm infants who had samples collected for this study, 7.61% patients developed late-onset infection, and the mean onset time was 21.43 ± 14.62 days. In our study, the incidence of late-onset bacterial sepsis for preterm infants is consistent with data from other NICUs in China and other countries [3], [16], [22]. A recent report from other countries showed that approximately 6.4% of infants develop late-onset sepsis in the neonatal intensive care unit, and the median age of development is 3 weeks after birth [22].

Infants generally have an abnormally regulated HPT axis when they are critically ill, with low T3 and T4 levels being high risk factors of poor prognosis of neonatal sepsis [9]. A similar study found that low TH levels are closely associated with bacterial sepsis development and poor survival outcome in infants and young children [23]. Another study analyzing 148 preterm infants with a GA less than 33 weeks showed that the serum T3 concentration in preterm infants negatively correlates with interleukin-6 (IL-6) and CRP levels, and the incidence of bacterial sepsis is significantly increased in preterm infants who have lower serum T3 level [10]. Our results are consistent with the above studies. In addition, we found that low birth weight, thyroid hypofunction, use of EI, and longer duration of UVC treatment before infection are risk factors for late-onset infection.

In the nomogram development cohort, 1% had a GA < 28 weeks and the validation cohort had 8%. In the development cohort, 18% had a GA 28–32 and in the validation cohort this was 9%. Although the two cohorts had inequalities in GA distribution, the incidences of late-onset bacterial sepsis rates were about 7–8% in both cohorts. The reason for these differences could be due to data for the development cohort being from preterm infants in the participating centers from January 2014 to December 2017, and data of validation cohort being from January 2018 to December. From 2018, the Second Affiliated Hospital of Shantou University Medical College and the Women and Children Hospital of Guangdong Province became designated treatment center for critically ill newborns in Guangdong Province, so the number of low GA infants in these two participating centers increased. We improve measures for the prevention and control of infection in the NICU, so the late-onset bacterial sepsis rates is not increase with the number of low GA infants.

Innovation and practicality of the nomograms with thyroid function for predicting individual prior probability of late-onset bacterial sepsis in preterm infants

We developed a nomogram for predicting individual prior probability of late-onset bacterial sepsis in preterm infants that provides an approach to the problem of how to rule out late-onset sepsis in preterm newborns. The nomogram combined the birth weight of infants, thyroid function, and subsequent clinically accessible information, including EI and UVC, can be used to guide evaluation and treatment decisions. Use of our predictive model will require neonatal clinicians to be explicit about specifying a level of risk at which one should evaluate newborns for late-onset bacterial sepsis. It permits clinicians to incorporate key clinical factors into risk estimates. For example, the model incorporates thyroid function and accounts for modification of a newborn’s risk as a result of being critically ill.

By taking full advantage of available information, our multivariate model can permit clinicians to make decisions that are more closely tailored to individual risk of late-onset bacterial sepsis. For example, if there are two preterm infants weighing 1200 g and 800 g respectively. Should we say that the risk of infection for a preterm infant weighing 800 g is greater than is the infant weighing 1200 g? It is difficult to determine the risk of infection in preterm infants by birth weight alone. In this study, we developed a nomogram to predict risk of late-onset sepsis in preterm infant base on ROC curves with large AUC and calibration curves. Therefore, we can accurately calculate the risk of individual infection in preterm infants based on the birth weight, thyroid function, and duration time of invasive operation. Moreover, the nomogram for predicting late-onset bacterial sepsis in preterm infants has specificity and a high NPV, which can accurately exclude preterm infants without infection. The risk of infection in the same child should also be assessed according to his clinical situation. For example, for a preterm infant weighing 1000 g, with normal thyroid function, no use of EI, and a duration UVC time of 4 days, the posterior rate of late-onset sepsis is about 18% according to the nomogram. If this scenario is modified to include thyroid hypofunction, use of EI, and a duration UVC time of 6 days, the preterm infant has a posterior rate of late-onset sepsis of about 70% according to the nomogram. Our model also reveals the additive value of thyroid function for predicting incidence of late-onset bacterial sepsis. In the examples cited here, the posterior rate of late-onset sepsis is decreased to 40% if the preterm infants have normal thyroid function. There are two main advantages of safely and accurately evaluating individual prior probability of late-onset bacterial sepsis in preterm infants. On one hand, the health benefit is exposing fewer uninfected infants to antibiotic treatment. On the other hand, the social benefits include decreased health care expenditures and separation of mothers and newborns.

Conclusion

We suggest that thyroid hypofunction in preterm infants may increase the incidence of late-onset bacterial sepsis. The discrimination and calibration of a nomogram with thyroid function were better than the nomogram without thyroid function for predicting late-onset bacterial sepsis in preterm infants. The proposed nomogram in this study establishes an individual prior probability of late-onset sepsis in preterm infants, contributing to clinical assessment and treatment decisions. To generalize the use of this nomogram of late-onset bacterial sepsis in preterm infants, validation with data from more institutions is required.

Compliance with ethics requirements

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2008 (5). Informed consent was obtained from all patients for being included in the study.

Compliance with Ethics requirements

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2008 (5). Informed consent was obtained from all patients for being included in the study.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.