A Novel Algorithm With Paired Predictive Indexes to Stratify the Risk Levels of Neonates With Invasive Bacterial Infections: A Multicenter Cohort Study.

BACKGROUND: Our aim was to develop a predictive model comprising clinical and laboratory parameters for early identification of full-term neonates with different risks of invasive bacterial infections (IBIs).
METHODS: We conducted a retrospective study including 1053 neonates presenting in 9 tertiary hospitals in China from January 2010 to August 2019. An algorithm with paired predictive indexes (PPIs) for risk stratification of neonatal IBIs was developed. Predictive performance was validated using k-fold cross-validation.
RESULTS: Overall, 166 neonates were diagnosed with IBIs (15.8%). White blood cell count, C-reactive protein level, procalcitonin level, neutrophil percentage, age at admission, neurologic signs, and ill-appearances showed independent associations with IBIs from stepwise regression analysis and combined into 23 PPIs. Using 10-fold cross-validation, a combination of 7 PPIs with the highest predictive performance was picked out to construct an algorithm. Finally, 58.1% (612/1053) patients were classified as low-risk cases. The sensitivity and negative predictive value of the algorithm were 95.3% (95% confidence interval: 91.7-98.3) and 98.7% (95% confidence interval: 97.8-99.6), respectively. An online calculator based on this algorithm was developed for clinical use.
CONCLUSIONS: The new algorithm constructed for this study was a valuable tool to screen neonates with suspected infection. It stratified risk levels of IBIs and had an excellent predictive performance.

Neonatal invasive bacterial infections (IBIs), including septicemia and meningitis, account for 25% of global newborn deaths annually.[1] The standard for IBI diagnosis is the isolation of pathogens from blood or cerebrospinal fluid (CSF) culture. However, IBIs in neonates are often misdiagnosed because of nonspecific clinical symptoms and a low positive blood culture rate, which ranges from 5% to 40% in patients with suspected sepsis.[2-4] Clinically, patients with IBIs often experience a delay in diagnosis, thereby delaying the treatment, which leads to rapid deterioration and severe adverse consequences. Contrastingly, patients with symptoms of bacterial infections are often administered excessive medications, including unnecessary parenteral broad-spectrum antibiotics.[5,6]

In our previous study, we had developed a sequential algorithm which could risk-stratify neonates for bacterial meningitis and help clinicians in LP-related decision-making.[7] Nevertheless, it cannot distinguish risk levels of neonatal septicemia. These 2 diseases are relevant but independent: less than one-fourth of patients with positive blood culture had meningitis, while up to 38% neonates with bacterial meningitis had a negative blood culture.[8] It is essential to modify the initial model to adapt to risk-stratify neonatal IBIs including sepsis.

Some clinical inflammatory markers, such as C-reactive protein (CRP) and procalcitonin (PCT) levels, are valued predictive indicators for neonatal IBIs. However, these biomarkers do not have an ideal predictive value when evaluated as solitary predictors.[9] In contrast, Galetto-Lacour et al have confirmed that a laboratory risk index score, which combines CRP, PCT, and urine dipstick results, was an accurate tool for predicting severe bacterial infections in febrile infants without a known source of infection.[10] Tamelytė et al also demonstrated that several parameters of complete blood count (neutrophils, neutrophil-lymphocyte ratio, platelet count, etc.), together with CRP level, helped discriminate viral infections from septicemia in all children, especially in early-onset cases.[11]

Therefore, we speculated that the combination of 2 indicators, paired predictive indexes (PPIs) related to IBIs independently, would improve the predictive performance for IBIs in neonates. Additionally, a step-by-step algorithm that sequentially evaluated clinical and laboratory parameters was validated as having high sensitivity and negative predictive value for ruling out IBIs in young febrile infants.[12] However, this algorithm might not be applied to afebrile neonates. In this study, we aimed to establish an algorithm with PPIs for risk stratification of neonatal IBIs in a multicenter cohort, helping clinicians with treatment strategies. A handy online calculator based on this algorithm was further designed for clinical use.

MATERIALS AND METHODS

Study Design and Setting

We conducted a retrospective multicentric study from January 2010 to August 2019 in the neonatal units of 9 tertiary university children’s hospitals in China (see Table, Supplemental Digital Content 1, http://links.lww.com/INF/E615). All participating hospitals had adequate research capabilities to conduct the study and each of them verified data accuracy and agreed to data sharing. A detailed description of the study recruitment, eligibility, data collection, and questionnaires have been published elsewhere.[7]

Selection of Participants

Each center routinely admitted neonates with suspected severe bacterial infections and performed complete blood and CSF examinations. Exclusion criteria were (1) neonates with a confirmed neurosurgical diagnosis before admission or a history of invasive instrumentation of the central nervous system; (2) presence of complicated chronic conditions, such as congenital malformations, immunosuppressive therapy or immunodeficiencies, and chronic lung diseases[13]; and (3) critically ill neonates because these patients needed to be resuscitated immediately and repeated assessment might delay the treatment.

Measurements

Detailed information, including demographic characteristics, maternal medical history, clinical manifestations, diagnosis, auxiliary tests during hospitalization and administrations after admission, was collected from the electronic medical records and integrated by experienced data administrators. This study was carried out in compliance with the ethical standards established in the Declaration of Helsinki (as revised in 2013). The Institutional Review Board of each hospital approved the study and permitted data sharing. The informed consent requirement for this study was waived owing to its retrospective nature.

Laboratory tests, including WBC count, neutrophil percentage (NPC), absolute neutrophil count, CRP level, PCT level and bacteriologic identification in the blood and CSF culture or metagenomic next-generation sequencing (mNGS), were examined for each patient with suspected severe bacterial infections. The mNGS of RNA and DNA from blood or CSF offered a new strategy for diagnosing IBIs. Leukopenia was defined as a WBC count of <5000 cells/μL.[14] The cutoff values of NPC (74%), PCT (2.4 ng/mL) and CRP (25 mg/L) were defined based on the maximal Youden index in the receiver operating characteristics curves. The second sets of cutoff value for PCT (18 ng/mL) and CRP (62 mg/L) were derived from the 90th percentiles in this dataset. Other investigations, such as stool culture, urine culture, urine dipstick, ultrasound and chest radiography, were performed at the discretion of the clinicians in charge. The patients received antibiotic therapy based on the local guidelines of each site. The treatment regimens were customized individually according to the different pathogens and complications.

Abnormal neurologic signs included seizures, irritability, abnormal tension, hyporeflexia and bulging anterior fontanel.[15] Ill-appearances included such conditions as hypothermia, lethargy, poor feeding, recurrent vomiting (excluding surgical disease), aggravation of jaundice, highly pitched cry, unconsciousness, poor perfusion, grunting, cyanosis and apnea.[16,17]

Fever was defined by peak temperature >38°C as measured at home, in the pediatric emergency department or outpatient clinic, or on admission.[18]

The definite source of infection was determined from several common causes of bacterial infection in neonates, including impetigo, urinary tract infection, omphalitis, respiratory infection and purulent arthritis.

Indications for lumbar puncture (LP) were as follows: (1) no contraindications for LP, such as uncorrected bleeding diathesis, noncommunicating obstructive hydrocephalus, local skin infections, spinal stenosis or spinal cord compression above the level of puncture, and spinal or cranial developmental abnormalities[19]; and (2) clinical manifestations suggesting bacterial infections, such as fever or hypothermia, recurrent vomiting, poor feeding, jaundice aggravation, lethargy or restlessness, weak or high-pitched crying, irregular respirations, cyanosis, groaning, seizure, bulging fontanel and hypotonia.[16,20] Written informed consent for LP was obtained from all guardians of the neonates.

Outcome Measures

Neonatal IBI, including septicemia and bacterial meningitis, was defined as the isolation of bacterial pathogens in a positive culture and mNGS of blood and CSF. Group B Streptococcus (GBS), Escherichia coli, Klebsiella species, Enterobacter species, Staphylococcus aureus, Enterococcus species, hemolytic Streptococcus, Listeria monocytogenes, etc., were considered as qualifying pathogens; coagulase-negative staphylococci, Bacillus noncereus/nonanthracis, Lactobacillus, diphtheroids, viridans group streptococci, Micrococcus, etc., were categorized as contaminants.[14,21,22]

Statistical Analyses

Categorical variables are summarized as counts and percentages and compared using Fisher exact test or χ2 test. Continuous variables are reported as means ± standard deviations or medians (interquartile ranges [IQRs]) and compared using Student’s t test or Wilcoxon rank-sum test accordingly.

Three steps were used to construct an algorithm for risk stratification of neonatal IBIs. First, demographic characters, laboratory tests and clinical manifestations with P < 0.25[23] in the univariable analysis were considered as 9 candidate predictors (Table 1). Seven of them, independently related to IBIs, were identified by the first stepwise regression analysis, combined pairwise and named PPIs. In the second step, 10-fold cross-validation was applied to ensure the models’ robustness. The dataset was randomly split into 10 exclusive partitions, each accounting for 10% of the total dataset. One partition was selected as a validation set, and the respective 90% complement data for each partition was used as a training set. In each training partition, PPIs closely associated with IBIs were selected by the second stepwise regression analysis and introduced into a step-by-step algorithm sequentially; while in each corresponding validation partition, the predictive performance of the algorithm was calculated. This process was iterated through all the ten subsamples. In other words, the predictive abilities of all models generated from training datasets were evaluated in the corresponding validation datasets. In the third step, the model with the highest predictive performance was considered as the optimal model for risk classification of neonatal IBIs and employed in the whole dataset. Based on the sequential algorithm, we developed a handy online calculator. It was freely accessible to facilitate its clinical implementation.

TABLE 1.

Clinical and Laboratory Characteristics by Invasive Bacterial Infections (n = 1053)

Characteristics	Without IBIs(n = 887)	With IBIs(n = 166)	F/χ2/Z	P *
Clinical variables
Birth weight, mean (SD), g	3385 (456)	3385 (491)	0	0.991 [a]
Gestational age, mean (SD), weeks	38.9 (1.4)	38.9 (1.3)	–0.35	0.723 [a]
Age at admission, No. (%), days			26	<0.001 [b]
≤3	266 (30.0)	18 (10.8)
4–28	621 (70.0)	148 (89.2)
Sex, No. (%)			0.1	0.718 [b]
Male	521 (58.7)	100 (60.2)
Female	366 (41.3)	66 (39.8)
Delivery method, No. (%)			0.1	0.958 [b]
Vaginal delivery	497 (56.0)	100 (60.2)
Cesarean section	380 (42.8)	63 (38.0)
Missing	10 (1.1)	3 (1.8)
Fever, No. (%)†	497 (56.0)	108 (65.1)	4.7	0.031 [b]
Neurologic signs, No. (%)‡	120 (13.5)	52 (31.3)	32.4	<0.001 [b]
Ill-appearances, No. (%)§	231 (26.0)	75 (45.2)	24.8	<0.001 [b]
Definite source of infection, No. (%)¶	567 (63.9)	88 (53.0)	7.1	0.008 [b]
Laboratory variables
WBC, median (IQR), cells per µL	12.2 (8.6–17.5)	12.1 (7.1–17.4)	–1.5	0.139 [c]
PCT, median (IQR), ng/mL	0.4 (0.2–1.6)	3.0 (0.6–20.1)	9.7	<0.001 [c]
CRP, median (IQR), mg/L	8 (5–19)	31 (8–70)	9.2	<0.001 [c]
NPC, median (IQR), %	56.4 (41.6–70)	64.3 (50.4–76.5)	4.5	<0.001 [c]
ANC, median (IQR), cells per µL	6.4 (3.7–10.9)	7.6 (3.8–11.3)	0.5	0.652 [c]

Characteristics with P values <0.05 were considered statistically significant.

Peak temperature >38°C as measured at home, in the pediatric emergency department or outpatient clinic, or on admission.

Seizure, tension, irritability, bulging anterior fontanel, etc.

Hypothermia, poor feeding, lethargy, vomit, jaundice aggravation, cyanosis, apnea, etc.

Impetigo, urinary tract infection, omphalitis, pneumonia, purulent arthritis, etc.

Student’s t test.

χ2 test.

Wilcoxon rank-sum test.

ANC indicates absolute neutrophil count; CRP, C-reactive protein; IBIs, invasive bacterial infections; IQR, interquartile range; NPC, neutrophil percentage; PCT, procalcitonin; SD, standard deviation; WBC, white blood cell count.

Compared with the published paper on meningitis, there were several difference between the statistical analyses of 2 studies. In the first step, both of the 2 studies used stepwise regression analysis to select the variables, which were the most relevant subsets for disease screening. The slight difference was that, the predictors in this current paper were combined in pairs, while the previous model did not. The most significant difference was in the second step. A 10-fold cross-validation was performed to select the model with the best predictive efficiency for IBIs in this current research, while the dataset in the previous research was split into derivation and external validation dataset, and the predictive performance of model for meningitis was tested in the external validation set. The third steps of these 2 studies were the same, in which the step-by-step algorithms were established.

Based on our pilot studies, the estimated sample size was at least 867, assuming a marginal error of 0.05, a sensitivity of 0.95, a specificity of 0.70 and a prevalence of 0.15.[24] All statistical analyses were conducted using SAS statistical software version 9.4 (SAS Institute, Inc., Cary, NC). A P value of <0.05 was considered statistically significant. The web calculator was developed using Microsoft Visual Studio version v2015 (Microsoft Corporation, Beijing, China [Chinese headquarters]).

RESULTS

Patient Population

Of the 2213 full-term neonates 0−28 days old, we excluded the patients whose first LP was performed beyond 72 hours of admission (n = 251); who underwent traumatic LP (CSF red blood cell count of >500/mm3) (n = 26)[25]; whose complete blood count, CRP and PCT were conducted beyond 24 hours after admission; in whom information on the test time was missing (n = 541); and who underwent antibiotic pretreatment before admission (for neonates >3 days) or 3 days before delivery (for neonates <3 days) (n = 342). Ultimately, 1053 neonates were eligible for the final analyses (see Figure, Supplemental Digital Content 2, http://links.lww.com/INF/E616).

Characteristics of the Study Subjects

Baseline characteristics of the study subjects, including 166 patients with IBIs and 887 without IBIs, are summarized in Table 1. The neonates with an age at admission >3 days and those with fever, abnormal neurologic signs, ill-appearance, absence of a source of infection and high levels of CRP, PCT and NPC were more likely to have IBIs (P < 0.05). No significant difference was found in birth weight, gestational age, sex, delivery method, WBC and absolute neutrophil count between the 2 groups of infants with and without IBIs (P > 0.05).

Main Results

First, 7 statistically significant single predictors were identified as having great diagnostic value for IBIs, selected by stepwise regression analysis, including WBC count, CRP level, PCT level, NPC, age at admission, abnormal neurologic signs and ill-appearances. As shown in Table 2, infants with age at admission >3 days and those with abnormal neurologic signs and ill-appearances had 3.4-fold (95% CI: 2.1−5.8), 2.9-fold (95% CI: 2.0−4.3) and 2.6-fold (95% CI: 1.8−3.7) high risks of IBIs, respectively. Infants with high levels of CRP of >25 mg/L or >62 mg/L had 5.0-fold (95% CI: 3.5−7.2) or 4.6-fold (95% CI: 3.0−7.1) and those with a PCT level of >18 ng/mL or >2.4 ng/mL had 5.0-fold (95% CI: 3.2−7.9) or 5.3-fold (95% CI: 3.7−7.5) increased risks of having IBIs, respectively. Compared with neonates with a WBC count ≥5000 cells/μL, or those with an NPC of ≤74%, others had 4.2-fold (95% CI: 2.5−7.0) and 2.2-fold (95% CI: 1.5−3.2) increased risks of IBIs, respectively. These 7 single predictors were then combined into 23 pairs of PPIs, and the predictive performance for IBIs of each PPI was calculated (see Tables, Supplemental Digital Content 3, http://links.lww.com/INF/E617 and 4, http://links.lww.com/INF/E618).

TABLE 2.

Risk of Invasive Bacterial Infections by a Single Predictor

Predictors	Without IBIsN (%)	With IBIsN (%)	cOR(95% CI)	aOR(95% CI)
	887 (84.2)	166 (15.8)	/	/
WBC <5000 cells/µL	48 (5.4)	29 (17.5)	3.7 (2.3–6.1)	4.2 (2.5–7.0)
CRP >25 mg/L	171 (19.3)	95 (57.2)	5.3 (3.7–7.5)	5.0 (3.5–7.2)
CRP >62 mg/L	59 (6.7)	46 (27.7)	5.1 (3.3–7.8)	4.6 (3.0–7.1)
PCT >2.4 ng/mL	169 (19.1)	93 (56.0)	5.4 (3.8–7.6)	5.3 (3.7–7.5)
PCT >18 ng/mL	59 (6.7)	43 (25.9)	4.9 (3.2–7.5)	5.0 (3.2–7.9)
NPC >74%	158 (17.8)	53 (31.9)	2.2 (1.5–3.1)	2.2 (1.5–3.2)
Age at admission >3 days	621 (70.0)	148 (89.2)	3.5 (2.1–5.9)	3.4 (2.1–5.8)
Neurologic signs*	120 (13.5)	52 (31.3)	2.9 (2.0–4.3)	2.9 (2.0–4.3)
Ill-appearances†	231 (26.0)	75 (45.2)	2.3 (1.7–3.3)	2.6 (1.8–3.7)

Seizure, tension, irritability, bulging anterior fontanel, etc.

Hypothermia, poor feeding, lethargy, vomit, jaundice aggravation, cyanosis, apnea, etc.

aOR indicates adjusted odds ratio, adjusted for hospitals; CI, confidence interval; cOR, crude odds ratio; CRP, C-reactive protein; IBIs, invasive bacterial infections; NPC, neutrophil percentage; PCT, procalcitonin; WBC, white blood cell count

Ten models, with 5 to 9 PPIs correspondingly, were derived in the process of 10-fold cross-validation (see Table, Supplemental Digital Content 5, http://links.lww.com/INF/E619). Table, Supplemental Digital Content 6, http://links.lww.com/INF/E620 shows the predictive performance of these models. A model having 7 PPIs and the highest predictive performance, an accuracy of 98.8%, a sensitivity of 94.1% and an NPV of 98.8% was selected for further model evaluation in the whole dataset.

Table 3 presents the predictive performance of each selected PPI after the 10-fold cross-validation process. Sensitivity was 11.7%–49.7% and specificity was 69.1%–99.8%, and all PPIs had lower sensitivity than specificity. The PPI of a combination of WBC count of <5000 cells/μL and PCT value of >18 ng/mL had the highest specificity (99.8 [95% CI: 99.5−100]) for IBIs but the lowest sensitivity (11.7 [95% CI: 8.0–17.7]), while the PPI of age at admission >3 days and PCT level of >2.4 ng/mL had the highest Youden index, that is, the highest sum of specificity (92.1 [95% CI: 90.4–93.5]) and sensitivity (48.7 [95% CI: 41.1–55.1]). These 7 PPIs were then introduced into a step-by-step algorithm (Fig. 1).

TABLE 3.

The Risks, Prevalence of Invasive Bacterial Infections and Predictive Performance for Invasive Bacterial Infections by 7 Paired Predictive Indexes

No. of PPI	PPI 1	PPI 4	PPI 5	PPI 11	PPI 12	PPI 14	PPI 15
Predictor 1	WBC<5000 cells/µL	PCT>2.4 ng/mL	CRP>25 mg/L	NPC>74%	Neurologic signs*	CRP>25 mg/L	NPC>74%
Predictor 2	PCT>18 ng/mL	Age at admission >3 days	Ill-appearances ‡	Age at admission >3 days	Age at admission >3 days	Age at admission >3 days	Ill-appearances†
	(n = 22)	(n = 151)	(n = 80)	(n = 109)	(n = 114)	(n = 211)	(n = 78)
Risks of IBIs, n (%)	20 (90.9)	80 (53.0)	41 (51.2)	45 (41.3)	46 (40.4)	82 (38.9)	30 (38.5)
cOR (95% CI)	60.6 (14.0, 261.8)	10.7 (7.2–15.8)	7.1 (4.4–11.5)	4.8 (3.1–7.3)	4.6 (3.0–7.0)	5.7 (4.0–8.2)	3.9 (2.4–6.3)
aOR (95% CI)	55.4 (12.7–240.9)	10.7 (7.0–16.2)	6.9 (4.2–11.3)	4.7 (3.1–7.3)	5.4 (3.4–8.5)	5.5 (3.8–8.1)	4.0 (2.4–6.6)
Sen (%) (95% CI)	11.7 (8.0–17.7)	48.7 (41.1–55.1)	24.5 (20.1–30.0)	27.8 (20.7–34.5)	27.0 (20.4–32.9)	49.7 (43.2–57.1)	48.8 (41.0–56.7)
Spe (%) (95% CI)	99.8 (99.5–100)	92.1 (90.4–93.5)	95.6 (94.4–96.5)	92.3 (90.4–93.8)	92.7 (91.2–94.7)	85.4 (83.0–87.2)	69.1 (66.0–72.1)
+LR (95% CI)	48.5 (22.3–100)	6.0 (4.9–7.6)	5.6 (4.3–7.4)	3.6 (2.6–4.9)	3.7 (2.6–5.3)	3.4 (2.8–4.1)	1.6 (1.3–1.9)
–LR (95% CI)	0.9 (0.8–0.9)	0.6 (0.5–0.6)	0.8 (0.7–0.8)	0.8 (0.7–0.9)	0.8 (0.7–0.9)	0.6 (0.5–0.7)	0.7 (0.6–0.9)
PPV (%) (95% CI)	90.9 (80.6–100)	52.9 (46.2–59.3)	50.7 (42.7–59.3)	40.4 (31.6–49.1)	41.3 (32.1–50.9)	39.4 (33.0–44.9)	22.8 (18.6–27.5)
NPV (%) (95% CI)	85.9 (84.2–87.6)	90.6 (88.8–92.0)	87.1 (85.9–88.5)	87.3 (85.6–89.1)	87.2 (85.7–88.7)	90.0 (88.6–91.6)	87.8 (85.2–90.2)

Seizure, tension, irritability, bulging anterior fontanel, etc.

Hypothermia, poor feeding, lethargy, vomit, jaundice aggravation, cyanosis, apnea, etc.

aOR indicates adjusted odds ratio, adjusted for hospitals; cOR, crude odds ratio; CRP, C-reactive protein; IBIs, invasive bacterial infections; -LR, negative likelihood ratio; +LR, positive likelihood ratio; NPC, neutrophil percentage; NPV, negative predictive value; PCT, procalcitonin; PPI, paired predictive index; PPV, positive predictive value; Sen, sensitivity; Spe, specificity

FIGURE 1.

Prevalence of neonatal invasive bacterial infections in the different risk subgroups (0−28 days). Neurologic signs: seizure, tension, irritability, bulging anterior fontanel, etc. Ill-appearances: hypothermia, poor feeding, lethargy, vomit, jaundice, cyanosis, apnea, etc. −LR indicates negative likelihood ratio; CRP, C-reactive protein; IBIs, invasive bacterial infections; NPV, negative predictive value; PCT, procalcitonin.

Using this new approach, the prevalence of IBIs in the subgroups of patients with different PPIs was determined (Fig. 1). The first step, evaluating a WBC count of <5000 cells/μL and a PCT level of >18 ng/mL, identified 22 patients with an IBI prevalence of 90.9% (20/22). Considering ill-appearances combined with either CRP level of >25 mg/L or NPC of >74%, we identified a subgroup of patients with a prevalence of IBIs of 39.4% (50/127). In the next 2 steps, we combined age at admission >3 days with abnormal laboratory tests (PCT level of >2.4 ng/mL, CRP level of >25 mg/L, or NPC of >74%) and neurologic signs, by which we further identified 42.8% (71/166) and 10.2% (17/166) of cases of IBIs, respectively. Ultimately, 95.2% (158/166) of neonatal IBI cases were ruled in and 58.1% (612/1053) of patients were classified as low-risk cases. The predictive performance of the algorithm was superior to the multivariate logistic regression with the 7 predictors (not pairwise) or even 7 PPIs in our pilot study (see Table, Supplemental Digital Content 4, http://links.lww.com/INF/E618). Additionally, the clinical features and laboratory tests of 8 misclassified patients are shown in Table, Supplemental Digital Content 7, http://links.lww.com/INF/E621.

In conformity with the step-by-step algorithm, a user-friendly online calculator was subsequently established to provide clinicians with a concise method to assess the risk of neonatal IBIs. The calculator can be accessed at http://infantsmc.cn/RC2/ (see Figure, Supplemental Digital Content 8, http://links.lww.com/INF/E622).

DISCUSSION

In this multicenter study, using both clinical (age at admission, neurologic signs and ill-appearances) and laboratory (WBC count, CRP level, PCT level and NPC) parameters, we derived a novel algorithm to identify neonates at different risks of IBIs. Rather than single predictors, PPIs were employed in each step, with an accuracy of 98.7% and a sensitivity of 95.3%. The prevalence of IBIs (15.8%) reported in this study was relatively higher than that reported in other studies (2.3%−4.0%).[12,26,27] This discrepancy could be explained by different denominators. Subjects in this study were included from neonatal units, with a higher probability of IBIs than in febrile neonates from emergency departments (ED).

We established the algorithm starting with abnormal laboratory tests (WBC count and PCT level), which were readily available.[14] Leukopenia is a risk factor for early-onset sepsis in neonates.[28] However, the peripheral WBC count alone is not appropriate for IBI screening,[29] which would be influenced by the age of onset, sex, mode of delivery, immune status and hematologic diseases.[30] Meanwhile, asphyxia, pneumothorax, surgery and other noninfectious inflammatory stimuli, such as burning, can cause increased PCT levels.[30,31] Additionally, in terms of physiologic status, PCT levels in healthy full-term infants may also be transiently high within the first 4 days of life.[32] Therefore, rather than using them in isolation, the combination of these parameters may significantly increase the interactions and help in stratifying the risk more efficiently.[10] In this step, we chose PCT level with 90th percentiles combined with a WBC count of <5000 cells/μL, which could help in identifying IBI cases with an extremely high risk.

Although the inflammatory biomarkers in serum could help to detect IBIs, the lack of associated symptoms and physical examination findings may augment the diagnostic uncertainty.[14,18] Therefore, the clinical symptoms combined with CRP and NPC were employed in the next step. Of note, fever was not introduced into this algorithm because it was not an independent risk factor in the stepwise regression. It was accepted that fever was not the specific symptom for neonatal IBIs, as more than half of the neonates diagnosed with sepsis are afebrile.[33] Early-onset patients (age of onset, ≤3 days), in particular, were less capable of producing sufficient inflammatory responses in the first few days after birth and did not present with temperature increases.[28]

Early-onset IBIs, within 72 hours or 7 days of birth, are different from late-onset IBIs in terms of etiology, risk factors, clinical manifestations and laboratory results.[30,34,35] Our research indicated that age at admission >3 days was a risk factor for IBIs, with an adjusted odds ratio of 3.4 (95% CI: 2.1−5.8). This result was consistent with that reported previously.[34] The possible explanation was that routine pathogenic screening in pregnant women, such as GBS, and reasonable antepartum or intrapartum prophylactic antibiotics may lead to a descendent trend of IBIs in neonates ≤3 days of age.[36] Although the specificity of the age of onset was low, its sensitivity reached 90%, implying that it may be an ideal index for IBI screening. In this case, the age of onset combined with laboratory results (CRP level, PCT level and NPC) or neurologic signs may improve the predictive performance of the model. Finally, the algorithm identified more than half of neonates with low-risk for IBIs, and most of these neonates might not need parenteral antibiotic treatment.

As shown in Table 3, the sensitivity of the 7 solitary PPIs was no >50%, indicating poor screening efficiencies. Our algorithm could greatly improve the sensitivity of prediction and increase the accuracy of screening. In contrast, the low specificity and PPV indicated a potential false-positive rate, that is, some non-IBI cases, such as urinary tract infections, were misclassified into a high-risk subgroup. This situation may be acceptable because a delay in the diagnosis could result in substantial mortality and disability rate.

In our study, we excluded 24.5% of patients (342/1395) who received antibiotics before admission. Table, Supplemental Digital Content 9, http://links.lww.com/INF/E623, indicated no difference in terms of clinical and laboratory parameters between patients with and without antibiotic pretreatment, except for NPC. This finding suggests that antibiotics have a significant effect on NPC, and the use of antibiotics before diagnosis may have a certain impact on the detection rate of IBIs. Additionally, although our results were derived from hospitalized neonates, our algorithm could also be applied to patients in the ED because we excluded cases with antibiotic pretreatment.

Moreover, 8 patients were misclassified into the low-risk subgroup by this algorithm. Among them, 4 cases were of Enterococcus, 2 Escherichia coli, 1 GBS and 1 Staphylococcus aureus. The identification rate of IBIs differed as per the bacterial species: 77.8% (14 of 18) for Enterococcus, 95.8% (46 of 48) for Escherichia coli and 98.0% (50 of 51) for GBS, respectively (see Tables, Supplemental Digital Content 7, http://links.lww.com/INF/E621, and 10, http://links.lww.com/INF/E624). Compared with identifying patients with IBIs of Enterococcus, the algorithm may have a better ability to identify neonates with IBIs of Escherichia coli and GBS. Additionally, 2 early-onset cases had a history of premature rupture of membranes for >18 hours, which was strongly associated with early-onset IBIs. Despite having fever, detectable ill-appearances, including vomit, lethargy or poor feeding, other late-onset cases did not present with any specified neurologic signs, and their laboratory findings were also not profoundly abnormal, which may be caused by the short duration from onset to admission (<2 hours). Therefore, an appropriate duration of observation is required, and the risk for IBIs must be reassessed repeatedly if low-risk neonates have unsatisfactory recovery after preliminary antibiotic treatment.

As this algorithm comprised complicated steps with 7 PPIs, which were not convenient for practical application, a free handy online calculator was further developed based on this algorithm to facilitate clinical use. For example, an 18-day-old full-term neonate without antibiotic pretreatment presented to the ED with an ill-appearance (fever, lethargy and poor feeding) but without any specified aforementioned neurologic signs. WBC count, CRP level, PCT level and NPC on admission were 3500 cells/μL, 56 mg/L, 24.3 ng/mL and 85.8%, respectively. When we entered this information into the online calculator, the patient was automatically classified into the high-risk subgroup with a recommendation: “This patient is at a high-risk of IBIs and strongly suggested to be hospitalized for pathogenic investigation.” Finally, the patient was diagnosed with neonatal bacterial meningitis by an LP procedure.

Our IBIs algorithm, to some degree, had a similar form to the initial one that developed for neonatal meningitis.[34] However, their derived processes and target population were distinctively diverse. As the similar method was not able to construct an IBIs model with good performance, we used PPIs and 10-fold cross-validation to increase the predictive efficiency. From the clinical perspective, the current algorithm had a wider clinical application in comparison to the initial model of meningitis, by which clinicians can identified the neonates with low-risks of IBIs, that is, septicemia and meningitis. Simultaneously, it also could optimize the management of antibiotic using in more patients and decrease the antibiotic resistance in the early life.

Our algorithm has several strengths. First, because of the low positive rate of culture, the diagnosis of some false-negative cases could be missed. Our algorithm may identify some of these clinically diagnosed cases and increase the accuracy of the IBI detection rate. Second, to our knowledge, similar clinical studies, especially for IBI prediction in neonates, have rarely been reported. Some previously developed prediction rules were mainly designed for old and/or febrile infants, which were not rigorous for neonates.[12] The transition from intrauterine to extrauterine life requires complex physiologic and biochemical changes in the neonatal period, and their disease characteristics could be distinguished from those of patients of other ages.[37,38] Third, the algorithm may also help medical decision-making, for example, LP procedure and the avoidance of excessive investigations and medications.

There are several limitations to this study. First, we did not have information on the duration from initial symptom onset to laboratory test performance. However, because most IBI cases exacerbate acutely, such that patients are usually referred to hospitals for medical advice soon after onset, we used data on the duration from admission to laboratory test performance instead. To further minimize the degree of interference, we only included patients whose laboratory test results were accepted within 24 hours after admission. Second, our study population did not include patients in whom blood culture or CSF analysis was not performed, some of whom might be having IBIs. However, the condition of these cases was relatively mild or sensitive to antibiotic treatment so that it might have a limited effect on the results. Third, our algorithm could not be applied to patients with underlying conditions, especially those with immunodeficiencies (eg, severe combined immunodeficiency), who could have IBIs without corroborating results and would be classified into the high-risk subgroup. Finally, although the 10-fold cross-validation has been one of the most popular methods for model selection and can diminish the negative influence of overfitting data to some degree,[39] it may not eliminate it. Further external validation studies are needed to test its generalizability.

CONCLUSIONS

Our algorithm was an evidence-based and valuable predictive model to help clinicians risk-stratify IBIs in full-term neonates by using rapidly acquired clinical signs and laboratory tests. It could identify the patients who need detailed observation and examination (such as blood and cerebrospinal fluid culture/metagenomic next-generation sequencing). Simultaneously, an easy-to-use online calculator had been further developed to facilitate a more efficient clinical diagnosis of IBIs.

Ethical approval was granted by the Institutional Review Board of each hospital involved in this study and conducted according to the Helsinki declaration.

ACKNOWLEDGMENTS

We are grateful to all those who contributed to the project design and implementation, including study coordinators, data collectors, data clerks and other partners.