Evaluation of risk stratification strategies in pediatric patients with febrile neutropenia.

OBJECTIVE: To evaluate the performance of risk stratification protocols for febrile neutropenia specific to the pediatric population.
METHODS: Retrospective study of a cohort of pediatric patients undergoing cancer treatment with episodes of neutropenia due to chemotherapy and fever, treated at the emergency department of a tertiary cancer hospital from January 2015 to June 2017. Patients who were bone marrow transplant recipients and patients with neutropenia due to causes other than chemotherapy were excluded. Six protocols were applied to all patients: Rackoff, Alexander, Santolaya, Rondinelli, Ammann 2003, and Ammann 2010. The following outcomes were assessed: microbiological infection, death, ICU admission, and need for more than two antibiotics. The performance of each protocol was analyzed for sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and receiver operator characteristic (ROC) curve.
RESULTS: This study evaluated 199 episodes of febrile neutropenia in 118 patients. Microbiological infection was identified in 70 samples from 45 distinct episodes (22.6%), 30 patients used more than two antibiotics during treatment (15%), eight required ICU admission (4%), and one patient died (0.8%). Three protocols achieved high sensitivity indices and NPV regarding the outcomes of death and ICU admission: Alexander, Rackoff, and Ammann 2010; however, Rackoff showed higher sensitivity (0.82) and NPV (0.9) in relation to the microbiological infection outcome.
CONCLUSION: The Rackoff risk rating showed the best performance in relation to microbiological infection, death, and ICU admission, making it eligible for prospective evaluation.

Introduction

Pediatric patients submitted to a chemotherapy regimen, presenting with febrile neutropenia, represent a group in which the possibility of an infectious process is greater, due to frequent hospitalizations, drug-induced immunodepression, colonization by multidrug-resistant bacteria, and devices such as venous catheters, catheters, and tracheostomies that compromise the body's natural defense barriers. Pediatric patients, in comparison to adults, are more likely to have an undetected infectious focus and have an approximate incidence of 60% of febrile neutropenia episodes caused by bacterial infection, with mortality rates of up to 80% in gram-negative infections.

The initial approach to the pediatric patient with febrile neutropenia involves the assessment of several factors, such as the type of neoplasia and its staging, the estimated time of neutropenia, detailed physical examination, laboratory assessment, and past history of colonization by multidrug-resistant microorganisms, among others. Historically, care of patients with febrile neutropenia necessarily involved hospitalization, infectious screening, and administration of broad-spectrum antibiotics.2, 3, 4, 5, 6 However, in the last decades, risk stratification protocols have been developed giving rise to alternative approaches,2, 3, 4, 5, 6, 7, 8 such as discharge after a short period of hospitalization and outpatient treatment.

In a recent international meta-analysis involving ten different countries, evidence-based guidelines for the empirical treatment of febrile neutropenia in pediatrics were developed. These guidelines referenced six validated risk stratification protocols for Pediatrics, namely: Ammann 2003, Santolaya 2001, Alexander 2002, Ammann 2010, Rackoff 1996, and Rondinelli 2006, as described in Annex1.

These risk stratification strategies are prediction protocols in pediatric patients with febrile neutropenia, ratified in prospective and/or retrospective studies that utilize several outcomes as evaluation measures, such as death, admission to ICU, severe infection, and bacteremia.2, 3, 4, 5, 6, 8 The evaluation of these studies does not allow a single low-risk prediction scheme to be recommended as more efficient than the others, nor does it allow safely recommending different protocols to predict specific outcomes. Geographic and temporal variations require local validation for the use of a routine protocol.

The aim of this article is to evaluate the performance of six available risk stratification protocols for febrile neutropenia in pediatric patients treated at the National Cancer Institute (Instituto Nacional do Câncer [INCA]) with febrile complications, during the neutropenia period.

Methods

Study site and period

This was a retrospective cohort of pediatric patients treated at INCA, Rio de Janeiro, Brazil, between January 2015 and June 2017.

Inclusion/exclusion criteria

All patients up to and including 18 years of age who were treated at the emergency department with a diagnosis of febrile neutropenia resulting from chemotherapy for the treatment of solid or hematological cancers who were not previously hospitalized were included, regardless of the stage of the disease. Bone marrow transplant recipients were excluded due to the particularities of this population, such as the immunosuppressive chemotherapy regimen, often associated with irradiation, high frequency of graft-vs.-host disease, infectious complications by bacteria and fungi due to the absence of neutrophils, below normal Tcell quantity and function, persisting for months after transplantation even when the total neutrophil count is already normalized, and, finally, the use of the consensus of transplanted patients’, adopted by all studied risk stratification protocols. Patients with neutropenia due to causes other than chemotherapy were excluded, as well as unconfirmed fever or lack of medical records.

Patient care flow

All patients with febrile neutropenia were admitted and treated with broad-spectrum antibiotic monotherapy, with the initial choice being cefepime or meropenem or piperacillin-tazobactam, according to the colonization, initial diagnostic suspicion, and past pathological history. Data collection covered the exams performed and the patient's clinical evolution from the moment of admission to the end of antibiotic administration, regardless of hospital discharge. Then, each service record was submitted to the classification of the six risk stratification strategies available in the literature, according to their respective protocols.

Data analysis source

Care spreadsheets of the pediatric emergency department and the medical records of patients admitted to the pediatric ward were used.

Definitions of variables used in the analysis

Fever was defined as a single axillary measure, either reported, or at admission, ≥37.8 °C. Neutropenia was conceptualized as an absolute neutrophil count ≤500 cells/mm3 or between 1000 and 500 cells/mm3 with a tendency toward decrease in the next 72 h. Confirmed microbiological infection was defined as the identification of a pathogen in a sterile site or isolation of a pathogen in a non-sterile site, as long as it was associated with the local symptoms of the affected site. Arterial hypotension was defined as a systolic blood pressure measurement below the 5th percentile for age. Hypoxemia was established as oxygen saturation in ambient air <94%. Severe mucositis was described as the oropharynx involvement by mucositis that required medication or venous hydration. Death was counted as death up to two weeks after admission, with the patient still being treated for a confirmed or suspected infectious process. ICU admission was established for all patients who required vasoactive amines, ventilatory prosthesis, or had imminent risk of death, as declared by the attending physician.

Protocols used for febrile neutropenia evaluation

Six validated risk stratification strategies for Pediatrics were applied to all included patients.2, 3, 4, 5, 6, 8

Analyzed outcomes

The primary outcome was the occurrence of microbiological infection. The secondary outcomes were death, ICU admission, and the need for more than two antibiotic agents.

Data analysis

Means, frequencies, odds ratios (ORs), and the respective 95% confidence intervals (95%CIs) were calculated. To assess the performance of each protocol in the risk classification of patients with febrile neutropenia, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated according to the following outcomes: death, ICU admission, microbiological infection, and administration of more than two antibiotic agents. To model the correlation between observations, considering that some patients had more than one episode of febrile neutropenia, the OR and its 95%CI were calculated using the generalized estimation equations (GEE) model with a compound symmetry matrix, having the microbiological infection outcome as a reference. Receiver operator characteristic (ROC) curves were constructed for each outcome according to the risk classification protocols, with the area below the curve being estimated. The analyses were performed using the R program (R Development Core Team [Internet]. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2003. Available from: http://www.R-project.org). The level of statistical significance was set at 5%.

Ethical aspects

The study was approved by the INCA Research Ethics Committee, CAAE 67238217.2.0000.5274, on 04/31/2017.

Results

General aspects and demographic data

A total of 220 episodes of febrile neutropenia were identified and 21 cases were excluded due to suspected, but not confirmed, fever or error in the admission diagnosis record. The 199 episodes included occurred in 118 patients. The mean age was 105.7 months (8.8 years), the median was 90 months (7.5 years), and they ranged from 11 to 222 months. The demographic profile comprised 42.4% Caucasians, 46.6% mixed-race, and 11% Afro-descendant individuals. As for gender, 49.2% were males and 50.8% were females. Regarding the weight assessment of the studied population, the mean body mass index (BMI) was 18.54 kg/m2 (8.6–47.48 kg/m2).

Seventy-five patients had one episode of febrile neutropenia, 20 patients had two episodes, 15 patients had three episodes, four patients had four episodes, two had five episodes, one had six episodes, and one patient had seven episodes. Thirty patients (15%) used more than two antibiotics during the treatment of febrile neutropenia, eight patients (4%) required admission to the ICU, and only one patient (0.8%) died. The presence of semi- or totally-implanted central venous catheter was verified in 90.7% of patients.

Underlying pathologies

Of the studied cancers, 26.2% were hematological tumors and 73.7% were solid tumors. Table 1 shows the frequency of the types of cancers found.

Table 1

Frequency of cancer cases in episodes of febrile neutropenia in children treated at the National Cancer Institute (January 2015–June 2017).

Diagnosis	n (%)
Leukemia	23 (19.5)
Central nervous system tumor	19 (16.1)
Osteosarcoma	19 (16.1)
Renal tumor	16 (13.6)
Rhabdomyo sarcoma	10 (8.5)
Lymphoma	8 (6.8)
Retinoblastoma	7 (5.9)
Neuroblastoma	6 (5.1)
Ewing	6 (5.1)
Ovarian tumor	2 (1.7)
Desmoplastic tumor	1 (0.8)
Germ cell tumor	1 (0.8)
Total	118 (100%)

Bacteremia and isolated infectious agents

Microbiological identification was attained in 70 samples, from 45 different episodes of neutropenia, in 27 patients.

The etiological profile of positive samples in patients with febrile neutropenia was as follows: 35 Gram-negative bacteria (50%), 27 Gram-positive bacteria (38.5%), and eight fungi (11.4%). Of this total, 41.4% were isolated from central venous catheters. Among the identified Gram-negative bacteria, the most frequent were Klebsiella pneumoniae (14.2%) and Pseudomonas aeruginosa (14.2%). Of the Gram-positive microorganisms, Staphylococcus sp. (12.8%) was the most common. The occurrence of microbiological infection was not associated with the presence of a central venous catheter (OR 2.78; 95% CI 0.56–13.80), gender (OR 0.77; 95% CI 0.37–1.61), ethnicity/skin color (OR 1.72; 95% CI 0.59–5.03), age (OR 1.00; 95% CI 1.00–1.01), or underlying pathology (OR 1.92; 95% CI 0.44–8.36).

Performance analysis of the classification and risk protocols for febrile neutropenia

Sensitivity, specificity, PPV, and NPV were calculated for the six protocols. The results are shown in Table 2. The graphical evaluation of the performance of tests for the outcomes of death, ICU admission, microbiological infection, and the need for more than two antibiotics was performed using the ROC curves (Fig. 1).

Table 2

Comparative analysis of the protocols used for the management of children with febrile neutropenia (National Institute of Cancer, January 2015–June 2017).

	Death				ICU				MI				>2 antibiotics
	S	Sp	PPV	NPV	S	Sp	PPV	NPV	S	Sp	PPV	NPV	S	Sp	PPV	NPV
Rackoff	1.00	0.43	0.03	1.00	1.00	0.45	0.10	1.00	0.82	0.49	0.32	0.90	0.76	0.47	0.30	0.87
Alexander	1.00	0.56	0.04	1.00	1.00	0.59	0.12	1.00	0.73	0.64	0.37	0.89	0.73	0.64	0.37	0.89
Rondinelli	0.75	0.71	0.05	0.99	0.91	0.73	0.17	0.99	0.51	0.76	0.38	0.84	0.58	0.78	0.43	0.86
Santolaya	0.67	0.72	0.04	0.99	0.56	0.73	0.09	0.97	0.46	0.76	0.36	0.83	0.51	0.78	0.42	0.84
Ammann (2003)	1.00	0.42	0.03	1.00	1.00	0.44	0.09	1.00	0.77	0.47	0.31	0.87	0.80	0.48	0.32	0.88
Ammann (2010)	0.75	0.48	0.03	0.99	0.82	0.49	0.09	0.98	0.64	0.51	0.28	0.83	0.69	0.52	0.30	0.85

MI, microbiological infection; PPV, positive predictive value; NPV, negative predictive value.

Figure 1

Comparative analysis of ROC curves of the protocols used for risk classification in pediatric patients with febrile neutropenia (National Cancer Institute, January 2015–June 2017).

In the 199 analyzed episodes, the frequency of patients classified as low risk showed differences. Using the Rackoff protocol, 42% of the episodes were considered as low risk, whereas this figure was 55% according to Alexander, 69% according to Rondinelli, 66% according to Santolaya, 39% according to Ammann2003, and finally, 47% according to Ammann2010.

Discussion

The onset of fever during cancer treatment is an event of concern, given the mortality involved, the additional costs of hospitalization and laboratory tests, the delay in chemotherapy treatment, and the psychosocial effects on the family and the multidisciplinary team.2, 7, 8 All efforts made to dichotomize and analyze the variables involved in the complex care for this group of patients will undoubtedly result in greater medical care efficiency and an increase in the characterization of the particularities pertinent to this reality. In several countries, hospital units that adopt a risk stratification system offer outpatient treatment for low-risk neutropenic patients who can tolerate oral medications and have easy access to the hospital unit.

In the present study, the sample consisted of children with no gender predominance, with a predominance of Caucasians and mixed-race individuals, with a mean age of 8.8 years. The literature shows little variation in relation to gender, with few studies and no predominance or with a slight majority of males.3, 4, 8 Skin color/ethnicity as statistical data was not estimated in the vast majority of studies.2, 3, 4, 5, 6

The mean age at the febrile neutropenia episodes also showed a wide variation (5.2–8.9 years).4, 6, 8 Only one study found significance for the age variable as a risk predictor, with patients younger than 5 years old showing a higher risk.

The presence of a central venous catheter shows an even higher variation, from studies where the presence of a central catheter in episodes of febrile neutropenia comprised less than 50% of the patients, to studies where the prevalence was 97%.2, 3, 4, 5, 6, 8 Of the six studies from which the evaluated protocols originated, the presence of a central venous catheter was considered in the statistical analysis in four of them,2, 3, 6, 8 but only two concluded that this variable was pertinent.3, 8 In the present study, with a prevalence of 90.7% of patients with central venous catheter, the occurrence of bacteremia was not associated with the presence of central access, reflecting a lower rate of complications associated with this device.

In the present study, the most common underlying disease was leukemia, which is in accordance with the current literature.2, 3, 4, 5, 6, 8, 13 The occurrence of microbiological infection was not associated with an oncological diagnosis. In the assessed series, the oncological diagnosis was also not determinant in the risk classification; other factors such as diagnosis of leukemia recurrence, recurrence with spinal cord involvement, progressive disease, or the chemotherapy type seem to be more important. Only Alexander, using a sequence of exclusion criteria, among them, the diagnosis of acute myeloid leukemia and Burkitt's lymphoma to differentiate between low and high risk, found significant differences between the groups.

The evaluation of protocol performance as diagnostic tests, considering only sensitivity and specificity through the ROC curve representation, illustrates the low values of the sensitivity/1-specificity ratio also found in the literature. In a comparative study, Ammann found the following sensitivity and specificity values for severe bacterial infection, respectively: Santolaya 0.93 and 0.19, Ammann 0.97 and 0.12, Rondinelli 0.84 and 0.43. Assessing the bacteremia outcome, the values of 0.87 and 0.49 were found for Rackoff.

The variables ICU admission, use of more than two antibiotic agents, and death have not been directly and individually researched in previous studies, thus not allowing a comparison with the present study. Considering that the study sample shows a high frequency of infection, sensitivity and NPV can be considered to be the markers with the protocols’ best performance. The post-test probability in this circumstance would safely identify the patients who could be followed on an outpatient basis.14, 15 It is noteworthy that in the present study, all protocols had low PPV values in all outcomes. Of the evaluated protocols, three showed high sensitivity and NPV for the outcomes of death and ICU admission, emerging as the best current options: Rackoff, Alexander, and Ammann 2003.

The evaluation in relation to the microbiological infection outcome obtained better results for the Rackoff risk classification strategy, with indices similar to those found by Ammann in a comparative study, where the Rackoff protocol showed a sensitivity of 0.87, specificity of 0.49, a PPV of 0.41, and an NPV of 0.9. Considering the other three outcomes studied, Rackoff showed maximum performance values for death and ICU admission and a good performance for the need for more than two antibiotics. The use of one of these strategies would require an appropriate antibiotic regimen for outpatient follow-up, which would ensure therapeutic coverage for the most common etiological agents. The use of more than two antibiotics did not offer outstanding results and was not evaluated in any previous studies; therefore, there are no criteria for comparison, which may be the subject of future studies.

The use of a certain risk classification strategy should include not only the result of the data analysis, such as sensitivity, specificity, PPV/NPV, and graphical representation, but should also include an impartial comparison with the current reality. The hospital centers where these protocols are used refer to the maintenance of the same mortality rate verified before the implementation of the strategies, with numbers ranging from 0.7% to 4%.2, 3, 4, 5, 6, 8 At INCA, with the current admission procedure for all episodes of febrile neutropenia, a mortality rate of 0.8% was found. It is emphasized that the less restrictive criteria for hospitalization ensured the low recorded mortality, with only one death in the sample. The cost reduction and the release of hospital beds for other purposes, combined with the protection offered by the protocol itself, would justify the use of these protocols in clinical practice.

Study limitations

Considering the heterogeneity of the studied population in relation to the age group, underlying pathologies, and chemotherapy protocols, it is possible that the performance of a prospective study carried out over a longer period and with a larger sample might establish better performance indicators in relation to the protocols proposed by the literature. As in any retrospective study, the records of the obtained data were not controlled.

In conclusion, among the evaluated risk classification strategies, Rackoff showed the best performance in relation to microbiological infection, death, and admission to ICU, making it eligible for prospective evaluation. The practical confirmation of a risk classification strategy should also include factors not covered in this study, such as the social reality of each patient, hospital infrastructure, and the adequacy of an appropriate antibiotic regimen for outpatient follow-up. Regarding the use of a certain strategy and taking into account the institutional differences regarding the definition of the variables used, availability of treatment in the intensive care unit, laboratory resources, promptly performed imaging tests, easy access to the institution, regional, and demographic characteristics, and empirical antibiotic schemes, the authors suggest the necessity of local validation.

Conflicts of interest

The authors declare no conflicts of interest.

Rackoff	Alexander	Rondinelli	Santolaya	Ammann (2003)	Ammann (2010)
Absolute monocyte count	AML, Burkitt lymphoma, induction ALL, progressive disease, relapsed with marrow involvement Hypotension, tachypnea/hypoxia <94%, new CXR changes, altered mental status, severe mucositis, vomiting or abdominal pain, focal infection, other clinical reason for in-patient treatment	2 points for central venous catheter, 1 point for age ≤5 years,4.5 points for clinical site of infection,2.5 points for no URTI, 1 point each for fever >38.5 °C, hemoglobin ≤70 g/L	Relapsed leukemia, chemotherapy within 7 days of episodeCRP ≥90 mg/L, hypotension, platelets ≤50 g/L	Bone marrow involvement, central venous catheter, pre-B-cell leukemiaAbsence of clinical signs of viral infection, CRP >50 mg/L, white blood cell count ≤500/μL, hemoglobin >100 g/L	4 points for chemotherapy more intensive than ALL maintenance5 points for hemoglobin ≥90 g/L, 3 points each for white blood cell count <300/μL, platelet <50 g/L
Absolute monocyte count ≥ 100/μL = low risk of bacteremia*	Absence of any risk factor = low risk of serious medical complication*	Total score <6 = low risk of serious infectious complication*	Zero risk factors or only low platelets or only <7 days from chemotherapy = low risk of invasive bacterial infection*	Three or fewer risk factors = low risk of significant infection*	Total score <9 = low risk of adverse FN outcome*

AML, acute myeloid leukemia; ALL, acute lymphoblastic leukemia; URTI, upper respiratory tract infection; CXR, chest radiograph; CRP, C-reactive protein; FN, fever and neutropenia.

refers to clinically adequate discrimination of a group at low risk of complications.