Evaluation of the clinical Index of Stable febrile neutropenia risk stratification system for management of febrile neutropenia in gynecologic oncology patients.

OBJECTIVE: Scoring systems have been developed to identify low risk patients with febrile neutropenia (FN) who may be candidates for outpatient management. We sought to validate the predictive accuracy of the Clinical Index of Stable Febrile Neutropenia (CISNE) score alone and in conjunction with alternative scoring systems for risk of complications among gynecologic oncology patients.
METHODS: We conducted a single institution retrospective cohort study of patients admitted to an academic gynecologic oncology service for FN. We examined the performance characteristics (sensitivity, specificity, positive and negative predictive value) of three scoring systems (Multinational Association of Supportive Care in Cancer (MASCC), CISNE cut-off 1 (Low risk = 0), CISNE cut-off 2 (Low risk = <3)), and the combination of MASCC and CISNE to predict complications: inpatient death, ICU admission, hypotension, respiratory/renal failure, mental status change, cardiac failure, bleeding, and arrhythmia.
RESULTS: Fifty patients were identified for study inclusion. No low-risk CISNE patients died during hospitalization. Fewer CISNE low-risk patients experienced complications compared to high-risk patients, regardless of cut-off. Sensitivity, specificity, positive and negative predictive values of the scoring systems were: CISNE 1-37.1%, 86.7%, 86.7%, 37.1%; CISNE 2-85.7%, 46.7%, 78.9%, 58.3%; MASCC-82.9%, 66.7%, 85.3%, 62.5%; MASCC + CISNE 1-37.1%, 93.3%, 92.9%, 38.9%; MASCC + CISNE 2-80%, 73.3%, 87.5%, 61.1%.
CONCLUSIONS: The CISNE scoring system is an appropriate tool for the identification of patients with gynecologic cancers and FN who may benefit from close outpatient management. CISNE cut-off 2 performed comparably to the MASCC, but CISNE cut-off 1 had a higher specificity and positive predictive value.

Introduction

Febrile neutropenia (FN) is a significant source of morbidity and mortality for patients with solid tumors, with annual US hospitalization rates estimated between 60,000 and 100,000 patients, depending on cancer type and treatment (Caggiano et al., 2005, Weycker et al., 2014). For many years, the standard of care has been to admit all patients with FN to the hospital for close monitoring and treatment with intravenous antibiotics (Freifeld et al., 2011). FN hospitalization is a large burden for patients and the health system. Mean length of hospital stay for patients with FN varies by cancer type, but often lasts between 5 and 10 days (Caggiano et al., 2005, Pathak et al., 2015, Kawatkar et al., 2017) and costs up to $15,000-$20,000 per hospitalization (Kawatkar et al., 2017, Michels et al., 2012). FN may also delay chemotherapy treatments (Khan et al., 2008); routine hospitalizations for FN can expose already vulnerable patients to nosocomial multi-drug resistant infections and affect patients’ quality of life. In 2021, The Centers for Medicare & Medicaid Services (CMS) began to monitor admissions of patients receiving outpatient chemotherapy. CMS tracks patients admitted within 30 days of outpatient chemotherapy for certain diagnoses, including FN. Hospitals exceeding established benchmarks may incur a 2% reduction in payment.

Prior studies have demonstrated that cancer patients with FN can be risk-stratified using readily available clinical criteria (Elting et al., 1997) to identify candidates for outpatient management who are at low risk for severe complications (Klastersky et al., 2000). Such outpatient management has proven to be not only safe for low-risk patients (Caggiano et al., 2005, Pathak et al., 2015), but also cost-effective compared to traditional inpatient management (Teuffel et al., 2011). In 2018, the American Society of Clinical Oncology (ASCO) published guidelines supporting outpatient management of low-risk patients with FN. The Multinational Association of Supportive Care in Cancer (MASCC) score is one such clinical tool supported by ASCO to risk stratify patients with FN and has been previously studied in the gynecologic cancer population (Gunderson et al., 2019, Gunderson et al., 2013). While useful, studies indicate that the MASCC may have inadequate specificity (Carmona-Bayonas et al., 2011, Carmona-Bayonas et al., 2015, Carmona-Bayonas et al., 2017), and that an alternative classification method, the Clinical Index of Stable Febrile Neutropenia (CISNE) has a higher specificity in the determination of a cohort at low risk for complications. This may make the CISNE a more useful tool in cases of clinical uncertainty (Carmona-Bayonas et al., 2015, Zheng et al., 2020, Coyne et al., 2017). Unlike the MASCC, the CISNE scoring system has never been validated in patients with gynecologic malignancies and thus its utility in this particular patient population remains unknown. The aim of our study was to both determine if the CISNE scoring system can identify gynecologic oncology patients with FN who are at low-risk for complications, and to compare the diagnostic test characteristics of the MASCC and CISNE scores to predict complications in a cohort of gynecologic oncology patients admitted for FN.

Methods

We conducted a retrospective cohort study of patients with gynecologic malignancies who were admitted to a single academic institution’s hospital for FN between January 1, 2011 and April 1, 2020. The study received IRB exemption. Eligible gynecologic oncology patient records were identified using a search through an internal institutional data repository for the applicable ICD-9 and ICD-10 codes (288.0, 780.6, D70.0, R50.0). For patients who had more than one admission, only the first admission was included in the analysis. Patients were excluded if they developed FN during an admission for a different diagnosis or had inadequate information in their hospital records to calculate CISNE and MASCC scores. FN was defined as a single oral temperature of ≥ 38.3 °C (101°F) or a temperature of ≥ 38 °C (100.4°F) sustained over a one-hour period (Freifeld et al., 2011) in the presence of an absolute neutrophil count (ANC) of ≤ 1000. Existing literature in febrile neutropenia and gynecologic oncology populations used an ANC cut off of 1500 (Gunderson et al., 2019, Gunderson et al., 2013), however, ANC of 1000 was chosen for this study to reflect the evolving management landscape.

Demographic and clinical data, including age, race/ethnicity, smoking status, medical co-morbidities, immunosuppression status, cancer type, and brief treatment history, were abstracted from the charts. Admission information included admission dates; number, types and duration of antibiotic use; admission absolute neutrophil count; use of granulocyte colony stimulating factor (G-CSF); any chemotherapy delay; and complications due to FN. Given that the most clinically relevant risk of outpatient management is the development of any medical complication, the MASCC and CISNE scores were calculated to classify patients based on their risk of complications. Similar to the published derivation studies for the scores (Klastersky et al., 2000, Carmona-Bayonas et al., 2011), we considered the following complications: death in hospital, intensive care unit admission, hypotension, respiratory failure, renal failure, confusion or mental status change, congestive cardiac failure, bleeding requiring transfusion, or arrhythmia requiring treatment.

The MASCC score was retrospectively calculated for each patient using 7 characteristics: symptom severity, hypotension, history of COPD, history of fungal infection, dehydration, outpatient or inpatient status, and age (Klastersky et al., 2000). A score ≥ 21 was considered low risk for the development of any of the complications listed above, and therefore potentially eligible for outpatient management. CISNE scores were retrospectively calculated for each patient using 6 variables: Eastern Cooperative Oncology Group (ECOG) ≥ 2, COPD currently on therapy, chronic heart failure, mucositis grade ≥ 2, monocytes < 200 mm3, and stress hyperglycemia upon presentation (Carmona-Bayonas et al., 2011). CISNE considers a score of 0 (risk category I) to indicate a low risk for developing complications, a score of 1 or 2 intermediate risk (risk category II), and ≥ 3 high risk (risk category III). Table 1, Table 2 describe the variables included in the MASCC and CISNE scores. We similarly analyzed the ability of the CISNE scale to identify a cohort of patients at low risk for complications in our population utilizing two different cut-off values to determine which classification better predicts the development of complications in our patient population: CISNE I vs. CISNE II/III (CISNE cut-off 1) and CISNE I/II vs. CISNE III (CISNE cut-off 2).

Table 1

MASCC Scoring System.

Burden of illness	None or mild	+5
	Moderate	+3
	Severe	+0
Hypotension (sBP < 90 mmHg)	No	+5
Active COPD	No	+4
Type of cancer	Solid tumor or hematologic with no prior fungal infection	+4
Dehydration requiring IV fluids	No	+3
Status at onset of fever	Outpatient	+3
	Inpatient	+0
Age	<60 years	+2
	≥60 years	+0

sBP, systolic blood pressure; COPD, chronic obstructive pulmonary disease; IV, intravenous

Table 2

CISNE Scoring System.

ECOG performance status ≥ 2 (capable of all self-care, out of bed > 50 % day)	+2
Stress-induced hyperglycemia	+2
COPD on therapy	+1
Cardiovascular disease history	+1
NCI mucositis grade ≥ 2	+1
Monocytes < 200/µL	+1

ECOG, Eastern Cooperative Oncology Group; COPD, chronic obstructive pulmonary disease; NCI, National Cancer Institute

Descriptive statistics were performed on all demographic and oncologic variables. To compare characteristics and outcomes of low- and high-risk patients, categorical variables were compared using Fisher’s exact test and medians using the Wilcoxon rank-sum test. Statistical significance was defined as a p-value of < 0.05. We compared the low-risk cohort defined by CISNE scores to that of the low-risk cohort identified by the MASCC tool to identify differences in patients and outcomes.

Finally, we characterized the performance of the MASCC scale alongside that of the CISNE scale in identifying a low-risk cohort in our study cohort. Sensitivity was defined as the percentage of patients who were truly low-risk (did not experience complications), who were correctly identified as low-risk by the MASCC/CISNE scores. Specificity was the percentage of patients who were not truly low-risk (experienced complications), who were correctly identified as high-risk by MASCC/CISNE. Positive predictive value (PPV) was defined as the number of patients testing as low-risk who also did not develop complications (true positives) divided by the total number of patients testing as low-risk (test positives). Negative predictive value (NPV) was defined as the number of patients testing as high-risk who did develop complications (true negatives) divided by the total number of patients testing as high-risk (test negatives). ASCO guidelines suggest first using MASCC followed by CISNE cut-off 2 to assist in the identification of patients eligible for outpatient management (Taplitz et al., 2018). The performance characteristics of using the MASCC and CISNE in succession were also calculated, utilizing both CISNE cut-off 2 (as outlined by ASCO) and CISNE cut-off 1. The study team placed the highest priority on the identification of a low-risk cohort consisting of only patients who do not develop severe complications, and therefore defined the best scoring system as one that has a high positive predictive value and high specificity.

Results

Study subject characteristics

Fifty patients met study inclusion criteria. The average age was 60 years. As shown in Table 3, the most commonly represented cancers were ovarian (50% of patients), uterine (30% of patients), and cervical (12% of patients). Twelve patients (24%) had received radiation in addition to chemotherapy at the time of admission. Fifteen patients (30%) had a delay in their chemotherapy treatment as a result of their hospitalization for FN.

Table 3

Demographic characteristics of the study population.

Characteristic
Age in years	Mean ± SD	60 ± 15
Cancer type	n (%)
	Ovarian	25 (50%)
	Uterine	15 (30%)
	CervicalVulvarVaginal	6 (12%)2 (4%)1 (2%)
	Other/Unknown	1 (2%)
Race	n (%)
	White	33 (66%)
	Black	13 (26%)
	Asian	0 (0%)
	2 or more	2 (4%)
	Unknown	2 (4%)
Comorbidities	n (%)Diabetes	10 (20%)
	Hypertension	28 (56%)
	Obesity	11 (22%)
	COPD	4 (8%)
	Autoimmune disease	3 (4%)
Days since last chemo treatment	Median, range	9, 3–45
Length of hospital stay	Median, range	4, 0–44
Chemotherapy delays due to febrile neutropenia episode	n (%)	15 (30%)

Clinical outcomes by CISNE score

Fifteen patients (30%) were assigned a CISNE score of 0 (risk category I), 27 (54%) patients 1 or 2 (risk category II), and 8 patients (16%) 3 or above (risk category III). Table 4 shows patient outcomes for low- and high-risk groups using both CISNE cut-offs. None of the low-risk patients by either CISNE cut-off died during their FN admission, compared to 3 of the CISNE high-risk patients (8.6% of CISNE cut-off 1 high-risk, p = 0.547; 25% of CISNE cut-off 2 high-risk, p = 0.01). The CISNE cut-off 2 low-risk group had a significantly lower proportion of patients requiring ICU admission compared to the high-risk group (7.9% versus 33.3%, p = 0.048). Significantly fewer patients in the low-risk groups had multiple complications compared to the high risk-groups (0% cut-off 1 low-risk and 28.6% high-risk, p = 0.022; 10.5% cut-off 2 low-risk and 50% high-risk, p = 0.007). Fig. 1 depicts the number of patients experiencing complications and their respective CISNE score. Median length of stay was not significantly different between the low-risk and high-risk groups by CISNE cut-off 2 (4 versus 5 days, p = 0.726), but the difference was significant with CISNE cut-off 1 (3 versus 5 days, p = 0.021).

Table 4

Low- and high-risk patient outcomes by CISNE cut-off 1 and 2.

Characteristic	CISNE cut-off 1		p-value	CISNE cut-off 2		p-value
	Low-risk(N = 15)	High-risk(N = 35)		Low-risk(N = 38)	High-risk(N = 12)
Age in years	55	63		59	64
ICU admission, n (%)	0 (0%)	7 (20%)	0.087	3 (7.9%)	4 (33.3%)	0.048*
One admission complication, n (%)	2 (13.3%)	3 (8.6%)	0.629	4 (10.5%)	1 (8.3%)	1
Multiple complications (≥2), n (%)	0 (0%)	10 (28.6%)	0.022*	4 (10.5%)	6 (50%)	0.007*
Death in-hospital, n (%)	0 (0%)	3 (8.6%)	0.547	0 (0%)	3 (25%)	0.01*
Length of hospital stay, median, range	3, 1–20	5, 2–44	0.021*	4, 1–44	5, 2–20	0.726

Abbrev: ICU, intensive care unit; ANC, absolute neutrophil count; * indicates significance at p < 0.05

Complications include: hypotension, respiratory failure, renal failure, confusion/mental status change, congestive cardiac failure, bleeding requiring transfusion, or arrhythmia requiring treatment

Fig. 1

Patient complication rates by CISNE score. CISNE scores of our study population ranged from 0 to 5. The above graph shows the number of patients scoring each value who experienced at least one complication (solid bars) or no complications (striped bars) during admission. A CISNE score of 0 categorizes a patient into CISNE I, a score of 1 or 2 into CISNE II, and a score of 3 or above into CISNE III. Total N = 60 patients.

Clinical outcomes by MASCC score

Thirty-four patients (68%) received a MASCC score of ≥ 21 (low-risk). The average age of the low-risk group was 57 compared to 67 in the high-risk group. One of the MASCC low-risk patients and two of the high-risk died during their FN admission, p = 0.24. The low-risk group had a lower percentage of patients requiring ICU admission compared to the high-risk group (5.9% versus 31.3%, p = 0.124). The low-risk group also had a significantly lower incidence of any complication during admission (14.7% of MASCC low-risk; 62.5% of MASCC high-risk; p = 0.002). 8.8% of low-risk patients and 43.8% of high-risk patients had more than one complication during the admission (p = 0.007).

The cohort of low-risk MASCC patients and low-risk CISNE cut-off 2 patients were similar. 68% of patients were low-risk by MASCC (compared to 76% with CISNE cut-off 2). There were 2 patients who were in the high-risk category by CISNE cut-off 2, but were classified as low-risk by MASCC. One of these two patients had no admission complications, while the other required blood transfusion, ICU admission, and died in the hospital. Conversely, 6 patients identified as low-risk by CISNE cut-off 2 were considered high-risk by MASCC. None of these six patients died in hospital, but three experienced episodes of hypotension and two patients required ICU admission.

Clinical outcomes by combined scoring systems

When MASCC and CISNE cut-off 1 were utilized in succession, 14 patients meet low-risk criteria. No patients in this combined cohort died during admission or required ICU admission. One of these 14 patients (7.1%) experienced a single complication, and no patients experienced more than one complication. MASCC and CISNE cut-off 2 in succession identified 32 patients that met low-risk criteria. There were no in-hospital deaths in this cohort and just 1 ICU admission. Two low-risk patients (6.3%) had at least one complication compared to the 8 (44.4%) in the high-risk group (p = 0.002).

Performance characteristics of scoring systems

Table 5 lists the performance characteristics of each scoring system. These characteristics are: CISNE cut-off 1: sensitivity − 37.1%, specificity − 86.7%, PPV − 86.7%, NPV − 37.1%; CISNE cut-off 2: sensitivity − 85.7%, specificity − 46.7%, PPV − 78.9%, NPV − 58.3%; MASCC: sensitivity − 82.9%, specificity − 66.7%, PPV − 85.3%, NPV − 62.5%; MASCC + CISNE cut-off 1: sensitivity − 37.1%, specificity − 93.3%, PPV − 92.9%, NPV − 38.9%; MASCC + CISNE cut-off 2: sensitivity − 80%, specificity − 73.3%, PPV − 87.5%, NPV − 61.1%.

Table 5

Descriptive statistics for CISNE and MASCC identification of a cohort at low-risk for any complication.

Scoring system	Sensitivity(95% CI), %	Specificity(95% CI), %	PPV(95% CI), %	NPV(95% CI), %
CISNE cut-off 1	37.1 (21.5–55.1)	86.7 (59.5–98.3)	86.7 (62.5–96.2)	37.1 (30.0–44.9)
CISNE cut-off 2	85.7 (69.7–95.2)	46.7 (21.3–73.4)	78.9 (69.6–86.0)	58.3 (34.6–78.8)
MASCC	82.9 (66.4–93.4)	66.7 (38.4–88.2)	85.3 (73.6–92.3)	62.5 (42.5–79.0)
MASCC + CISNE cut-off 1	37.1 (21.5–55.1)	93.3 (68.1–99.8)	92.9 (65.1–98.9)	38.9 (32.3–45.9)
MASCC + CISNE cut-off 2	80 (63.1–91.6)	73.3 (44.9–92.2)	87.5 (74.9–94.3)	61.1 (43.1–76.5)

PPV, positive predictive value; NPV, negative predictive value; CI, confidence interval

Discussion

Our study demonstrates that the CISNE score may provide a useful addition to the MASCC in identifying a cohort of low-risk patients with gynecologic malignancies who may be candidates for outpatient management. We defined positive predictive value as the probability that patients meeting low-risk criteria truly will not have complications. Positive predictive value and high specificity (the probability of correctly identifying patients who do not ultimately develop complications), are the most important test characteristics for clinical decision making in the setting of FN. Conversely, since it is clinically acceptable to admit a patient to the hospital with FN who will not ultimately experience a serious complication, sensitivity and negative predictive value are less important.

In the current study, CISNE cut-off 2 had a similar performance in our patient population to that of the MASCC. However, CISNE cut-off 1 resulted in higher positive predictive value and specificity. While ASCO includes both MASCC and CISNE cut-off 2 in its guidelines, we obtained the highest specificity (93.3%) and PPV (92.9%) by instead combining MASCC with CISNE cut-off 1. The performance characteristics that we report for both the CISNE score cut-offs and the MASCC score are similar to those previously reported in the literature in non-gynecologic populations (Carmona-Bayonas et al., 2015, Zheng et al., 2020, Moon et al., 2018, Ahn et al., 2018, Koppaka et al., 2018).

Originally, ASCO guidelines recommended only the MASCC score (Flowers et al., 2013); in 2018 CISNE cut-off 2 was added as an additional tool (Taplitz et al., 2018). Patients with gynecologic malignancies have been shown to be at increased risk of FN compared to patients with other solid tumors, likely due to their complex, multi-modality treatment (Arslan et al., 2017, Klastersky et al., 2016). Unlike in other solid tumors, the presence of metastatic disease is a risk factor for FN in patients with gynecologic cancers (Smith et al., 2006, Aapro et al., 2011, Hashiguchi et al., 2015). The MASCC score has been investigated in this specific population (Gunderson et al., 2019, Uys et al., 2004) and found to be a promising risk stratification tool for determining suitability of outpatient management for gynecologic oncology patients with FN. However, data supporting use of the CISNE tool have not previously been presented in the gynecologic cancer population. The MASCC and CISNE scoring systems are often used in succession to determine appropriateness of outpatient management (Taplitz et al., 2018), so knowing that both scores and their combination are safe and effective for our patients is crucial.

In our cohort of 50 admissions for FN, none of the CISNE low-risk patients died in the hospital, regardless of which cut-off was used. None of the low-risk patients by CISNE cut-off 1 required ICU admission, although 3 of the low-risk patients by CISNE cut-off 2 did. In addition to a risk assessment scoring system, it is important that providers use clinical judgement to identify other high-risk details that may preclude safe outpatient management. For example, patients should not be discharged regardless of their MASCC or CISNE score if they are having syncope, nausea/vomiting, are unable to tolerate oral medications, have impaired renal or hepatic function, or if they are physically or medically frail. Two of the patients admitted to the ICU from the low-risk CISNE cut-off 2 group likely would not have been discharged upon presentation based on clinical instability.

There are several limitations of the study, some of which are inherent to retrospective analyses. First, we had a small sample size of only 50 admissions. The most important limitation is that none of the low-risk scoring patients were actually managed as outpatients. Although previous studies have argued that patients with low-risk FN can be treated with oral antibiotics as effectively as with intravenous therapy (Freifeld et al., 1999, Kern et al., 1999, Vidal et al.,2013), outcomes still may have been different had they been managed as outpatients. This study was designed to test the potential performance of the scoring systems, but their actual performance can only be truly tested when low-risk patients are identified and managed as outpatients. Based on current ASCO guidelines and our retrospective analysis of a larger institutional cohort that includes non-gynecologic cancers, we are now planning a feasibility project at our institution that utilizes the MASCC and CISNE cut-off 2 scoring systems to identify low risk patients for outpatient management of FN. Although our data shows better performance with MASCC and CISNE cut-off 1, this institutional project is based on analysis of a larger gynecologic and non-gynecologic cohort of cancer patients, thus CISNE cut-off 2 will be used in alignment with ASCO guidelines.

FN is a dangerous complication of cytotoxic therapy that is often also a burden on the hospital system, requiring patients to stay for several days and undergo treatment with costly antibiotics and other stabilizing measures (Tai et al., 2012). Our study demonstrates that the CISNE score has the potential to identify gynecologic oncology patients presenting with episodes of febrile neutropenia who are at low risk for complications. However, this tool needs to be piloted in a population of gynecologic patients who are managed as outpatients before further conclusions can be drawn about its clinical utility. CISNE cut-off 2 performed comparably to the MASCC score in this retrospective cohort study of hospitalized patients. A combination of the MASCC and CISNE cut-off 1 yielded the most favorable diagnostic characteristics in our gynecologic cancer cohort.

CRediT authorship contribution statement

Karen A. Monuszko: Conceptualization, Validation, Data curation, Formal analysis, Methodology, Project administration, Writing – original draft, Writing – review & editing. Benjamin Albright: Conceptualization, Methodology, Writing – review & editing. Mary Katherine Montes De Oca: Data curation. Nguyen Thao Thi Nguyen: Data curation. Laura J. Havrilesky: Writing – review & editing. Brittany A. Davidson: Conceptualization, Methodology, Investigation, Supervision, Project administration, Writing – review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.