The NEAT Predictive Model for Survival in Patients with Advanced Cancer.

PURPOSE: We previously developed a model to more accurately predict life expectancy for stage IV cancer patients referred to radiation oncology. The goals of this study are to validate this model and to compare competing published models.
MATERIALS AND METHODS: From May 2012 to March 2015, 280 consecutive patientswith stage IV cancerwere prospectively evaluated by a single radiation oncologist. Patients were separated into training, validation and combined sets. TheNEAT model evaluated number of active tumors ("N"), Eastern Cooperative Oncology Group performance status ("E"), albumin ("A") and primary tumor site ("T"). The Odette Cancer Center model validated performance status, bone only metastases and primary tumor site. The Harvard TEACHH model investigated primary tumor type, performance status, age, prior chemotherapy courses, liver metastases, and hospitalization within 3 months. Cox multivariable analyses and logisticalregressionwere utilized to compare model performance.
RESULTS: Number of active tumors, performance status, albumin, primary tumor site, prior hospitalizationwithin the last 3 months, and liver metastases predicted overall survival on uinvariate and multivariable analysis (p < 0.05 for all). The NEAT model separated patients into four prognostic groups with median survivals of 24.9, 14.8, 4.0, and 1.2 months, respectively (p < 0.001). The NEAT model had a C-index of 0.76 with a Nagelkerke's R2 of 0.54 suggesting good discrimination, calibration and total performance compared to competing prognostic models.
CONCLUSION: The NEAT model warrants further investigation as a clinically useful approach to predict survival in patients with stage IV cancer.

Introduction

Radiation therapy is a cost-effective and time efficient treatment to control pain, neurologic symptoms and obstructive symptoms from advanced cancer that can achieve local control of tumors in any organ system [1]. It is now well accepted that clinician intuition and experience alone is inadequate to accurately predict survival for stage IV cancer, with physicians typically overestimating survival [2-4]. There is significant interest in developing a more accurate model to predict survival for radiation oncology patients with stage IV cancer [5-7]. An accurate estimate of survival is important for patients to inform treatment decisions and to initiate endof-life planning and timely palliative care consultation [8]. Inaccurate survival prognostication can result in the patients receiving high intensity cancer care towards the end of life [9,10].

A model developed at Good Samaritan Hospital Medical Center separated patients with stage IV cancer into four prognostic groups using four prognostic factors that were found to be significant predictors of overall survival [5]. The acronym NEAT describes these factors: number of active tumors (“N”), Eastern Cooperative Oncology Group (ECOG) performance status (“E”), serum albumin (“A”), and primary tumor type (“T”). Four groups were identified with median survivals ranging from 1.2 months, 4.1 months, 14.5 months, and >31.4 months. The TEACHH model developed at Harvard Medical School divided incurable cancer patients into three life expectancy groups using type of cancer, performance status, age, prior palliative chemotherapy, prior hospitalizations, and hepatic metastases [7]. The three groups had median survivals of 1.7 months, 5 months, and 19.9 months, respectively. A third model developed at Odette Cancer Center in Toronto separated patients with advanced cancer into three groups using performance status, primary tumor type and site of metastases [6]. The three groups had survivals of 2.1 months, 6.0 months, and 13.8 months. Despite extensive research, there is no current consensus on the optimal approach to develop an accurate prognosis for patients with stage IV cancer.

The objectives of the current study were to (1) validate the results of the Good Samaritan model using a temporal validation set and (2) to evaluate the relative performance of the Good Samaritan model compared to the Odette Cancer Center and Harvard TEACHH models.

Materials and Methods

1. Inclusion criteria

This study included 280 consecutive patients older than 18 years with metastatic stage IV solid tumor who were referred to a single physician in a large community hospital-based radiation oncology department between May 2012 and March 2015. Since 45 patients did not have a recent serum albumin available, the analysis was limited to 235 patients. The first 116 analyzed patients were evaluated between May 2012 and August 2013 and represented the training cohort. The subsequent 119 analyzed patients were evaluated between September 2013 and March 2015 and represented the validation cohort.

2. Data collection

The charts of all patients were assessed for previously validated prognostic factors including performance status (ECOG 0 to 1 vs. ECOG 2 vs. ECOG 3-4), primary tumor type (breast, prostate, kidney, lung or other), number of tumors (1 to 5 vs. ≥ 6), serum albumin (≥ 3.4 g/dL vs. 2.4 to 3.3 g/dL vs. ≤ 2.4 g/dL), location of metastases (bone only vs. liver vs. other), age (> 60 vs. ≤ 60), hospitalization within the past 3 months (0 vs. ≥ 1), and prior palliative chemotherapy courses (≥ 2 vs. 0-1). The calculations of risk groups using the Good Samaritan NEAT, Odette Cancer Center, and Harvard TEACHH models are shown in Table 1.

Table 1.

Risk group calculations by previously identified models

Variable
NEAT model
No. of tumors	Points
1-5	0
> 5	1
ECOG performance
0-1	0
2	1
3-4	2
Albumin (g/dL)
≥ 3.4	0
2.4-3.3	0.5
< 2.4	1
Primary tumor site
Breast, kidney, prostate	0
Other	1
Composite score	Group
0-1	Very low risk
1.5-2	Low risk
2.5-3.5	Intermediate risk
4-5	High risk
Odette Cancer Center model	Risk factor score
Primary tumor site
Breast	0
Nonbreast	1
Metastasis location
Bone only	0
Other than bone only	1
KPS
> 60	0
≤ 60	1
No. of risk factors	Group
0 to 1	1
2	2
3	3
TEACHH model	Risk factor score
Primary tumor site
Breast, prostate	0
Lung, other	1
ECOG performance
0-1	0
2-4	1
Age (yr)
≤ 60	0
> 60	1
No. of prior palliative chemotherapy courses
0-2	0
> 2	1
Prior hospitalizations within the last 3 mo
No	0
Yes	1
Liver metastasis
No	0
Yes	1
No. of risk factors score	Group
0 to 1	A
2 to 4	B
5 to 6	C

ECOG, Eastern Cooperative Oncology Group; KPS, Karnofsky performance status.

3. Statistical methodology

Statistical analysis was performed with Stata ver. 13 (StataCorp., College Station, TX). The primary outcome was overall survival, defined as time from initial radiation oncology consultation to date of death. Patients who were lost to follow-up were censored at the date of last follow-up. Survival data was analyzed using the Kaplan-Meier method and summarized by median and 6-month survival. Chi-square tests were performed to determine differences in categorical variables between cohorts. Differences in survival were assessed through the log-rank method. The multivariable analyses were performed by Cox proportional hazards models and verified by tests of correlations over time with examination of residual plots. The ability of various models to predict 6-month survival was analyzed using logistical regression. Patients who were lost to follow-up with less than 6-month follow-up were excluded from the logistical regression. Statistical tests were two sided and p-values of < 0.05 were considered statistically significant.

The three prognostic models were assessed using the performance measures discrimination, calibration and overall performance [11]. Predictive discrimination is the probability of concordance between predicted and observed survival. Predictive discrimination is measured by the Harrell’s C index with 0.5-0.59 indicating poor, 0.6-0.69 indicating moderate, 0.7-0.79 indicating good, 0.8-0.89 indicating very good, and > 0.9 indicating excellent discrimination [12,13]. The C indices of various models were compared using the techniques described by Newson [14]. Calibration quantifies whether the observed outcomes and predicted probabilities agree. For calibration < 10% point difference between predicted and observed mortality is considered a well calibrated model [13]. Overall performance of various prediction models is the distance between predicted outcome and actual outcome and is measured by Nagelkerke’s R2 from a logistical regression [11].

4. Ethical statement

This minimal risk study was approved by the Good Samaritan Hospital Medical Center Institutional Review Board with waiver of informed consent.

Results

1. Training and temporal validation sets

The characteristics of the 116 patients in the training cohort and 119 in the validation are summarized in Table 2. There were differences between the populations with respect to primary tumor site, performance status, age and frequency of bone only metastases.

Table 2.

Patient characteristics

	Training set (n=116)	Validation set (n=119)	p-value
NEAT model
No. of tumors
1-5	32 (28)	34 (29)	0.867
≥ 6	84 (72)	85 (71)
ECOG performance
0-1	31 (27)	49 (41)	0.01
2	34 (29)	39 (33)
3-4	51 (44)	31 (26)
Albumin
≥ 3.4	54 (47)	58 (49)	0.479
2.4-3.3	52 (45)	46 (39)
< 2.4	10 (9)	15 (13)
Primary tumor site
Breast, kidney, prostate	16 (14)	33 (28)	< 0.001
Other	100 (86)	86 (72)
Odette Cancer Center model
Primary tumor site
Breast	7 (6)	19 (16)	0.015
Nonbreast	109 (94)	100 (84)
Metastasis location
Bone only	8 (7)	26 (22)	0.001
Other than bone only	108 (93)	93 (78)
KPS
> 60	31 (27)	49 (41)	0.019
≤ 60	85 (73)	70 (59)
TEACHH model
Primary tumor site
Breast, prostate	11 (9)	30 (25)	0.001
Lung, other	105 (91)	89 (75)
ECOG performance
0-1	31 (27)	49 (41)	0.019
2-4	85 (73)	70 (59)
Age (yr)
≤ 60	33 (28)	21 (18)	0.049
> 60	83 (72)	98 (82)
No. of prior palliative chemotherapy courses
0-2	103 (89)	106 (89)	0.945
> 2	13 (11)	13 (11)
Prior hospitalizations within the last 3 mo
0	28 (24)	38 (32)	0.184
≥ 1	88 (76)	81 (68)
Liver metastasis
No	93 (80)	92 (77)	0.592
Yes	23 (20)	27 (23)

Values are presented as number (%). ECOG, Eastern Cooperative Oncology Group; KPS, Karnofsky performance status.

Median survival was 4.1 months in the training cohort (95% confidence interval [CI], 3.0 to 5.8) and 6.7 months in the validation cohort (95% CI, 4.4 to 10.1) (p=0.07). The median overall survival for the entire cohort was 5.4 months (95% CI, 4.1 to 6.6) with a median follow-up for surviving patients of 20.7 months (95% CI, 8.5 to 26.1) (Fig. 1).

Fig. 1.

Cumulative survival curve of the combined cohort.

All four components of the NEAT model, including number of tumors, ECOG performance status, serum albumin and primary tumor site predicted survival on both univariate and Cox multivariable analysis (Table 3). For the Odette Cancer Center model, primary tumor site and Karnofsky performance status were predictive on univariate and Cox multivariable analysis but bone only metastasis was not predictive (Table 3). With respect to the TEACHH model, primary tumor site, ECOG performance status, prior hospitalization within the last 3 months and liver metastases predicted survival on both univariate and Cox multivariable analysis (Table 3). However, age and extent of prior palliative chemotherapy were not predictive.

Table 3.

Univariate and multivariate analysis of predictors of overall survival for each model (n=235)

	Univariate analysis	Multivariate analysis
	p-value	HR	95% Confidence interval	p-value
NEAT mode
No. of active tumors (1 to 5 vs. ≥ 6)	< 0.001	3.27	2.26 to 4.75	< 0.001
ECOG performance (0-1 vs. 2 vs. 3-4)	< 0.001	2.29	1.85 to 2.84	< 0.001
Serum albumin (≥ 3.4 vs. 2.4-3.3 vs. < 2.4)	< 0.001	1.59	1.28 to 1.98	< 0.001
Primary tumor site (breast, kidney, or prostate vs. other)	0.004	3.10	2.06 to 4.68	< 0.001
Odette Cancer Center model
Primary tumor site (breast vs. nonbreast)	0.004	1.90	1.13 to 3.19	0.015
Metastasis location (bone only vs. other)	0.147	1.15	0.75 to 1.76	0.523
KPS (> 60 vs. ≤ 60)	< 0.001	3.22	2.32 to 4.45	< 0.001
TEACHH model
Primary tumor site (breast and prostate vs. lung and other)	0.025	2.00	1.28 to 3.14	0.002
ECOG performance (0-1 vs. 2-4)	< 0.001	2.91	2.07 to 4.09	< 0.001
Age (≤ 60 yr vs. > 60 yr)	0.160	1.17	0.83 to 1.66	0.374
No. of prior palliative chemotherapy courses (0-1 vs. ≥ 2)	0.423	1.46	0.91 to 2.34	0.114
Prior hospitalizations within the last 3 mo (0 vs. ≥ 1)	< 0.001	1.46	1.02 to 2.08	0.038
Liver metastasis (no vs. yes)	< 0.001	1.49	1.04 to 2.12	0.028

HR, hazard ratio; ECOG, Eastern Cooperative Oncology Group; KPS, Karnofsky performance status.

2. Training cohort

In the training group, Good Samaritan NEAT model divided the patients into four distinct groups with median survivals of 37.5, 14.8, 4.0, and 1.2 months, respectively (p < 0.01). The Odette Cancer Center model separated the patients into three groups with median survivals of 37.5, 12.7, and 2.2 months (p < 0.01). The Harvard TEACHH model divided the patients into three groups with median survivals of 25.0, 4.8, and 1.2 months (p < 0.01). In all cases, there was good visual agreement between the estimated survival from the Cox model compared to actual outcome estimated by the Kaplan-Meier method (Fig. 2A-C).

Fig. 2.

Survival probability based on groups according to the NEAT model (A), Odette Cancer Center model (B), and TEACHH model (C) developed from the training cohort. Survival probability based on groups according to the NEAT model (D), Odette Cancer Center model (E), and TEACHH model (F) developed from the validation cohort. Survival estimated using the Cox’s model (dashed lines) and the actual survival calculated by the Kaplan-Meier method (solid lines) are shown.

3. Temporal validation cohort

In the temporal validation cohort, the Good Samaritan NEAT model identified four distinct groups of patients with median survivals of 19.7, 14.9, 4.3, and 0.9 months (p < 0.01). The Odette Cancer Center model separated patients into groups with median survivals of 10.4, 9.2, and 3.0 months (p < 0.01). The Harvard TEACHH model divided patients into three groups with median survivals of 16.5, 6.8, and 1.8 months (p < 0.01). There remained good visual agreement between predicted and observed survival in all three models (Fig. 2D-F).

4. Combined analysis of all patients

In the combined dataset, the NEAT model separated patients into four prognostic groups with median survivals of 24.9, 14.8, 4.0, and 1.2 months, respectively (Fig. 3A). The Odette Cancer Center model divided patients into three prognostic groups with median survivals of 16.4, 12.4, and 2.6 months, respectively (Fig. 3B). The Harvard TEACHH model identified cohorts with median survivals of 25.0, 5.7, and 1.6 months, respectively (Fig. 3C).

Fig. 3.

Overall survival based on groups according to the NEAT model (A), Odette Cancer Center model (B), and TEACHH model (C) developed from the combined cohort.

5. Comparing performance of models developed at Good Samaritan, Odette Cancer Center, and Harvard

The C indices for the Good Samaritan NEAT model were 0.76, 0.75, and 0.76 for the training, validation and combined sets (Table 4). The corresponding C indices for the Odette Cancer Center model were 0.67, 0.64, and 0.66, respectively for the training, validation and combined groups. Finally, the C indices for the Harvard TEACHH model were 0.59, 0.60, and 0.60, respectively for the training, validation and combined groups. In all groups, the C indices were significantly higher for the Good Samaritan NEAT model than the Odette Cancer Center or Harvard TEACHH models (p < 0.01 for all).

Table 4.

Comparing model performance

	6-Month mortality (95% confidence interval)
	Training (n=116)	Validation (n=119)	Combined (n=235)
NEAT risk group
Very low risk	0 (0-21)	14 (3-34)	8 (2-21)
Low risk	18 (4-43)	15 (4-34)	16 (7-30)
Intermediate risk	70 (54-83)	70 (53-83)	70 (59-79)
High risk	95 (83-99)	90 (68-99)	93 (84-98)
C index	0.76	0.75	0.76
Nagelkerke R2	0.61	0.45	0.54
Odette Cancer Center risk group
1	0 (0-52)	22 (6-48)	17 (5-39)
2	31 (16-48)	33 (20-48)	32 (22-43)
3	80 (69-88)	77 (61-88)	79 (70-86)
C index	0.67	0.64	0.66
Nagelkerke R2	0.35	0.24	0.30
TEACHH risk group
A	14 (0-58)	10 (0-44)	12 (1-36)
B	60 (49-70)	46 (35-58)	54 (46-61)
C	88 (62-98)	68 (59-98)	87 (70-96)
C index	0.59	0.60	0.60
Nagelkerke R2	0.13	0.18	0.16

The calibration of the Good Samaritan NEAT model was good, with close agreement between the observed 6-month mortality rates in the low risk, intermediate risk, and high risk groups. The Odette Cancer Center model had comparable mortality rates for risk groups 2 and 3. The TEACHH model showed similar mortality rates for risk group A.

Finally, in the combined group the Nagelkerke’s R2 for predicting 6-month survival was 0.54 for the Good Samaritan NEAT model compared to 0.30 for the Odette Cancer Center model and 0.16 for the Harvard TEACHH model.

Discussion

Approximately 20% to 40% of patients treated in radiation oncology departments have metastatic cancer requiring radiotherapy [7]. Patients with stage IV cancer referred to radiation oncology are heterogeneous ranging from the actively dying with survival measured in days to patients with asymptomatic oligometastases who may achieve longterm remission [15-17]. We previously developed a robust model that separated patients into four distinct groups with markedly different life expectancies [5]. In the training, validation and combined cohorts, the Good Samaritan NEAT model performed well compared to previously published models evaluating radiation oncology patients with metastatic cancer.

There is significant focus in the literature on preventing overtreatment of poor prognosis patients [18]. Conversely, accurate identification of better prognosis patients may prompt the treating physician to consider treatment intensification to improve disease-free survival and even overall survival [19-21]. Broader use of predictive models may allow for personalized treatment consistent with patient’s needs, values and preferences [1].

There is a robust literature on estimating life expectancy for terminally ill cancer patients seen in inpatient medical oncology, palliative care and hospice units who were no longer candidates for treatment [7,22]. Site-specific prognostic factors for brain metastases, spinal cord compression and bone metastases are particularly useful for subspecialists with practice focused on a specific organ site [23-25]. For the practicing general radiation oncologist, the ideal prognostic model would account for both poor and favorable prognosis patients in multiple clinical contexts over diverse cancer types [1]. The Good Samaritan NEAT model accounts for tumor histology and tumor burden to accurately predict survival for patients with longer life expectancy while strongly weighting both performance status and albumin for patients with short life expectancy.

While performance status and primary tumor are also included in other radiation oncology predictive models, there is significant divergence in other prognostic factors. After analyzing 29 prognostic factors, the Good Samaritan NEAT model identified serum albumin and number of active tumors as novel predictors of survival in radiation oncology patients [5]. For oncology patients, albumin is an objective measure of anorexia-cachexia syndrome and malnutrition that is widely available on comprehensive metabolic panels [26]. Additionally, hypoalbuminemia can also reflect physiologic causes of third spacing, inflammation and sepsis that may be associated with poor prognosis. Oligometastases are defined as one to five distant metastases that are potentially amenable to involved site radiation to all areas of known disease [16,17,27]. Although the exact number of dominant masses and pathologic lymph nodes is not always specified on radiology reports, radiation oncologists have extensive experience with reviewing recent computed tomography, magnetic resonance imaging, positron emission tomography, and bone scans and can readily quantify number of tumors through electronic medical records and picture archiving and communication systems. Interestingly, performance status, extent of active disease and intake are key elements of the Palliative Performance Scale, which is widely used in palliative medicine to predict survival [28]. The NEAT model further refines and extends the Palliative Performance Scale by quantifying the relative contribution of performance status, extent of active disease, intake and primary tumor site on prognosis.

In community practice, the Good Samaritan NEAT model appeared to have a higher C-index and Nagelkerke’s R2 than the Odette Cancer Center and Harvard TEACHH models. In this study, differences in model performance were driven by variables other than performance status and primary tumor site (Table 3). The NEAT model included number of tumors and albumin as variables that strongly predicted survival on multivariable analysis. The TEACHH model included age and extent of prior palliative chemotherapy and the Odette Cancer Center included bone only metastases that were not statistically significant on multivariable analysis.

The Good Samaritan NEAT model was developed in the community hospital setting and may not be generalizable to major academic medical centers where the Odette Cancer Center and Harvard TEACCH models were developed. The Odette Cancer Center study included 33% of patients with bone only metastases compared to 14% in the current dataset. While these clearly reflect differences in practice patterns the causes is unclear. One might speculate that 64% of the patients in the Odette Cancer Center study were patients from a novel Rapid Response Radiotherapy Program treated in 1999 to 2000 that affected referral patterns to increase utilization of palliative radiotherapy for bone metastases that would otherwise be treated with systemic therapy alone in other settings. The Harvard TEACHH dataset included 45% of patients who were 60 years of age or younger compared to 23% in the current dataset. The Harvard TEACHH patient population included 17% patients that were treated with > 2 prior palliative chemotherapy courses compared to 11% in this dataset. From these data, one can infer that patients in community practice tend to be older and may be less likely to receive greater than second-line chemotherapy than a major academic center with an extensive cancer clinical trial program. Further development of the NEAT model and other predictive models for stage IV cancer require external validation in diverse community and academic settings.

A limitation of the NEAT model is that a small minority of stage IV patients do not have recent albumin or extent of disease imaging available. If these tests have not been ordered, patients can still be evaluated with alternate models. Finally, currently available predictive models can only explain a modest percentage of observed survival due to the effect of unmeasured patient-specific prognostic factors. In the future, biomarkers and measures of antitumor immunity will supplement currently available clinical, radiologic, laboratory and pathologic prognostic factors to further refine existing models [29,30]. Although the accuracy of experienced clinicians in making unsupervised predictions in cancer is poor, the role of clinical intuition to further improve algorithmdefined predictions in a complex system such as oncology is worthy of investigation for both clinicians and data scientists.