Validation and extension of the METSSS score in a metastatic cancer patient cohort after palliative radiotherapy within the last phase of life.

Introduction and background: Choosing the right treatment for the right patient in a setting of metastatic cancer disease remains a challenge. To facilitate clinical decision-making, predictive tools have been developed to personalize treatment. Here, we aim to assess the use of the recently proposed "METSSS score" as a prognostic tool for overall survival of cancer patients after palliative radiotherapy in the last phase of life.
Methods: All patients treated with palliative radiotherapy at the end-of-life at the Department of Radiation Oncology of the University Hospital Zurich between January 2010 and December 2019 were included in this study. Data on demographics, diagnosis, treatment and comorbidities was extracted from the treatment planning and the electronical medical records system. To statistically assess the validity of the "METSSS score", the mortality risk score was calculated, followed by stratification of all patients to prognostic risk groups. The prediction of the 1-year overall survival estimates was subsequently calculated.
Results: Over the past decade, 274 patients have received palliative radiotherapy during the end-of-life period. One third of patients was female (34%, n = 93). The most frequent primary tumor was lung cancer (n = 121, 44%), and 55% of patients (n = 152) had no comorbidities according to the Charlson-Deyo comorbidity index. The most common radiotherapy site was the brain and eye region (42%, n = 115). The median actual overall survival of all patients was 40 days from the start of radiotherapy. The "METSSS score" survival model predicted that 269 patients (98.1%) belong into the high-risk, four patients (1.5%) into the medium-risk, and one patient (0.4%) into the low-risk group. The predicted median 1-year overall survival was 10%. Discussion: The METSSS score correctly predicted the survival of our end-of-life patient cohort by assigning them into the highest risk category, and it can therefore serve as a decision-making tool when assigning patient to symptomatic radiotherapy.

Introduction and background

While it is well documented that radiotherapy (RT) is a very effective treatment modality for patients with symptomatic metastatic cancer, prediction of survival after palliative RT remains difficult for treating physicians [1], [2], [3], [4]. Yet it is very important for radiation oncologists to predict survival with reasonable accuracy, as prognostic awareness influences decision-making and therapeutic choice for physicians, patients and dependents alike. Despite the need to improve survival prognostication in patients qualifying for palliative RT, no single, clinically useful and externally validated prediction tool has emerged so far. However, over the recent past, several authors have proposed models to estimate survival after palliative RT.

In 2002, Chow et al. developed a predictive model by examining 16 covariates in a cohort of 395 patients qualifying for palliative RT. Primary cancer site, site of metastasis, Karnofsky performance score (KPS), fatigue, appetite, and shortness of breath proved to carry prognostic value [5]. More than ten years later, in 2014, Krishnan et al. published the “TEACHH model” to predict life expectancy in patients with metastatic cancer receiving palliative RT. By retrospectively reviewing and analyzing 862 patients, the authors found type of cancer, Eastern Cooperative Oncology Group performance status (ECOG-PS), age, prior palliative chemotherapy, prior hospitalizations, and hepatic metastasis (“TEACHH”) to be significantly associated with shorter survival. These six risk factors were used to stratify the patient cohort into three patient groups with a different prognosis [6]. And more recently, in 2020, Zaorsky et al. expanded on these previous attempts of developing prediction models by proposing the “METSSS score” for use in patients receiving palliative RT during the initial course of treatment. This model was developed based on the treatments of 68,505 metastatic cancer patients, and it also found six variables to have predictive power. Metastasis location, elderly (age), tumor primary, sex, sickness/comorbidity, and site of radiotherapy (“METSSS”) were included into the predictive nomogram, and the authors also used these predictors to stratify patients into three risk groups with varying prognosis [7].

With the prospect of personalized medicine on the horizon, a wealth of prediction models has been proposed to tailor treatment to the specific circumstances and characteristics of each patient. Such models are only rarely externally validated or extended to similar patient populations, independent of the medical sub-specialty [8]. However, external validation and/or model extensions are important quality measures for prediction models. This is especially relevant, as such models are known to perform better in the patient cohort, which they were developed in. Yet to be practically relevant, assessment in other patient cohorts is important, which may be slightly different in terms of patient characteristics or inclusion criteria [9]. Hence, the real clinical value of a predictive model is to be determined by conducting comprehensive validation or extension studies before they can be implemented in practice [8]. To this end, we partially validate and assess the extension of the most recently compiled, most comprehensive and robust predictive tool for palliative RT. Having been developed in a very large number of patients, the METSSS score fulfills these criteria. Here, we aimed to extend its use to a cohort of metastatic cancer patients having received palliative RT in the last phase of life. We had previously reported on the prevalence, indications and outcomes of palliative RT in this patient cohort in a single-institution analysis [10].

Materials and methods

METSSS score

The METSSS score was compiled to consist of six covariates, which are either binary, categorical or linear. Metastasis location (“M”) included bone, brain, liver and lung, and each was classified as either “present”, “absent” or “unknown”. Age (“E”) was modeled as a linear variable, yet in the nomogram, only the 10-year age cut points were depicted. Primary tumor location (“T”) was defined to include prostate, breast, lungs, and others. Sex (“S”) of the patient was taken to be a binary variable, male or female. Comorbid conditions (“S”) were captured via an adjusted version of the Charlson-Deyo comorbidity index (CCI). The CCI was obtained from the American National Cancer Data Base (NCDB), with scores ranging from “0” (no classified comorbid conditions) to “3+” (highest number of combined comorbid conditions). Site of radiotherapy (“S”) had nine categories: (1) brain and eye, (2) head and neck, skin, and thyroid, (3) thorax, (4) stomach, liver, pancreas, kidney, abdomen, (5) breast, (6) bone, (7) soft tissue, (8) pelvis, and (9) others. Univariate and multivariate Cox proportional hazards regression models were employed to assess the impact of the selected risk factors on mortality risk to come up with clinical predictors. Subsequently, the clinical risk factors were used to come up with a regression equation to calculate the mortality risk score (MRS), where higher values indicate higher mortality risk. The authors then used tertile cut-off points to divide patients into low-, medium-, and high-risk groups [7].

Patient cohort

All patients who received RT within 60 days of death at the Department of Radiation Oncology of the University Hospital Zurich (USZ) between January 2010 and December 2019 were screened for this study. Inclusion and exclusion criteria from the original METSSS publication were largely applied for the purposes of this external validation study. All patients (1) were adults, (2) had a histologically confirmed metastatic cancer diagnosis, and (3) received a palliative intent course of RT. While patients with curative intent RT in the last phase of life were excluded from the analysis, concomitant palliative chemotherapy was not an exclusion criterion. With respect to three aspects, we allowed for a difference in inclusion criteria in comparison to the original METSSS publication: (a) RT courses needed not to be part of the initial treatment regimen of patients, (b) RT courses needed not to be completed for patients to be included in the analysis, and (c) RT regimens other than 10x3Gy, 5x4Gy or 1x8Gy, which were prescribed with a clear palliative intent, were included. Patient demographics and treatment data, including data on the six METSSS score covariates, were extracted from the treatment planning system (TPS) ARIA® and the electronical medical records (EMR) system KISIM®. Data on fractionation and dosage schedules as well as treatment sites were obtained from the TPS. Data on metastasis location, age, year of diagnosis, primary tumor histology, sex, CCI, and date of death were retrieved from the EMR. This study, as part of a project series on palliative treatments at the end-of-life, was approved by the Swiss Cantonal Ethics Committee (BASEC ID #2019-02488). This study also followed the TRIPOD checklist for model validation (see Supplements 1).

Data streamlining and statistical analysis

Upon extraction of the data into the spreadsheet software Microsoft® Excel® (Version 16.0), patient data was encoded and the six METSSS covariates were categorized as per the original METSSS publication. Data for all six covariates for every single patient were available, so there were no missing data values. Subsequently, the following regression equation was employed to calculate the MRS for each patient:where the six factors represent the METSSS covariates and constitutes the different coefficients. The values of the coefficients and the standard errors were provided to us by the authors of the original METSSS score publication. With their permission, they are displayed in the Supplements 2 of this paper. Based on the tertile cut-off points provided in the original METSSS publication (low risk < –0.122; medium risk: –0.122 ≤ X ≤ 0.242; high risk: >0.242), patients were stratified into their respective risk group. Analogously to the original METSSS publication, the 1-year overall survival (OS) estimates were calculated. All statistical analyses were conducted using the statistical software package STATA® (Version 16.1). Figures were compiled using the visualization software package GraphPad PRISM® (Version 8).

Results

Clinical patient characteristics

For the 274 patients having received palliative end-of-life RT, the mean age at therapy was 65 years (standard deviation (SD), 13). More than half of the patients, 55%, had a CCI of 0 (n = 152), while 56 (20%), 37 (14%), and 29 (11%) patients had a CCI of 1, 2 and 3 + points, respectively. The most common primary cancer was lung (n = 121, 44%), with the remaining 143 (52%) patients having other malignancies. Five patients (2%) had prostate cancer, and five patients (2%) had breast cancer. Bone metastasis was present in 73 (27%) patients. Nineteen percent of patients (n = 51) had brain metastasis. The large majority of patients, 71% (n = 194), did not have lung metastasis, while a little less than a third of patients (29%, n = 80) had diagnosed lung metastasis. Two thirds of patients (66%, n = 181) were male, one third female (34%, n = 93). The most common RT site was the brain and eye region (42%, n = 115). The second and third most common RT sites were bone (25%, n = 68) and thorax (14%, n = 39), respectively. RT to other sites was much less common. Soft tissue was the RT site in 22 patients (8%), and ten patients (4%) were irradiated in the stomach, liver, pancreas, kidney, and abdomen region. Eight patients (3%) each received RT to the head and neck, skin and thyroid region as well as to otherwise not classified sites. Two patients (1%) each received RT to the breast and pelvis region. RT intent was palliative for all patients and treatment indication included pain, dyspnea, bleeding, amongst others [10]. For an overview of patient characteristics according to the METSSS nomenclature, see Table 1. Analogously to the original METSSS publication, selected patient characteristics are also displayed in graphical form to facilitate comparison with the original METSSS publication (see Fig. 1).

Table 1

Summary of clinical patient characteristics (own cohort; METSSS cohort [7]).

Data variables	n = 274	n = 68,505*
Age in years, mean (SD)	65 (13)	66 (12)
CCI (#), n (%)
• 0	152 (55)	45,798 (67)
• 1	56 (20)	15,783 (23)
• 2	37 (14)	4,878 (7)
• 3+	29 (11)	2,046 (3)
Primary tumor histology, n (%)
• Prostate	5 (2)	2,789 (4)
• Breast	5 (2)	4,364 (6)
• Lung	121 (44)	43,746 (64)
• Other	143 (52)	17,606 (26)
Bone metastasis
• No	201 (73)	24,298 (36)
• Yes	73 (27)	29,253 (43)
• Others/unknown	0 (0)	14,954 (22)
Brain metastasis
• No	223 (81)	33,905 (50)
• Yes	51 (19)	19,470 (29)
• Others/unknown	0 (0)	15,130 (22)
Liver metastasis
• No	228 (83)	42,225 (62)
• Yes	46 (17)	10,922 (16)
• Others/unknown	0 (0)	15,358 (22)
Lung metastasis
• No	194 (71)	40,537 (60)
• Yes	80 (29)	12,086 (18)
• Others/unknown	0 (0)	15,882 (23)
Sex, n (%)
• Male	181 (66%)	31,008 (46)
• Female	93 (34%)	37,497 (55)
Radiotherapy site
• Brain and eye	115 (42)	22,027 (32)
• Head and neck, skin, thyroid	8 (3)	967 (1)
• Thorax	39 (14)	11,190 (16)
• Stomach, liver, pancreas, kidney, abdomen	10 (4)	1,234 (2)
• Breast	2 (1)	965 (1)
• Bone	68 (25)	30,329 (44)
• Soft tissue	22 (8)	219 (0.3)
• Pelvis	2 (1)	396 (0.6)
• Others	8 (3)	1,178 (2)

Abbreviations: CCI = Charlson-Deyo comorbidity index; SD = Standard deviation.

*For comparative purposes, this table as set up analogously to the one in the original METSSS publication [7].

Fig. 1

Overview of radiotherapy sites (# of patients). Abbreviations: GIT = Gastrointestinal tract; HNC = Head and neck cancer; HP = Hepatobiliary. Note: *Stomach, liver, pancreas, kidney, abdomen.

Comparison of patient cohorts

When compared to the original METSSS patient cohort, the mean age and SD in the present cohort were similar (66; 12 [METSSS] vs. 65; 13 [present cohort]). Due to the much larger sample size of the METSSS patient cohort, the gender split was more even in METSSS patient cohort when compared to the present patient cohort (M: 55%, F: 45% [METSSS] vs. M: 66%, F: 34% [present cohort]). In terms of comorbid conditions, the present patient cohort was slightly sicker than the METSSS patient cohort, which is exemplified by the higher CCI scores: While 67% and 23% of METSSS patients had a CCI of 0 or 1, only 55% and 20% did do so in the present patient cohort, respectively. Consequently, the proportion of patients with a CCI of 2 or 3 + was higher in the present cohort than in the METSSS patient cohort (CCI 2: 7% [METSSS] vs. 14% [present cohort]; CCI 3+: 3% [METSSS] vs. 11% [present cohort].

Further differences between the patient cohorts exist when looking at the patient distribution with respect to the tumor primary. While the largest proportion of primary tumor was lung cancer in the METSSS patient cohort (64%), it was other primary cancers in the present patient cohort (52%). Other tumor primaries was the second most common primary cancer in the METSSS patient cohort (26%), whereas a lung primary was the second most common primary cancer in the present patient cohort (44%). In both patient cohorts, the proportions of the most prevalent primary tumors in women and men, breast and prostate, were small (breast: 6% [METSSS] vs. 2% [present cohort]; prostate: 4% [METSSS] vs. 2% [present cohort]). With respect to metastatic disease status, data was available for all patients of the present patient cohort in contrast to the METSSS patient cohort. While the proportions of patients in the METSSS patient cohort with bone and brain metastasis were higher than in present patient cohort (bone: 43% [METSSS] vs. 27% [present cohort]; brain: 28% [METSSS] vs. 19% [present cohort]), the proportions of patients with liver and lung metastasis were higher in the present patient cohort than in the METSSS patient cohort (liver: 16% [METSSS] vs. 17% [present cohort]; lung: 18% [METSSS] vs. 29% [present cohort]). In both patient cohorts, the same three anatomical regions were the most common RT sites (brain and eye, bone, and thorax) [7]. Overall, when all METSSS variables are taken into consideration, the present patient cohort was slightly sicker and had a slightly more advanced metastatic tumor state than the METSSS patient cohort at the time when patients were prescribed palliative RT. Refer to Table 1 for a comparative overview of both cohorts.

Mortality risk scores and overall survival predictions

The median actual OS of all patients included in this study was 40 days (IQR, 25–56 days), after palliative RT. According to the METSSS regression equation, 269 patients (98.1%) had a MRS > 0.242, having them fall into the high-risk group. The MRS of four patients (1.5%) was between –0.122 and 0.242, placing them into the medium-risk group. And one patient (0.4%), with a MRS smaller than –0.122, fell into the low-risk group. The predicted median 1-year OS was 0.1 (interquartile range (IQR), 0.0–0.3). For an overview of outcome statistics, see Table 2 and Fig. 2.

Table 2

Summary of outcome statistics.

Data variables	Patients (n = 274)
Actual OS (days), median (IQR)	40 (25–56)
MRS, n (%)
• >0.242	269 (98.1)
• –0.122 and 0.242	4 (1.5)
• <–0.122	1 (0.4)
Predicted 1-year OS, median (IQR)	0.1 (0.0–0.3)
Predicted 5-year OS, median (IQR)	0.0 (0.0–0.0)

Abbreviations: IQR = Interquartile range; MRS = Mortality risk score; OS = Overall survival.

Fig. 2

Overview of risk group prediction by METSSS algorithm (# of patients, %).

Discussion

With regard to mortality risk and survival predictions, the METSSS algorithm yielded MRSs, which slotted the overwhelming majority of patients, namely 98.1%, in the high-risk group. An allotment into the high-risk group indicates a high mortality risk when compared with patients who are scored into the medium- and low-risk groups. The median OS of a patient in the high-risk group in the METSSS patient cohort was 3.28 months, highlighting that half of the patients did not live substantially longer than 120 days [7]. Given the median actual OS in the present patient cohort was just 40 days, with the longest period a patient lived after palliative RT being 56 days, i.e., barely one and a half months, goes to show that the risk group of these patients was rightly predicted. In the TEACHH model, which also stratified patients after palliative RT into three risk groups, patients in the high-risk group also had an OS of 1.7 months only [6]. For the remaining 1.9% of patients, the METSSS algorithm yielded a wrong estimation, as they were predicted to belong into the medium- and low-risk groups. Yet while patients in the medium-risk and high-risk groups in the original METSSS patient cohort had a median OS of 5.09 and 11.66 months, respectively, the five patients in question died in between 17 and 41 days after palliative RT. The METSSS model should have also predicted these patients to belong into the high-risk group. Given all patients in the present patient cohort belong into the high-risk group, it is not surprising that the 1-year OS estimate is a mere 10%.

One important focus of our analysis consisted in answering the question whether the use of the METSSS score could be extended to rightly predict MRS and OS in patients after palliative RT at the end-of-life. In doing so, it became clear that the METSSS model correctly identified high-risk group vs. other risk group (medium- and low risk) patients after palliative RT at the end-of-life. However, by study design, the findings of our analysis cannot serve the purpose to conclusively settle the question on how good the METSSS model’s ability is to discriminate between predictions for patients with a medium- and low-risk profile.

Even so the METSSS score yielded very good results both in internal and external validation exercises, one striking difference when comparing it to the Chow et al. (2002) or the TEACHH model is the lack of accounting for patient’s performance status. One might hypothesize that, given KPS and ECOG-PS are two good prognostic markers, they might further enhance the predictive power of the METSSS score. Nonetheless, the METSSS score remains not only the most recently developed model, but also the one which was developed in the largest patient cohort by far, adding to its discriminative capacity and forecasting power. Future comprehensive external validation efforts should therefore continue to focus on the METSSS and might consider including either KPS, like the model of Chow et al. (2020), or ECOG-PS, such as the TEACHH model. This might indeed add to the usability of the score, especially in the palliative setting.

It is a strength of this study to have extended the use of the METSSS model to a cohort of patients which differed in terms of inclusion criteria from the original METSSS publication. More specifically, the two patient cohorts differed on three fronts: (a) the RT being part of the initial treatment regimen, (b) the completion of palliative RT, and (c) the fractionation scheme. The major limitation of this study lies in its small sample size. Ideally, external validation or extension studies would be conducted in series of tens of thousands of patients. However, the facts that databases of that size on cancer patients having received palliative RT are a rarity and that, to this date, no validation study or extension proposal of the METSSS score has been published, elevate the importance of smaller studies attempting such endeavors.

In conclusion, in this small group of metastatic cancer patients having received palliative RT during the last phase of life, the METSSS score correctly predicted the risk group for the large majority of patients. Based on this encouraging result, the authors suggest to conduct further, especially larger external validation or extension studies of the METSSS score, to ascertain that the predictive tool can be employed for palliative RT treatments during the entire disease journey of a patient. Once such studies exist, the clinical practicability and usefulness of the METSSS score can be evaluated in the clinical routine, where the regular use of predictive models, especially in the setting of prescribing palliative RT, is highly favored to make radio-oncological treatments even more personalized.

Funding

Sebastian M. Christ received support through the “Young Talents in Clinical Research” Beginner’s Grant from the Swiss Academy of Medical Sciences (SAMW) and the Bangerter-Rhyner Foundation. Caroline Hertler obtained funding by the “Filling the Gap” program, Faculty of Medicine, University of Zurich, Switzerland.

Availability of data and material

Collected patient data is private and not available for publication. All model specifications are included into the appendix/supplements of this publication, and can therefore be easily replicated by other researchers.

Code availability

Not applicable for this publication.

Authors' contributions

All authors made important contributions to this project. The idea and conceptualization of the project were developed by NA, MG, CH and SC. MS, CH and SC collected and quality-checked all data. SC conducted all statistical analysis. MA prepared the figures. SC compiled the manuscript and received critical input from MS, JW, MA, AS and DB. NA, MG and CH further assessed and critically reviewed the manuscript and the analysis. The final version of the paper was approved by all authors.

Declaration of Competing Interest

NA received research support from ViewRay, BrainLab, SNF, the Swiss Cancer League, the Staffanini Foundation, and received honoraria from ViewRay, AstraZeneca, BrainLab, and Debiopharm.