Validation of the prognosis in palliative care study predictor models in terminal cancer patients.

BACKGROUND: Prognosis in Palliative Care Study (PiPS) predictor models were developed in 2011 to estimate the survival of terminal cancer patients in the United Kingdom. The aim of this study was to validate the PiPS model for terminal cancer patients in Korea, and evaluate its value in clinical practice.
METHODS: This study included 202 advanced cancer patients who were admitted to the cancer hospital's palliative care ward from November 2011 to February 2013. On admission, physicians recorded the PiPS-A, PiPS-B, and doctor's survival estimates in inpatients.
RESULTS: The median survival across PiPS-A categories was 9, 28, and 33 days, and the median survival across PiPS-B was 9.5, 27, and 43 days. The median actual survival was 25 days; overall accuracy between the PiPS-A, PiPS-B, doctor's estimates of survival, and actual survival was 52.0%, 49.5%, and 46.5%, respectively. The PiPS-A and PiPS-B groups for survival in 'days' showed a sensitivity of 48.4% and 64.1%, and specificity of 87.7%, and 77.5%, respectively. The PiPS-A and PiPS-B groups for survival in 'weeks' showed a sensitivity of 59.2%, and 44.7%, and specificity of 61.6%, and 64.7%, respectively. The PiPS-A and PiPS-B 'months' group showed a sensitivity of 37.1% and 37.1%, and specificity of 74.9% and 78.4%, respectively. The 'weeks' and 'months' groups showed significantly prolonged survival rates than 'days' group did in both PiPS-A and PiPS-B, by the Kaplan-Meier method.
CONCLUSION: The PiPS predictor models effectively predicted the survival ≥14 days in terminal cancer patients, and were superior to doctor's estimates.

INTRODUCTION

Cancer is one of the most critical public health problems globally. As of 2010, the total number of cancer-related deaths in Korea was 72,046, accounting for 28.2% of all deaths.1,2,3) Korea has witnessed an emerging interest in palliative care, which seeks to improve the quality of life through symptom management, and psychosocial and spiritual support.4,5,6,7) Prediction of survival in palliative care enables doctors to plan more appropriate end-of-life care, and helps patients and their families prepare for a more comfortable death. However, it is difficult to predict accurate survival in advanced cancer patients due to disease progression and unexpected events, which may include sepsis, massive bleeding, and multiple organ failure.8)

Previous studies have developed systematic and accurate prognostic models, including the palliative prognostic index (PPI),9) the palliative prognostic score (PaP),10) prognostic nomograms,11) and the Prognosis in Palliative Care Study (PiPS) models.12) Performance status, symptoms, signs, laboratory data, physicians' prediction, and predictive models have all been documented as prognostic factors related to survival.9,11,12,13,14,15) Several studies have suggested that accurate prognostic models which incorporate clinicians' estimates of survival are superior to clinicians' estimates alone.10,16,17,18)

The PiPS models were developed for patients in a palliative care service who were diagnosed with advanced cancers in the United Kingdom in 2011;12) the models help predict whether advanced cancer patients are likely to survive for 14 or 56 days. The PiPS models are categorized as PiPS-A or PiPS B, depending on inclusion of blood test results. These models have shown an overall accuracy of 57.3% compared with actual survival. It has been shown that the median survival across the PiPS categories corresponds to the actual median survival. The PPI and the PaP have been validated in many countries, including Korea;19,20) however, there have been no reports on the validation of the PiPS prediction models. The aim of the present study was to evaluate the validation of the PiPS models in Korean advanced cancer patients.

METHODS

We evaluated advanced cancer patients who were admitted to the palliative care ward in Samsung Medical Center Cancer Hospital between November 1, 2011 and February 1, 2013. Of the 256 inpatients, 202 were included as subjects. Fifty patients lacking the records required for the PiPS models, and 4 patients who had undergone chemotherapy within the previous 1 month were excluded. Advanced cancers were defined as those that no longer responded to treatment and underwent progression.

On admission, clinicians recorded the PiPS prediction results, which were calculated using the PiPS prognosticator (www.pips.sgul.ac.uk). Parameters of PiPS prediction model and mental test score in the model are listed in Appendix 1, 2, 3. Survival prediction was stratified into the 'days' (0-13 days), 'weeks' (14-55 days), and 'months' (>55 days) groups. The PiPS-A model comprised diagnosis, symptoms, and general health. The patients' diagnoses, which included breast cancer, male genital organs, distant metastasis, liver metastasis, and bone metastasis, were recorded by reviewing medical records. Cognitive status, Eastern Cooperative Oncology Group (ECOG) performance status, and global health status were assessed by clinicians during admission. Cognitive status was assessed by using the abbreviated mental test score21) comprising 10-item scales. If the mental test score was >3, the clinician recorded a score of 0, and if the mental test score was ≤3, the clinician recorded a score of 1. An ECOG score of 0 indicated normal active performance without restriction, while an ECOG score of 4 indicated complete disability or confinement to a bed or a chair. The global health scores ranged from 1 to 7, score 1 indicating extremely poor health, and score 7 indicating normal health. Heart rate, anorexia, dyspnea, dysphagia, loss of weight in the previous month, and fatigue were assessed on admission.

The PiPS-B model comprised diagnosis, symptoms, general health, and blood test results, which included white blood cell, neutrophils, lymphocytes, platelets, urea, alanine aminotransferase, alkaline phosphatase, albumin, and C-reactive protein, taken within 4 days of admission. In addition, the clinicians' estimates were recorded in order to compare to the PiPS predictions.

Data on the patients' age, sex, cancer treatment history, oxygen therapy, delirium, family support status, religion, and laboratory test results of hemoglobin, total bilirubin, and sodium were retrieved from the medical records. The family support status was assessed by a social worker of the hospital.

Actual survival time in days was defined as the time elapsed from admission to death. Patients who survived at the time of analysis (April 26, 2013) were regarded as censored data. Repetitive admissions were regarded as distinct admissions.

This study was performed in accordance with the institutional review board of the Samsung Medical Center, Seoul, Korea (IRB 2013-03-043).

1. Statistical Analysis

The sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of the PiPS-A and the PiPS-B models in the prediction of survival were calculated for each of the 3 groups ('days,' 'weeks,' and 'months' groups). We estimated the overall accuracy levels to compare the PiPS and clinician's predictions against actual survival.

The Kaplan-Meier curve analysis and log-rank tests were used to compare the survival between the 3 groups of the PiPS-A and the PiPS-B models; P-values were corrected by Bonferroni's method due to multiple comparisons. All statistical assessments were considered significant when P < 0.05. Statistical analysis was performed using IBM SPSS ver. 21.0 (IBM Co., Armonk, NY, USA) and SAS ver. 9.3 (SAS Institute Inc., Cary, NC, USA).

RESULTS

1. General Characteristics of Study Population, Symptoms, Signs, and Blood Test Results

Of the 202 patients, 195 (96.5%) died during the study period. Seven patients survived and were considered censored data.

Patients' characteristics are shown in Table 1. There were 99 males (49%) and 103 females (51%). The mean age of the patients was 62.6 ± 12.1 years, and the median survival time was 25 days (95% confidence interval [CI], 19 to 30 days).

Table 1

Characteristics of patients (n = 202)

Values are presented as number (%), mean ± SD, or median (95% confidence interval).

*Intrathoracic, pharynx, esophagus, biliary, genitourinary, brain, musculoskeletal, skin, blood and immune system, nervous system. †Liver and bone metastasis. ‡Chemotherapy, radiotherapy, and surgery.

The most frequent primary cancer was colorectal cancer (19.3%), followed by gastric (16.8%), and pancreatic cancers (9.9%). Liver metastasis alone was noted in 39.1%, bone metastasis alone in 14.9%, both liver and bone metastases in 12.4%, and no metastasis in 33.6% of patients.

Most of the patients (90.1%) received anticancer treatment. Of these patients, 46 (22.8%) were treated with surgery, chemotherapy, and radiotherapy, 44 (21.8%) were treated with chemotherapy alone, 12 (5.9%) were treated with surgery, and 4 (2.0%) were treated with radiotherapy alone. Forty patients (19.8%) were treated with surgery and chemotherapy, 21 patients (10.4%) with chemotherapy and radiotherapy, 15 patients (7.4%) with surgery and radiotherapy. Twenty patients (9.9%) did not receive any treatments.

Fatigue was present in 90.6%, anorexia in 73.8%, weight loss in the previous month in 59.4%, dyspnea in 31.7%, dysphagia in 30.2%, and delirium in 10.9% of the patients. A mental test score of ≤3, which indicates poor cognitive function, was noted in 26.7%. Oxygen therapy was needed in 32%, and the mean pulse rate was 96.7 per minute (Table 2).

Table 2

Symptoms, signs, and laboratory results of patients

Values are presented as number (%) or mean ± SD.

*Abbreviated mental test score. †A study-specific seven-point scale. ‡Available within four days of study entry.

An ECOG score of 3 was recorded in 51.0%, and 4 in 30.7% of the patients. Approximately two-thirds of the patients had a global health status score between 3 to 5 points.

2. Comparison of Prognosis in Palliative Care Study-A, Prognosis in Palliative Care Study-B, and Clinicians' Estimates against Actual Survival

In the PiPS-A prediction of survival, the number of patients and median survival were 48 (23.8%) and 9 days (95% CI, 7 to 13), respectively, in the 'days' group; similarly, there were 99 patients (49.0%) and 28 days (95% CI, 22 to 34), in the 'weeks' group, and 55 patients (27.2%) and 33 days (95% CI, 27 to 45), in the 'months' group. For the PiPS-B prediction models of survival, the number of patients and median survival were 72 (35.6%) and 9.5 days (95% CI, 8 to 15), respectively, in the 'days' group; Similarly, there were 81 patients (40.1%) and 27 days (95% CI, 23 to 34), in the 'weeks' group, and 49 patients (24.3%) and 43 days (95% CI, 32 to 48), in the 'months' group.

For the clinicians' estimates of survival, the number of patients and median survival were 32 (15.8%) and 9 days (95% CI, 5 to 12), respectively, in the 'days' group; similarly, there were 118 patients (58.4%) and 23 days (95% CI, 16 to 30), in the 'weeks' group and 52 patients (25.8%) and 39 days (95% CI, 30 to 48), in the 'months' group.

The actual survival data were as follows. In the 'days' group, 64 patients (31.7%) survived with a median survival of 7 days (95% CI, 5 to 8). In the 'weeks' group, 103 patients (51.0%) survived with a median survival of 30 days (95% CI, 27 to 34), and in the 'months' group, 35 patients (17.3%) survived with a survival of 84 days (95% CI, 75 to 116) (Table 3).

Table 3

Distribution of PiPS-A, PiPS-B, and doctors' survival predictions against actual survival durations

PiPS: Prognosis in Palliative Care Study.

3. Validation of Prognosis in Palliative Care Study

The sensitivity, specificity, PPV, and NPV of the PiPS-A and PiPS-B models were calculated for the 3 groups (Table 4). In the prediction of survival, the model PiPS-A showed a sensitivity of 48.4%, a specificity of 87.7%, a PPV of 64.6%, and an NPV of 35.4% for the 'days' group; a sensitivity of 59.2%, a specificity of 61.6%, a PPV of 61.6%, and an NPV of 38.4% for the 'weeks' group; a sensitivity of 37.1%, a specificity of 74.9%, a PPV of 23.6%, and an NPV of 76.4% for the 'months' group.

Table 4

Accuracy of PiPS and doctors' predictions

Values are presented as %.

PiPS: Prognosis in Palliative Care Study.

In the prediction of survival, the PiPS-B model showed a sensitivity of 64.1%, a specificity of 77.5%, a PPV of 56.9%, and an NPV of 43.1% for the 'days' group; a sensitivity of 44.7%, a specificity of 64.7%, a PPV of 56.8%, and an NPV of 43.2% for the 'weeks' group; a sensitivity of 37.1%, a specificity of 78.4%, a PPV of 26.5%, and an NPV of 73.5% for the 'months' group.

The overall accuracy between the PiPS-A prediction and actual survival was 52.0%. The overall accuracy between PiPS-B prediction and actual survival was 49.5%. The overall accuracy between clinicians' estimates and actual survival was 46.5%, which were lower than those of the PiPS models.

Kaplan-Meier survival curves of the PiPS-A and PiPS-B models in the 3 groups are shown in Figures 1 and 2. The 'weeks' and 'months' groups showed significantly longer survival than the 'days' group (for the PiPS-A: P = 0.018 between the 'days' and 'weeks' groups, P < 0.001 between the 'days' and 'months' groups) (for the PiPS-B: P < 0.001 between the 'days' and 'weeks' groups, P < 0.001 between the 'days' and 'months' groups). However, there were no significant differences in actual survival between the 'weeks' and 'months' groups (PiPS-A, P = 0.183; PiPS-B, P = 0.114).

Figure 1

Kaplan-Meier survival curves of the three prognostic groups categorized by PiPS-A scores. Kaplan-Meier survival curves of the three prognostic groups categorized by PiPS-A scores: log rank test and P-values were corrected by Bonferroni's method. Solid line: days group, big dotted line: weeks group, small dotted line: months group. P = 0.018 between the 'days' and 'weeks' groups; P < 0.001 between the 'days' and 'months' groups; P = 0.183 between the 'weeks' and 'months' groups. PiPS: Prognosis in Palliative Care Study.

Figure 2

Kaplan-Meier survival curves of the three prognostic groups categorized by PiPS-B scores. Kaplan-Meier survival curves of the three prognostic groups categorized by the PiPS-B scores: log rank test and P-values were corrected by Bonferroni's method. Solid line: 'days' group, big dotted line: 'weeks' group, small dotted line: 'months' group. P < 0.001 between the 'days' and 'weeks' groups; P < 0.001 between the 'days' and 'months' groups; P = 0.114 between the 'weeks' and 'months' groups. PiPS: Prognosis in Palliative Care Study.

DISCUSSION

In the 'days' and 'weeks' groups, the PiPS-A/PiPS-B predictions and clinicians' estimates were consistent with actual median survival. However, in the 'months' group, where the PiPS-A/PiPS-B predictions and clinicians' estimates were 14 to 55 days, 17.3% of the patients in our study were included in the 'months' group, compared to 40% of the patients in the original study.12) These differences in patient numbers might have affected the results among individual groups. In our study, the actual median survival of 25 days was shorter than the actual median survival of 34 days observed in the original study;12) this result indicates a relationship between median survival and the patient population. The patients enrolled in our study were inpatients at a cancer hospital, whereas patients included in the original study were enrolled in various palliative care services, including a hospice, a hospital support team, a home-based palliative care service, and a community team.12) The results of the present study are consistent with those of earlier studies which reported that the median survival of inpatients in a tertiary cancer hospital is relatively shorter than that of patients in various types of palliative care.10,19) One plausible reason for the shorter survival times may be due to the delay in referral to a palliative care service, as patients and their family members may choose to receive aggressive anticancer treatment even at an advanced cancer stage.22)

The PiPS-B scores, which included laboratory results, predicted shorter than actual survival times. The PiPS-B prediction of survival was the most accurate in the 'days' group (35.6%), while clinicians' estimates were the least accurate in the 'days' group (15.8%). These results demonstrate that advanced cancer patients' laboratory results are abnormal due to acute symptoms and long course of the disease on admission. Previous studies have reported that clinicians overestimate survival times.16,19,23) Clinicians cannot accurately predict the time point of death, as they are optimistic due to their acquaintance with patients.15)

The sensitivity, specificity, PPV, and NPV of predictions of survival using the PiPS-A and the PiPS-B predictor models were variable. Using the PiPS-A and the PiPS-B scales, the 'weeks' and 'months' groups showed significantly prolonged survival than the 'days' group, as measured by the Kaplan-Meier method. Although there was no significant difference between the 'weeks' and 'months' groups, our results suggest that the PiPS predictor models were useful for predicting whether advanced cancer patients could survive for more than 14 days. The PiPS-A model, which comprises diagnosis, symptoms, and general health, is non-invasive and useful when laboratory results are unavailable. Although the PiPS-B model, which includes blood test results, can objectively predict patient's condition, it predicted shorter than actual survival times in our study. In cancer hospitals, these laboratory results may be influenced by the treatment to manage acute symptoms.

There have been no previous reports on the validation of the PiPS predictor models in literature; in contrast, the PPI model has been validated in many countries.9,19,20,24) The PPI, which is based on performance status, oral intake, edema, dyspnea, and delirium provides predictions of survival of 3 weeks. In earlier studies, the sensitivity and specificity of the PPI model ranged from 56% to 90%, and 60% to 94%, respectively. The PPV and NPV of the PPI model ranged from 71% to 87%, and 52% to 90%, respectively. The specificity of PiPS in our study was similar to that of the PPI model in previous studies; however, the sensitivity, PPV, and NPV of the PiPS model were lower in our study.19)

With the exception of weight loss in the previous month, anorexia, dyspnea, dysphagia, fatigue, decreased cognitive function, and increased pulse rate occurred with greater frequency in the present study than in the original study.12) These results may have been influenced by the fact that our subjects are admitted to the palliative care ward in a cancer hospital to manage acute symptoms. The laboratory test results were superior in the original study than in our study.12) The findings of the present study indicate that early treatment may influence the results of blood tests taken within 4 days of admission.

The results of the present study are subject to some limitations. First, the generalizability of our results is limited because the present study included only inpatients in the palliative care ward of a cancer hospital, excluding outpatients and other patients from various care systems, including hospices, and home and community care systems. Further research remains to be conducted, in order to generalize the findings for advanced cancer patients in various settings. Second, it is difficult to accurately predict survival due to some fluctuations in symptoms and laboratory results after treatment. Most patients were admitted to our palliative care ward to manage acute symptoms, and we used the laboratory results obtained within 4 days of admission. Third, the 'months' group of the present study was smaller than that reported in the original study.12) Therefore, large-scale studies are needed for clinical practice. Fourth, clinicians' criteria for survival estimates vary due to ambiguity of validation standards in global health; consequently, objective validation tools need to be developed.25) Furthermore, education on survival prediction is important to improve the accuracy of survival prediction; a previous study has shown that the accuracy of survival prediction improved after training.26)

Despite these limitations, the present study was the first to evaluate the PiPS model, and has provided a basis for applying the findings to Korean advanced cancer inpatients. The PiPS prediction models of survival were superior to clinician's estimates. The results of this study suggest that accurate prediction of survival may require comprehensive prognostic tools that which include clinicians' estimates. The PiPS predictor models are useful for predicting whether advanced cancer patients will survive for more than 14 days. Further studies on long-term survival prediction and comparisons of accuracy of several prognostic tools are needed to develop more relevant prognostic factors in order to contribute to palliative care.