Development and validation of a prediction model of poor performance status and severe symptoms over time in cancer patients (PROVIEW+).

BACKGROUND: Predictive cancer tools focus on survival; none predict severe symptoms. AIM: To develop and validate a model that predicts the risk for having low performance status and severe symptoms in cancer patients.
DESIGN: Retrospective, population-based, predictive study. SETTING/PARTICIPANTS: We linked administrative data from cancer patients from 2008 to 2015 in Ontario, Canada. Patients were randomly selected for model derivation (60%) and validation (40%). Using the derivation cohort, we developed a multivariable logistic regression model to predict the risk of an outcome at 6 months following diagnosis and recalculated after each of four annual survivor marks. Model performance was assessed using discrimination and calibration plots. Outcomes included low performance status (i.e. 10-30 on Palliative Performance Scale), severe pain, dyspnea, well-being, and depression (i.e. 7-10 on Edmonton Symptom Assessment System).
RESULTS: We identified 255,494 cancer patients (57% female; median age of 64; common cancers were breast (24%); and lung (13%)). At diagnosis, the predicted risk of having low performance status, severe pain, well-being, dyspnea, and depression in 6-months is 1%, 3%, 6%, 13%, and 4%, respectively for the reference case (i.e. male, lung cancer, stage I, no symptoms); the corresponding discrimination for each outcome model had high AUCs of 0.807, 0.713, 0.709, 0.790, and 0.723, respectively. Generally these covariates increased the outcome risk by >10% across all models: lung disease, dementia, diabetes; radiation treatment; hospital admission; pain; depression; transitional performance status; issues with appetite; or homecare.
CONCLUSIONS: The model accurately predicted changing cancer risk for low performance status and severe symptoms over time.

What is already known about the topic?

There are numerous predictive cancer tools that focus on survival. However, no tools predict risk of low performance status or severe symptoms, which are important for patient decision-making and early integration of palliative care.

What this paper adds

Our cancer study validated a model showing certain covariates (i.e. lung disease, dementia, diabetes, radiation treatment, hospital admission, pain, depression, transitional performance status, issues with appetite, and receipt of homecare) increase one’s risk by >10% of having low performance status, severe pain, well-being, dyspnea, and depression in the subsequent 6 months respectively. Generally these covariates were consistently associated with these outcomes even years after diagnosis.

Implications for practice, theory, or policy

Providing accurate predictions of future performance status and symptom severity can support decision-making and earlier initiation of palliative care, even alongside disease modifying therapies.

Introduction

There is evidence from several randomized trials showing the benefits of palliative care integration at time of diagnosis for cancer patients.[1-3] A clinical practice guideline from the American Society of Clinical Oncology also supports the early integration of palliative care concurrently with standard oncologic care. Yet at the population level, palliative care is often provided very late in the disease trajectory or not at all. In the US, palliative care was accessed in 45% of all deaths at a median of 17 days before death. An enabler to integrate timely palliative care is the use of prognostic tools, particularly online tools, that can inform discussions about survival and support patient decision-making. A systematic review identified 22 online prognostic tools addressing 89 different cancers.

The systematic review however, also identified several major challenges in using prognostic tools to integrate palliative care earlier in the disease trajectory. First, tools focus on predicting mortality, but “no tool used quality-of-life as one of its outcomes. . .yet quality-of-life outcomes are most meaningful and important to patients when making treatment decisions.” Clinicians and patients may be more inclined to discuss performance status and symptoms which affect their quality of life, rather than an estimated date of death. Second, the tools neither account for patient-reported outcomes, such as their current symptoms, nor prior health services use, which are clinically relevant, and predictive variables that vary over time.[7,8] This limits their utility since decision-making often centers around potential trade-offs between quality-of-life now, in the future, and survival expectancy, which changes as the disease progresses. Third, the majority of tools used biological and laboratory variables (e.g. cancer antigen levels, elevated C-reactive protein, leukocytosis, etc.)[9,10] which are not typically known by patients. This prevents patients from obtaining individualized prognostic information that could help them initiate discussions about palliative care.

Our team sought to address these limitations. Previously we developed and validated a model, including patient-reported outcomes to predict survival across the disease trajectory for patients with any cancer type. In this study, we validated the model to also predict the risk for having low performance status and several severe symptoms. We call the final model PROVIEW+. By providing information about survival in the context of outcomes related to quality-of-life, such as anticipated symptom severity, PROVIEW+ can support decision-making and initiating palliative care earlier, even alongside disease modifying therapies.

Methods

Study design and population

We performed a population-based, retrospective cohort study of adults diagnosed with cancer, as confirmed by the provincial cancer registry in Ontario, Canada, from January 1 2008, to December 31, 2015. The study was reviewed by Hamilton Integrated Research Ethics Board and deemed exempt because it used de-identified secondary administrative data. This study followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) reporting guideline.

Data sources

We used the following linked administrative databases [and corresponding covariates]: (1) Ontario Cancer Registry [cancer type, stage]; (2) Vital Statistics [age, sex, date of death]; (3) Statistics Canada [distance from cancer center]; (4) Activity Level Reporting [chemotherapy regime, radiation treatment]; (5) Discharge Abstract Database [hospitalization dates, diagnoses, cancer surgery, comorbidity]; (6) National Acute Care Registry System [Emergency Department (ED) visits]; (7) physician billing [physician home visits for palliative care, rostered patient]; (8) Home care database [end of life home care service use]; 9) Ontario Drug Benefit [long-term care status]; (10) Symptom Management Dataset [symptoms, performance status]; (11) interRAI database [performance status, symptoms].

The Symptom Management database contains data from a province-wide systematic screening program where oncology outpatients at each outpatient visit voluntarily complete valid tools, namely the Edmonton Symptom Assessment System (ESAS) for symptoms and the Palliative Performance Scale (PPS) for performance status.[14,15] The ESAS asks patients to self-report the severity of nine symptoms (i.e. pain, depression, well-being, shortness of breath, anxiety, nausea, tiredness, drowsiness, and appetite) on a scale of 0 (symptom absent) to 10 (most severe). The PPS summarizes a patient’s performance status based on a patient’s level of ambulation, activity, and self-care. It is scored from 80 to 100 indicating stable, 40–70 indicating transitional, 10–30 indicating end of life, and 0 indicating dead. The PPS was completed by a clinician starting in 2007. In 2013, Ontario switched to collecting functional scores using a patient-completed Eastern Cooperative Oncology Group score. Research has shown that the patient-reported PPS is comparable to and highly correlated with clinician-reported PPS.[16-18]

The interRAI database contains data from the Resident Assessment Instrument for Home Care (RAI-HC), a standardized tool for patients receiving publicly-funded home care services. The assessment is akin to the Minimum Data Set used internationally and has strong validity and reliability.[19-21] Seventy percent of cancer patients use home care in the last year of life. The assessment collects various quality-of-life related items, such as the presence of pain or symptoms of depression. The assessment is completed by the case manager at intake and reassessed approximately every 6–12 months. All above mentioned databases were linked using unique encoded identifiers and analyzed at ICES.

Outcomes

The model predicts the below five dichotomous (Yes/No) outcomes:

Low performance status: Defined as a score of 30–10 on clinician- or patient-reported PPS; or high health instability as indicated by a score of 4 or 5 on the CHESS Scale (Changes in Health, End-stage Disease, Signs and Symptoms) embedded within the RAI-HC (the scale includes items related to change in Activities of Daily Living status, evidence of end stage disease, etc.). Both tools are validated to be highly predictive of mortality, including in hospital and community-based settings, and include physical functioning to complete activities of daily living as a main predictor of survival.[23-26]

Severe pain: Defined as: a score of 7–10 (severe) for pain on ESAS; or a score of 3 (severe or excruciating) for pain intensity from an item on the RAI-HC.

Severe dyspnea: Defined as: a score of 7–10 (severe) for shortness of breath on ESAS; or Yes for the presence of “shortness of breath” item on the RAI-HC.

Poor well-being: Defined as a score of 7–10 (poor) well-being on ESAS; or Yes for “client feels he/she has poor health when asked” from the RAI-HC.

Moderate-severe depression: Defined as a score of 4 to 10 (moderate to severe) for depression on ESAS; or a score of three or more on the Depression Rating Scale from the RAI-HC (e.g. made negative statements, expressions of unrealistic fears, repetitive anxious complaints, crying/tearfulness).

Index date for the models

For each of the five outcomes, prediction methods for logistic regression models were implemented independently starting at diagnosis in Year 0 (abbreviated as Y0) and re-implemented at Y1–Y4 after diagnosis. Thus, we have 25 unique models. Doing this means that to be included in each model, one must be alive at the start of each year and have an outcome measurement during the 6 month follow-up window, which aims to address loss to follow-up or death over time. In the Y0 model, all baseline covariates are measured from index to 3 months (after index), and the outcome window is from month 3 to month 9. In the Y1 model (and subsequent yearly models), baseline covariates are measured from ± 3 months from the new 12-month index point and the outcome is measured from month 3 to month 9 from the new index. Where multiple assessments were available, we used the assessment closest to the 6 month end point (for outcome) and closest to the index date (for baseline covariate).

Covariates

Each model included the following covariates: demographic characteristics (age at diagnosis, sex, caregiver living with patient (yes/no), lives within 50 km of a cancer center (yes/no)); clinical data (cancer type, cancer stage, presence of 1 of 13 other chronic diseases as determined by validated algorithms,[28,29] type of chemotherapy (publicly funded oral drugs, immunotherapy, and systemic agents), receipt of radiation treatment (yes/no) and/or cancer surgery (yes/no) in the past (from diagnosis up to 3 months previously), and ongoing (within the past 3 months)); patient-reported outcomes (Performance status and nine symptom scores within 3 months of index date); and health care use within 3 months of index date (prior hospitalization, hospitalizations for palliative care (including palliative care consultations), living in long-term care, receipt of end-of-life homecare services, having a regular family physician, and received physician home-visit).

Statistical analysis for each model

Development

We randomly selected 60% of eligible patients for model derivation and used the other 40% for validation. To ensure random sampling, we assessed and compared the distribution of baseline characteristics between the derivation and validation cohorts. Each outcome was examined separately. Using the derivation cohort, we used a multivariable logistic regression model with baseline (time-fixed) characteristics to predict the risk of an outcome at 6 months from index. A priori, we created a multivariable model consisting of all potential predictors mentioned above. We then used a backward stepwise selection procedure for variable selection with a liberal 2-sided p-value <0.15 as the retention criterion. We centered continuous covariates such as age and explored both linear and quadratic terms. Missing information from patient-reported categorical variables were handled by creating an additional missing category for that variable. Most of the missing data arose due to patients not completing an ESAS. As there was no obvious missing pattern, we elected to create a missing category rather than to impute or remove these patients from the analysis. Interactions between cancer type and stage were also incorporated to achieve maximal discrimination.[32,33]

Validation

For each outcome, the predictive performance of the derived model was assessed and compared using the validation cohort. The regression model estimates were applied to every individual in the validation cohort to obtain their corresponding estimated risk probability. The predicted number of outcomes was then compared to the actual number of outcomes in the validation cohort by composing a confusion matrix; we calculated sensitivity (true positive fraction), specificity (true negative fraction), accuracy (true positive or negative fraction), and discrimination. Discrimination was measured using the area under the ROC curve (AUC).

Calibration plots were also constructed using the validation cohort. This was done by grouping patients into deciles based on their predicted risk, and then plotting the observed outcome risk within a decile against the corresponding mean predicted risk within that decile.[7,35] Points closer to the 45° line indicate better calibration. Characteristics of individuals belonging to the highest predicted risk decile were also examined. As a sensitivity analysis, we assessed model performance using complete case data (i.e. excluding patients with missing covariate values) and determined that the concordance statistics were very similar to our model using all patients (including missing covariate values). All analyses were conducted using the statistical software R version 2.15 and SAS version 9.3.

Results

Our population-based cohort identified 255,494 patients diagnosed with cancer between 2008 and 2015 in Y0. Each total cohort was then randomly split into derivation (60%) and validation cohorts (40%). As we repeated the derivation and validation process each year up to 4 years after diagnosis, conditional on survival and having an assessment in the outcome window, the total cohort reduced each year. For instance, the validation cohort in dyspnea model reduced over time from 101,696 (Y0) to 61,511 (Y1), 43,759 (Y2), 30,383 (Y3), and 20,672 (Y4).

The demographics for each of the derivation models in Y0 are presented in Table 1. (Supplemental Appendix e Table 1 includes all variables across all years). Across the five models in Y0 generally, the median age at diagnosis was 64 years old, 57% were female, and the most common cancers were breast (24%), lung (13%), prostate (9%), and colorectal (12%). Approximately 34% of the cohorts had Stage 3 or 4 disease, 42% had Stage 1 or 2, and 24% had unknown stage in the cancer registry. Within the first 3 months of diagnosis, 49% had cancer-related surgery, 34% received chemotherapy, and 26% received radiation therapy. In the cohort, 5% had high pain, while 33% had no pain, and 32% had missing values (e.g. did not complete an ESAS). In Y0, the prevalence of outcomes among the derivation and validation cohorts were very similar, which ranged from 2.4% (low performance status) to 10.5% (poor well-being). Across all years and models, the validation cohorts were very similar in their distribution of patient profiles to the derivation cohorts.

Table 1.

Baseline characteristics of cohort at time of first diagnosis.

Variable	Value	Y0
		Functional status	Pain	Dyspnea	Depression	Wellbeing
		Cohort A (analysis) N = 75,287	Cohort A (analysis) N = 100,578	Cohort A (analysis) N = 101,696	Cohort A (analysis) N = 99,915	Cohort A (analysis) N = 100,828
Prevalence of outcome		1829 (2.4%)	7354 (7.3%)	9867 (9.7%)	9277 (9.3%)	10,577 (10.5%)
Age at diagnosis	Median (IQR)	64 (55–73)	64 (55–73)	64 (55–73)	64 (55–73)	64 (55–73)
Sex	Female	42,950 (57.0%)	57,377 (57.0%)	57,810 (56.8%)	57,258 (57.3%)	57,305 (56.8%)
Cancer type**	Breast	18,255 (24.2%)	24,210 (24.1%)	24,665 (24.3%)	24,439 (24.5%)	24,494 (24.3%)
	Colorectal	9251 (12.3%)	11,863 (11.8%)	11,962 (11.8%)	11,785 (11.8%)	11,941 (11.8%)
	Lung	9398 (12.5%)	12,525 (12.5%)	12,464 (12.3%)	12,162 (12.2%)	12,303 (12.2%)
	Prostate	6089 (8.1%)	8827 (8.8%)	9157 (9.0%)	8805 (8.8%)	9175 (9.1%)
Cancer stage	Stage 1	13,824 (18.4%)	20,319 (20.2%)	20,569 (20.2%)	20,211 (20.2%)	20,165 (20.0%)
	Stage 2	16,719 (22.2%)	22,861 (22.7%)	23,390 (23.0%)	22,917 (22.9%)	23,301 (23.1%)
	Stage 3	14,302 (19.0%)	18,684 (18.6%)	18,937 (18.6%)	18,535 (18.6%)	18,765 (18.6%)
	Stage 4	11,563 (15.4%)	15,132 (15.0%)	15,044 (14.8%)	14,877 (14.9%)	14,980 (14.9%)
	Uknown	18,879 (25.1%)	23,582 (23.4%)	23,756 (23.4%)	23,375 (23.4%)	23,617 (23.4%)
Radiation (within 3 months)	Yes	19,445 (25.8%)	26,548 (26.4%)	26,697 (26.3%)	26,269 (26.3%)	26,486 (26.3%)
Chemotherapy (within 3 months)	Yes	26,398 (35.1%)	33,600 (33.4%)	33,944 (33.4%)	33,741 (33.8%)	33,821 (33.5%)
Cancer surgery (within 3 months)	Yes	36,576 (48.6%)	49,667 (49.4%)	50,076 (49.2%)	49,670 (49.7%)	49,878 (49.5%)
Chronic diseases †	CHF	4263 (5.7%)	5527 (5.5%)	5577 (5.5%)	5444 (5.4%)	5465 (5.4%)
	COPD	6808 (9.0%)	8874 (8.8%)	8885 (8.7%)	8708 (8.7%)	8872 (8.8%)
	Dementia	1317 (1.7%)	1634 (1.6%)	1582 (1.6%)	1566 (1.6%)	1578 (1.6%)
	Diabetes	16,927 (22.5%)	22,066 (21.9%)	22,086 (21.7%)	21,532 (21.6%)	21,936 (21.8%)
	Renal disease	3583 (4.8%)	4523 (4.5%)	4511 (4.4%)	4423 (4.4%)	4483 (4.4%)
Distance from regional cancer center	< = 50 km	60,737 (80.7%)	79,976 (79.5%)	80,708 (79.4%)	79,390 (79.5%)	79,964 (79.3%)
Was pt hospitalized in the past 3 months?	Yes	4757 (6.3%)	6286 (6.2%)	6384 (6.3%)	6163 (6.2%)	6283 (6.2%)
Functional score at index (+3 months) ‡	0 = 100	24,314 (32.3%)	26,944 (26.8%)	27,386 (26.9%)	26,692 (26.7%)	27,186 (27.0%)
	1 = 90–80	15,895 (21.1%)	16,756 (16.7%)	16,868 (16.6%)	16,740 (16.8%)	16,857 (16.7%)
	2 = 70–60	5304 (7.0%)	5786 (5.8%)	5734 (5.6%)	5610 (5.6%)	5585 (5.5%)
	3 = 50–40	2795 (3.7%)	2921 (2.9%)	3006 (3.0%)	2917 (2.9%)	2949 (2.9%)
	4 = 30–10	869 (1.2%)	975 (1.0%)	945 (0.9%)	905 (0.9%)	931 (0.9%)
	Missing	26,110 (34.7%)	47,196 (46.9%)	47,757 (47.0%)	47,051 (47.1%)	47,320 (46.9%)
Pain score at index (+3 months)	None	25,071 (33.3%)	33,236 (33.0%)	33,727 (33.2%)	33,030 (33.1%)	33,172 (32.9%)
	Low	13,969 (18.6%)	18,530 (18.4%)	18,597 (18.3%)	18,397 (18.4%)	18,538 (18.4%)
	Moderate	8953 (11.9%)	11,265 (11.2%)	11,356 (11.2%)	11,241 (11.3%)	11,275 (11.2%)
	High	4386 (5.8%)	5404 (5.4%)	5314 (5.2%)	5300 (5.3%)	5304 (5.3%)
	Missing	22,908 (30.4%)	32,143 (32.0%)	32,702 (32.2%)	31,947 (32.0%)	32,539 (32.3%)
Wellbeing score at index (+3 months)	0 = Best	13,383 (17.8%)	18,053 (17.9%)	18,292 (18.0%)	17,999 (18.0%)	18,021 (17.9%)
	1–3	21,626 (28.7%)	27,462 (27.3%)	27,739 (27.3%)	27,501 (27.5%)	27,671 (27.4%)
	4–6	13,850 (18.4%)	17,824 (17.7%)	18,035 (17.7%)	17,649 (17.7%)	17,786 (17.6%)
	7–10 = Worst	5240 (7.0%)	6928 (6.9%)	6880 (6.8%)	6756 (6.8%)	6776 (6.7%)
	Missing	21,188 (28.1%)	30,311 (30.1%)	30,750 (30.2%)	30,010 (30.0%)	30,574 (30.3%)
Dyspnea score at index (+3 months)	Yes	4659 (6.2%)	5814 (5.8%)	5803 (5.7%)	5696 (5.7%)	5804 (5.8%)
	No	50,065 (66.5%)	65,135 (64.8%)	65,833 (64.7%)	64,882 (64.9%)	65,114 (64.6%)
	Missing	20,563 (27.3%)	29,629 (29.5%)	30,060 (29.6%)	29,337 (29.4%)	29,910 (29.7%)
Depression score at index (+3 months)	Yes	10,059 (13.4%)	13,136 (13.1%)	13,080 (12.9%)	12,898 (12.9%)	12,960 (12.9%)
	No	41,688 (55.4%)	54,624 (54.3%)	55,221 (54.3%)	54,413 (54.5%)	54,667 (54.2%)
	Missing	23,540 (31.3%)	32,818 (32.6%)	33,395 (32.8%)	32,604 (32.6%)	33,201 (32.9%)
Pt has had palliative home care (nursing/personal support)		3233 (4.3%)	4383 (4.4%)	4366 (4.3%)	4266 (4.3%)	4308 (4.3%)

CHF: congestive heart failure; COPD: chronic obstructive pulmonary disease; IQR: interquartile range.

Other cancer disease sites were other genitourinary, other gastrointestinal, hematologic, head and neck, gynecologic, and other sites.

Other chronic diseases measured but not reported were acutemyocardial infarction, arrythmia, asthma, coronary heart disease, diabetes, hypertension, inflammatory bowel disease, mood disorder, osteoarthritis, osteoporosis, renal disease, rheumatoid arthritis, and stroke; mental health hospital admission was also measured but not reported.

Functional score ranges from 0 to 100 (in 10-point increments), with 80 to 100 indicating stable, 40 to 70 indicating transitional, 10–30 indicating end of life, and 0 indicating dead.

After backward stepwise selection, each outcome model had a slightly different set of variables included in the final prediction model. We present the results of our models for Year 0 in Table 2. Covariates that increased the risk of having a low PPS in 6 months by >10% were: Chronic Obstructive Pulmonary Disease, dementia, diabetes; radiation treatment; a hospital admission in the prior 3 months; high pain; symptoms of depression; a current PPS score of 70–10; any issues with appetite; or received end-of-life homecare. Having an existing poor score on a symptom at index was one of the biggest predictors of having a poor score on the same symptom in 6 months’ time. Generally, these variables were also usually associated with a >10% increased risk of having other high symptoms in Y0. Moreover, these variables were also usually highly predictive in the other models, though this varied by year and by symptom model (Supplemental Appendix eTable 2).

Table 2.

Fully adjusted main effects model associations fo having low performance status or severe symptoms at 6 months from diagnosis across cohorts in Year 0 .

Parameter at index (diagnosis)	Y0
	Low PPS at 6 months	Severe pain at 6 months	Severe dyspnea at 6 months	Moderate-severe depression at 6 months	Worst eellbeing at 6 months
	OR (95% CI)	OR (95% CI)	OR (95% CI)	OR (95% CI)	OR (95% CI)
Age at index date	1.03 (1.02, 1.03)	1 (0.99, 1)	1.02 (1.02, 1.02)	N/A	1 (1, 1)
Female (ref = male)	N/A	1.15 (1.08, 1.22)	N/A	1.18 (1.12, 1.25)	1.11 (1.05, 1.17)
Chronic diseases
CHF	N/A	1.11 (1, 1.23)	1.48 (1.36, 1.6)	N/A	1.11 (1.02, 1.21)
COPD	1.17 (1.03, 1.35)	1.31 (1.21, 1.42)	2.21 (2.08, 2.36)	1.19 (1.1, 1.28)	1.22 (1.14, 1.31)
Dementia	1.52 (1.2, 1.93)	N/A	N/A	1.36 (1.18, 1.58)	N/A
Diabetes	1.17 (1.05, 1.31)	1.2 (1.14, 1.28)	1.1 (1.05, 1.16)	1.16 (1.1, 1.22)	1.17 (1.11, 1.23)
Cancer type (ref = lung)**
Prostate	0.39 (0.28, 0.52)	0.61 (0.53, 0.69)	0.22 (0.19, 0.25)	0.41 (0.36, 0.47)	0.43 (0.38, 0.48)
Colorectal	0.78 (0.65, 0.95)	0.84 (0.75, 0.93)	0.34 (0.31, 0.37)	0.85 (0.78, 0.94)	0.72 (0.66, 0.78)
Breast	0.57 (0.46, 0.71)	0.76 (0.69, 0.84)	0.33 (0.3, 0.36)	0.76 (0.7, 0.84)	0.67 (0.61, 0.73)
Cancer stage (ref = stage 1)**
4	2.24 (1.83, 2.74)	1.71 (1.55, 1.88)	1.59 (1.46, 1.73)	1.62 (1.48, 1.76)	1.57 (1.45, 1.7)
3	1.41 (1.14, 1.74)	1.37 (1.25, 1.5)	1.33 (1.23, 1.44)	1.41 (1.3, 1.53)	1.22 (1.13, 1.31)
2	1.03 (0.82, 1.3)	1.15 (1.05, 1.26)	1.17 (1.08, 1.27)	1.22 (1.13, 1.32)	1.09 (1.01, 1.17)
Radiation in last 3 months (yes)	1.59 (1.43, 1.78)	1.31 (1.23, 1.38)	N/A	1.08 (1.03, 1.14)	1.19 (1.13, 1.25)
Chemotherapy in last 3 months (yes)	N/A	0.87 (0.82, 0.92)	N/A	0.88 (0.84, 0.93)	0.88 (0.83, 0.92)
Surgery in last 3 months (yes)	0.78 (0.69, 0.88)	0.75 (0.71, 0.8)	0.79 (0.75, 0.84)	0.85 (0.81, 0.9)	0.89 (0.84, 0.93)
Within 50 km from cancer centre	0.84 (0.75, 0.94)	N/A	0.87 (0.82, 0.92)	0.94 (0.89, 1)	1.07 (1.01, 1.13)
Admitted to hospital in last 3 months (yes)	1.24 (1.06, 1.45)	1.12 (1.02, 1.22)	1.13 (1.04, 1.22)	1.18 (1.09, 1.28)	1.18 (1.09, 1.28)
Pain score at index (ref = Level 0)*
3	1.19 (1, 1.42)	3.92 (3.56, 4.31)	1.26 (1.15, 1.39)	1.26 (1.15, 1.39)	1.32 (1.21, 1.45)
2	1.08 (0.93, 1.25)	2.29 (2.1, 2.49)	1.1 (1.02, 1.19)	1.26 (1.17, 1.36)	1.34 (1.25, 1.44)
1	0.97 (0.84, 1.12)	1.47 (1.35, 1.6)	1.13 (1.06, 1.21)	1.12 (1.05, 1.21)	1.18 (1.1, 1.26)
Wellbeing score at index (ref = Level 0)*
3	N/A	N/A	1.14 (1.02, 1.27)	1.49 (1.33, 1.66)	1.99 (1.79, 2.2)
2	N/A	N/A	1.14 (1.05, 1.25)	1.41 (1.28, 1.55)	1.39 (1.27, 1.52)
1	N/A	N/A	1.06 (0.98, 1.15)	1.29 (1.18, 1.41)	1.27 (1.17, 1.38)
Dyspnea at index (yes)*	N/A	N/A	3.01 (2.8, 3.23)	N/A	1.09 (1.01, 1.18)
Depression at index (yes)	1.16 (1.01, 1.33)	1.18 (1.1, 1.27)	1.11 (1.04, 1.19)	2.26 (2.12, 2.42)	1.31 (1.23, 1.4)
Functional score at index (ref = Level 0)* ‡
4	4.16 (3.32, 5.23)	0.84 (0.67, 1.05)	1.3 (1.08, 1.56)	1.47 (1.23, 1.76)	1.28 (1.08, 1.53)
3	1.65 (1.36, 1.99)	1.04 (0.91, 1.18)	1.25 (1.12, 1.4)	1.3 (1.17, 1.46)	1.21 (1.09, 1.35)
2	1.09 (0.92, 1.31)	1.21 (1.09, 1.33)	1.26 (1.15, 1.38)	1.18 (1.08, 1.3)	1.13 (1.03, 1.23)
1	1.04 (0.9, 1.21)	1.09 (1, 1.18)	1.13 (1.05, 1.21)	1.09 (1.01, 1.17)	1.02 (0.95, 1.1)
Appetite at index (ref = Level 0)*
3	1.58 (1.31, 1.92)	1.09 (0.97, 1.21)	1.09 (0.98, 1.21)	N/A	1.13 (1.03, 1.25)
2	1.28 (1.08, 1.51)	1.05 (0.96, 1.15)	1.12 (1.03, 1.22)	N/A	1.13 (1.05, 1.22)
1	1.12 (0.94, 1.32)	1 (0.91, 1.09)	1.09 (1.01, 1.18)	N/A	1.05 (0.97, 1.13)
Palliative homecare (yes) (ref = no homecare)	1.33 (1.14, 1.55)	0.82 (0.74, 0.91)	0.82 (0.75, 0.9)	0.77 (0.7, 0.85)	1.09 (1, 1.19)

CHF: congestive heart failure; COPD: chronic obstructive pulmonary disease; HR: hazard ratio; NA: not applicable (indicating covariate was not significant in the final model).

A full list of covariates for each model is given in eTable 2 in the Appendix.

Missing category is not shown.

The HR estimates are from the main effects–only model (without the interaction between cancer type and cancer stage).

Functional score ranges from 0 to 100 (in 10-point increments), with 80 to 100 indicating stable, 40 to 70 indicating transitional, 10 to 30 indicating end of life, and 0 indicating dead.

Calibration plots for all models in Year 0 in our validation cohorts show the observed values plotted along the predicted values closely falling along the 45° line (Figure 1). Model discrimination in our validation cohorts is very high. The AUC for all our models are shown in Table 3, an average AUC across all 25 models is 0.7676 (ranging from 0.8202 (Y3-Dyspnea) to 0.6630 (Y4-Well-Being)) (Supplemental Appendix e Figure 1).

Figure 1.

Calibration plots by deciles of predicted probability in Y0.

Table 3.

Area under the curve (AUC) scores, positive predictive values (PPV) and negative predictive values (NPV) by year.

		Y0	Y1	Y2	Y3	Y4
PPS	AUC	0.807	0.804	0.818	0.802	0.792
	PPV	0.061	0.074	0.073	0.062	0.062
	NPV	0.992	0.990	0.992	0.991	0.991
Pain	AUC	0.713	0.762	0.761	0.754	0.768
	PPV	0.129	0.155	0.149	0.143	0.164
	NPV	0.962	0.967	0.967	0.967	0.966
Dyspnea	AUC	0.790	0.820	0.820	0.820	0.819
	PPV	0.215	0.230	0.239	0.249	0.255
	NPV	0.961	0.965	0.963	0.963	0.962
Depression	AUC	0.723	0.786	0.787	0.779	0.789
	PPV	0.173	0.184	0.179	0.196	0.183
	NPV	0.948	0.966	0.966	0.961	0.969
Well-being	AUC	0.709	0.743	0.684	0.677	0.663
	PPV	0.182	0.187	0.196	0.196	0.179
	NPV	0.942	0.952	0.953	0.947	0.947

Application of the model

The model was developed into an online tool, called PROVIEW+. To exemplify how the model could be used, we consider the following hypothetical scenario: a 70-year old male who was diagnosed with stage III lung cancer 3 years ago (i.e. the calculator would use the Year 3 model). His baseline characteristics at Year 3 are: while he received chemotherapy and radiation since diagnosis, he only continues to receive chemotherapy in the past 3 months; he has diabetes and hypertension; no symptoms except moderate dyspnea (score of 4); and has a PPS of 70. For someone with these baseline characteristics in our model, the probability in the next 6 months of having a poor PPS is 4.6% (95%CI: 3.0–7.2), severe pain is 2.1% (95%CI: 1.6–2.9), dyspnea is 10.1% (95%CI: 7.3–14.0), depression is 4.3% (95%CI: 3.0–6.1), and worst well-being is 9.3% (95%CI: 6.7–12.7). If the man was hospitalized shortly thereafter, and now has a PPS of 30 and severe scores of “8 out of 10” for all nine ESAS symptoms, his probability in the next 6 months of having a poor PPS is 28.2% (95%CI: 16.0–44.6), severe pain is 35.1% (95%CI: 22.9–49.5), dyspnea is 71.7% (95%CI: 57.9–82.3), depression is 78.6% (95%CI: 69.4–85.6) and worst well-being is 71.3% (95%CI: 59.3–80.9). These statistics coupled with his severe symptom burden and low PPS may trigger the man to discuss with his doctor additional surveillance and care planning to address potentially severe symptoms. They could also discuss uncertainty around the predictions in the context of where he is on his illness trajectory and his goals of care.

Discussion

In this study, we developed and validated several predictive models for the risk of poor performance status and severe symptoms for all cancer types over time. The models achieved high calibration and discrimination. The model shows how risk for various patient-reported outcomes changes due to changes in a patient’s condition, such as hospitalization, a decline in PPS, or a prolonged symptom exacerbation. Moreover, because this work advances a previously validated survival model, users can examine the trade-offs between future healthcare use, survival, and performance status and symptoms.

There are a few features of our model that are novel. Compared with other online tools that predict cancer survival,[37,38] to our knowledge, our model is the only one that uses symptoms and performance status as covariates and outcomes. This is important because other tools might account for treatments, but they do not differentiate among individuals who had the same treatments but have different performance status or symptom profiles. Our model was re-developed each year post-diagnosis (up to first 5 years) based on updated covariate information at each new anchor point. Thus it can be used at any time within the first 5 years after diagnosis, while accounting for changes in a patient’s condition or treatment over time.

Patients and families often make decisions that try to achieve both longer survival, but also improve the quality of life remaining. The model can support discussions about a palliative approach to care and shared decision-making, particularly for patients, families or clinicians who are hesitant to discuss a specific timeframe for death. For instance, a high risk of poor performance status has implications for the quality-of-life of both patient and family, and options for managing this potential transition can be explored without discussing risk of death. Further, by providing risks for short-term outcomes related to quality of life, such as severe pain or dyspnea in the next 6 months, the tool can then trigger discussions on how to manage those risks, such as initiation of palliative care services. Finally, because the tool can be completed by patients and families directly, pre-contemplative discussions could occur before visits with the oncologists, so that the clinic time is used most productively and the discussions do not necessarily need to be initiated by clinicians.

Limitations

Our study does not incorporate genetic biomarkers and specific targeted therapies, which were not available for this project. These variables, along with other patient-reported covariates (e.g. preferences, ethnicity, etc.) could be pursued in another iteration. Because patient-reported outcomes were either voluntary or among those receiving a homecare assessment, we do not have these data for all eligible patients at every time point which is a risk for selection bias, though we have among the largest, longitudinal databases with this information. Although the model was validated and the initial online calculator is available, future research is needed to test the model’s usefulness for shared decision-making, early palliative care integration, and improved outcomes. Testing and refining the online tool with patients and family users, as well as clinicians, is a planned subsequent step in the research program.

Conclusion

Our models showed that changing cancer risk for poor performance status and severe symptoms can be accurately predicted using administrative clinical and patient-reported outcomes data. Combining these risks with survival risk can help patients and families to understand how transition points (e.g. hospitalization or performance status decline) and treatment decisions (e.g. continuing treatment or initiating homecare) might affect different aspects of their disease journey ahead. In this way, the model can support the initiation of conversations about palliative care supports earlier in the disease trajectory.

Click here for additional data file.

Supplemental material, sj-xlsx-1-pmj-10.1177_02692163211019302 for Development and validation of a prediction model of poor performance status and severe symptoms over time in cancer patients (PROVIEW+) by Hsien Seow, Peter Tanuseputro, Lisa Barbera, Craig C Earle, Dawn M Guthrie, Sarina R Isenberg, Rosalyn A Juergens, Jeffrey Myers, Melissa Brouwers, Semra Tibebu and Rinku Sutradhar in Palliative Medicine