Prognostic value of a patient-reported functional score versus physician-reported Karnofsky Performance Status Score in brain metastases.

INTRODUCTION: Our aim was to investigate the added prognostic value of a patient-reported functional outcome score over Karnofsky Performance Status (KPS) in patients with non-small-cell lung cancers (NSCLC) with brain metastases.
MATERIALS AND METHODS: The baseline data are from a prospective cohort study involving 140 consecutive patients presenting at our institute. A patient reported performance status (PRPS) was obtained by summing the physical- and role-functioning scale scores of the EORTC QLQ C30 questionnaire. Nested cox proportional hazards models predicting survival were developed including both KPS and PRPS (full model), KPS only (KPS Model), and PRPS only (PRPS model). The incremental value of the addition of KPS or PRPS was ascertained using the likelihood ratio test, model adequacy index and integrated discrimination Improvement (IDI).
RESULTS: PRPS was an independent and statistically significant prognostic factor and had only a moderate degree of agreement with KPS. All models showed nearly the same discrimination and calibration accuracy, but the likelihood ratio test comparing the full model to the KPS model was significant (L.R. Chi2 = 5.34, p = 0.02). Model adequacy index for the KPS model was 85% versus 95% for the PRPS model. IDI when comparing the KPS model to the full model was 0.0279, while it was 0.008 for the PRPS model versus the Full model.
CONCLUSIONS: Use of patient-reported functional outcomes like PRPS can provide the same prognostic information as KPS in patients of NSCLC with brain metastases. HIGHLIGHTS: Patient-reported functional status (PRPS) has a moderate degree of agreement with KPS.PRPS is an independent and significant predictor of survival in brain metastases.PRPS can replace KPS without loss of prognostic information.

Introduction

Performance status (PS) has been defined by the National Cancer Institute as ‘a measure of how well a patient is able to perform ordinary tasks and carry out daily activities’ [38]. The baseline PS of patients is one of the most important factors influencing their prognosis [2]. In the context of patients with brain metastases, the baseline PS is the key prognostic factor determining survival [13, 32–34]. This is exemplified by the fact that the baseline PS is included in the diagnosis-specific graded prognostic assessment (DS-GPA) score across different primary sites [33]. The physician-reported performance status is usually reported in the form of a summary score, and the two most commonly used scales are the Karnofsky performance status (KPS) [18], and the Eastern Cooperative Oncology Group Performance Status (ECOG-PS) scales [23]. Of these, KPS has been used in successive trials in brain metastases patients and is also used for validated prognostic indices like the GPA [33].

PS is usually estimated by health care professionals during the course of a routine health care visit. However, estimation of PS using the KPS or ECOG scales suffers from interobserver variation due to the subjective nature [7, 28, 31, 36] of the assessment scale. Further, given the subjective nature of assessment, it is not surprising that there is only a moderate degree of agreement between the PS score as assessed by the patient themselves and the physician [3, 5, 6, 17, 22].

It is well known that physicians and other health care workers’ judgement about subjective measures, like patient’s pain, quality of life, anxiety, depression, etc., is different from that of the patients themselves [30]. Given the inherent subjectivity in PS assessment and the previous reports of disagreement between physician and patient assessed PS, it is reasonable to question if the use of patient-reported functional status would allow us to get better prognostic estimates as compared to KPS.

Functional status can be ascertained through the functional domains of Health-Related Quality of Life (HR-QOL) instruments like the European Organisation for Research and Treatment of Cancer (EORTC) Quality of Life Questionnaire C-30 (QLQ-C30) [20]. This questionnaire comprises of 30 items, of which 15 items contribute to five functional scales looking at physical, role, emotional, cognitive and social functioning.

Between June 2012 to April 2015 we enrolled 140 consecutive patients with non-small-cell lung cancer (NSCLC) with brain metastases planned for palliative whole brain radiotherapy in a prospective cohort study (CTRI Number: CTRI/2013/01/003299). As a part of this study, patients received usual care and underwent quality of life assessment using the EORTC QLQ C30. The objective of the current study is to ascertain if the patient reported functional status as ascertained using the EORTC QLQ C30 provides additional prognostic information over and above KPS.

Materials and methods

One hundred and forty (140) consecutive patients with NSCLC with brain metastases were enrolled in this IRB approved prospective cohort study after written informed consent (Clinical Trial Registry of India number: CTRI/2013/01/003299). The study was conducted between June 2012 to April 2015 at a tertiary cancer centre in India. The quality of life (QOL) assessments and mini-mental state examination (MMSE) [11] was performed at baseline and information regarding traditional prognostic variables was recorded. QOL was assessed using the EORTC QLQ C-30 [1], LC-13 [4], and BN-20 questionnaires [21]. All questionnaires were self-reported. All patients underwent palliative whole brain radiotherapy (WBRT) to a dose of 20 Gy in five fractions over one week. Similar QOL and MMSE assessments were also performed at subsequent follow-up. Data regarding baseline quality of life scores were available for all patients.

For the purpose of this study, we calculated the baseline values of the functional scales of the EORTC QLQ C30 questionnaire. The calculation was done as per the methodology suggested by the EORTC QLQ scoring manual [10]. This involved calculation of the raw score for each scale by obtaining the average score for all the items in the scale. This was followed by linear transformation of the raw score to obtain a score ranging between 0 and 100. For each functional scale, increasing scores represented a better functioning. As KPS primarily focusses on the physical functioning, we chose the physical- and role-functioning scales for constructing a Patient-Reported Performance Scale (PRPS). The seven items which are a part of this PRPS are shown in Table 1.

Table 1.

Items taken from the EORTC QLQ C30 used for calculating the PRPS.

Q No.	Item	Scale
1	Do you have any trouble doing strenuous activities, like carrying a heavy shopping bag or a suitcase?	Physical
2	Do you have any trouble taking a long walk?	Physical
3	Do you have any trouble taking a short walk outside of the house?	Physical
4	Do you need to stay in bed or a chair during the day?	Physical
5	Do you need help with eating, dressing, washing yourself or using the toilet?	Physical
6	Were you limited in doing either your work or other daily activities?	Role
7	Were you limited in pursuing your hobbies or other leisure time activities?	Role

The scale to which each item belongs is also indicated. Items taken from the EORTC QLQ C30 questionnaire.

The PRPS was the average of the physical and role functioning scale scores. No further score transformation was done.

PRPS = (Physical Functioning Scale Score + Role Functioning Scale Score)/2 .... Equation 1.

The above calculation methodology was adopted, as it retained the original scale definition as well as the original score calculation method as defined by the EORTC. However, we also used another method of calculation where the PRPS was derived from the seven questions directly using the formulae:

PRPS Raw = (Q1+Q2+ .... +Q7)/7 .......(2)

PRPS = [1 - ((PRPS Raw-1)/3)]x100 .... (3)

However, the PRPS calculated using this alternate methodology did not alter the essential results in the study (results not shown).

Prior to analysing the utility of PRPS in predicting prognosis, we ascertained the degree of agreement between the PRPS and the KPS. Traditional measures to define agreement like the weighted kappa score are not useful in this setting as the two scales are different. Hence, we used polychoric correlation as a measure of agreement. Polychoric correlation allows estimation of agreement between two ordinal scales measuring an underlying latent construct which is assumed to be continuous [12]. After quantifying the degree of agreement between the KPS and PRPS, we ascertained the impact of the addition of PRPS as a prognostic factor. In order to ascertain the successive nested, cox proportional hazards models were fit. These models were as follows:

Full model: age, gender (male/female), number of brain metastases (1–3 or 3 or more), extracranial disease (present/absent), epidermal growth factor receptor (EGFR) mutation (mutated/wild type/not tested), KPS and PRPS.

KPS Model: Same as model 1 except PRPS was excluded.

PRPS Model: Same as model 1 except KPS was excluded.

Proportional hazards and linearity assumptions were checked for all models. In order to check for nonlinearity, continuous variables in the model (age, PRPS and KPS) were expanded using restricted cubic splines with four knots and ANOVA test was used to determine if linearity assumptions needed to be relaxed. The likelihood ratio test was used to determine whether the addition of PRPS or KPS resulted in a better model fit, with a p value of < 0.05 considered as statistically significant. Thus, the likelihood ratio test when comparing KPS model to full model, estimated if the inclusion of PRPS resulted in significantly better fit as compared to the KPS model. Similarly, the likelihood ratio test when comparing PRPS model to the full model, estimated the benefit of the addition of KPS to the PRPS model. A model adequacy index was also calculated, which is the ratio of the likelihood ratio of the subset model to that of the full model [16]. An adequacy index of 1 indicates that the prognostic information in the subset model is same as that of the full model, or in other words inclusion of the additional covariate is not needed.

Model discrimination was checked using the Harrell’s concordance index [16], time-dependent receiver operating curves (ROC) and the area under curve (AUC) calculated at various time points. The ROC estimates were derived using cumulative case/dynamic control method based on inverse probability-of-censoring weights as proposed by Uno et al [37]). Risk assessment plots were generated and Integrated Discrimination Index (IDI) were calculated using the method proposed by Pickering et al [26] using mortality prediction estimates at six months. Ninety-five per cent confidence intervals of the IDI were also calculated using 1000 bootstraps. Model calibration at six months (182.5 days) was checked using calibration plots. Model discrimination and calibration statistics were internally validated using bootstrapping (1000 bootstrap samples). Full details of the analytic methodology and accompanying analysis with comments are available in Appendix.

All analyses were conducted using R (version 3.3.3, Vienna, Austria) [27] in RStudio IDE (version 1.0.136, RStudio, Inc., Boston, MA, USA). Packages used for the analysis were polycor, rms, pec, rap, survAUC and survminer (cited in Appendix).

Results

Patient characteristics

The demographic and disease-related characteristics of the patient population are described in Table 2. The database was closed for analysis in August 2016, by which time, 111 patients (79.3%) had died. The median overall survival calculated from the date of completion of WBRT was 166 days (95% CI: 108–242 days). All deaths were related to disease progression. The 30 day, 60 day and 120 day survivals were 92%, 79% and 55% (Figure 1). Assuming a rule of thumb of 10 events per variables, 111 events gave us sufficient power to examine 11 variables in a prognostic model [24, 25].

Table 2.

Basic demographic- and disease-related characteristics of the entire population.

Variables	Category	Number ( Total = 140)
Age	Median (IQR)	54 (45.8–60) years
Gender	Male	85 (60.7%)
	Female	55 (39.3%)
KPS	< 70	39 (27.9%)
	70–80	86 (61.4%)
	90	15 (10.7%)
Comorbidities	Yes	28 (20.0%)
	No	112 (80.0%)
Pathology	Adenocarcinoma	135 (96.4%)
	Non-adenocarcinoma	5 (3.6%)
Symptomatic brain metastases	Yes	127 (90.7%)
	No	13 (9.3%)
Motor deficits	Yes	26 (18.6%)
	No	114 (81.4%)
Cranial nerve palsy	Yes	13 (9.3%)
	No	127 (90.7%)
Number of brain metastases	1	6 (4.3%)
	2	25 (17.9%)
	3	12 (8.6%)
	4 or more	97 (69.3%)
RPA class	I	8 (5.7%)
	II	96 (68.6%)
	III	36 (25.7%)
EGFR mutation	Present	40 (28.6%)
	Absent	50 (35.7%)
	Not tested	50 (35.7%)
MMSE score category	No cognitive impairment (24–30)	72 (51.4%)
	Mild impairment (18–23)	41 (29.3%)
	Severe impairment (0–17)	27 (19.3%)

Figure 1.

Kaplan–Meier survival curve for the entire population with 95% confidence intervals of the estimate. The number at risk are represented at each time interval are represented below the curve.

Figures in parentheses represent the percentage of patients in the given category for categorical variables and interquartile range (IQR) for continuous variables.

EGFR: epidermal growth factor receptor; IQR: interquartile range; KPS: Karnofsky Performance Status; MMSE: Mini-mental Status Examination Score Category; RPA: recursive partitioning analysis,

Agreement between PRPS and KPS

The polychoric correlation coefficient between KPS and PRPS was 0.46 (standard error: 0.07) indicating that only a modest degree of agreement existed between KPS and PRPS. The test for bivariate normality was not significant (Chi-square = 159.8, df = 229, p = 0.9998) indicating that the assumptions of polychoric correlation were not violated. As can be seen in Figure 2, there is a wide variability between the PRPS and KPS. The important prognostic variables which have been included in the model did not seem to influence the correlation between PRPS and KPS (Figure 2 and Appendix). Patients with a poorer cognitive function in general had a better agreement between the KPS and PRPS scores, though the degree of agreement was still modest with the highest polychoric coefficient of 0.67 for patients with poor cognitive function as defined by the MMSE test (Appendix).

Figure 2.

Scatter plots of PRPS score versus KPS for the patients in the study. Points have been color coded as per the important prognostic variables (age, sex, number of lesions, the presence of extracranial disease, EGFR status) and the RPA category. Lines represent linear regression fits and shaded bands the 95% confidence intervals of the same. In order to demonstrate the effect of age, we have categorised the age into two categories, > 65 years and ≤ 65 years. Lesions = Number of brain lesions (1-3 vs. > 3), ECM: extracranial disease, EGFR: epidermal growth factor receptor; RPA: recursive partitioning analysis class. Individual subset correlation coefficients can be seen in the Appendix.

Comparative evaluation of prognostic models

The three pre-specified models are presented in Table 3 and as can be seen, in the full model as well as the PRPS model, PRPS was found to be a significant predictor of survival. While KPS was a significant predictor of survival in the KPS model, it was not significant in the full model when PRPS was also included. Model assumptions were checked as shown in Appendix.

Table 3.

Showing the model coefficients expressed as hazard ratios with 95% confidence intervals of the same shown inside parentheses.

Variable	Contrast	Full model	KPS model	PRPS model
Age	40:70	1.28 (0.71–2.32)	1.21(0.67–2.16)	1.34(0.75–2.39)
Gender	Male:Female	1.78 (1.17–2.71)*	1.87(1.23–2.84)*	1.72(1.14–2.62)*
Extracranial metastases	No:Yes	1.50(0.97–2.34 )	1.51(0.97–2.35)	1.53(0.99–2.34)
Number of lesions	> 3:1–3	1.46(0.95–2.22)	1.47(0.96–2.33)	1.46(0.96–2.23)
EGFR mutation	+ve:-ve	0.67(0.41–1.12)	0.77(0.47–1.26)	0.64(0.39–1.06)
EGFR mutation	NA:-ve	1.50(0.95–2.36)	1.60(1.01–2.51)*	1.45(0.93–2.30)
KPS	90:70	1.26 (0.90–1.79)	1.56(1.15–2.11)*	NA
PRPS Score	80:40	1.41(1.05–1.88)*	NA	1.55(1.21–2.00)*
LR Chi2		37.15	31.81	35.37

Contrasts have been depicted for continuous variables for a better understanding of the hazard ratios. Statistically significant variables in the model are indicated with*. p values < 0.05 is taken as significant and all coefficients have been rounded to two decimal places.

EGFR: epidermal growth factor receptor; KPS: Karnofsky performance status; LR Chi2: likelihood ratio chi-square value; PRPS: patient-reported performance scale. See Appendix for the graphical representation and the summary statistics for the individual models.

The likelihood ratio (LR Chi2) test comparing the full model and the KPS model was significant indicating that addition of PRPS improved the goodness to fit (LR Chi2 = 5.34, p = 0.02*). However, the likelihood test comparing the full model and the PRPS model was not significant, indicating that addition of KPS did not improve the goodness to fit significantly (LR Chi2 = 1.77, p = 0.18) when PRPS was already present in the model. The model adequacy index for the PRPS model was 95.2%, while for the KPS model it was 85.6%. In other words, the PRPS model explained about 95% of the predictive information contained in the full model, while the KPS model could explain only 85% of the same.

The Harrell’s C-Statistic were 0.69, 0.68, and 0.68 for the full model, KPS model, and PRPS model respectively. Optimism corrected values (using 1000 Bootstrap samples) of the C-statistic were 0.66, 0.65, and 0.65, respectively. The time-dependent cumulative case/dynamic control-integrated AUC values for the three models were 0.72, 0.72, and 0.71 for the full, KPS and PRPS models, respectively. Figure 3 shows that actual time-dependent AUC values for the three models at different time points. As can be seen, the KPS model has slightly better discrimination ability at shorter follow-up times, while the converse is true about the PRPS model. However, on the whole, the discriminative ability of the PRPS model is nearly similar to that of the full model at all time points.

Figure 3.

Line plots showing the time-dependent cumulative case/dynamic control ROC values for the three models plotted from times 30 days to 300 days.

The findings were confirmed in the risk assessment plots (RAP) and the values of integrated discrimination improvement (IDI) (Figure 4 and Appendix). While the addition of PRPS to a model with KPS resulted in a slight improvement in the discrimination between non-events, there was no difference in discrimination for patients with events. On the other hand, addition to KPS to a model with PRPS does not result in any discernible improvement in the predictive ability for either patients with or without events.

Figure 4.

Risk assessment plots of the KPS model versus the full model (Panel A) and the PRPS model versus the full model (Panel B). IDI: integrated discrimination improvement. 95% CI: 95% confidence intervals of the estimate. A higher IDI indicates a better model predictive performance. Plots drawn from mortality estimates at six months.

Bootstrap optimism–corrected calibration plots of the three models showed a good fit with mean absolute errors and 0.9 quantile of the absolute error being 0.035 (0.07), 0.036 (0.06), and 0.037 (0.08) for the full model, KPS model, and PRPS model, respectively. Model calibration plots are shown in the Appendix.

Discussion

As a construct, performance status (PS) has both a subjective and an objective domain, but the current methods of assessment in the clinic use mainly subjective measures [19]. It is, therefore, surprising that physicians continue to rate the patient’s performance status, while other subjective issues, like pain and quality of life, are usually patient reported. The EORTC QLQ C30 has seven questions that deal with the physical and role function, and these questions are more specific as compared to traditional PS assessment methods like KPS or ECOG scales. Hence, intraobserver variability is likely to be less as compared to traditional PS assessment as the subjectivity arising out of physician assessment would be eliminated [35].

Guzelant et al have previously shown that of all the scales in the EORTC QLQ C30 questionnaire, physical functioning and role functioning scales have the strongest correlation with KPS [15]. In the current study, we demonstrate that a composite score of the physical and role functioning score (PRPS) provides valid prognostic information over and above KPS and may be used instead of KPS with little loss in model predictiveness and accuracy.

Previous studies evaluating physician- and patient-reported performance scales have shown significant disagreement between the two [3, 5, 6, 17, 22]. As shown in this study, there is only a modest agreement between patient-reported functional scores and physician-reported PS. In addition to inter-observer variability in rating PS [3], it is known assessment of PS is influenced by physician bias. For example, Broderick et al have demonstrated that younger patients are generally assigned more favourable PS scores by physicians [7]. This is despite the fact that PS does not correlate well with comorbidity and the disease stage [9].

While the current study demonstrates that PRPS can substitute for KPS without loss of prognostic information, results from previous studies have been conflicting. For example, Ando et al have reported that oncologist rated PS scores best fit the observed survival [3]. However, unlike the current study, the prognostic model comparisons were not done based on nested models and robust internal validation or calibration was not performed.

One of the concerns regarding the use of patient performance status has been that patients tend to rate their own performance status poorly [3, 8, 22, 29]. The reason behind this are poorly understood but may be related to depression [17] or a subconscious desire to seek help [3]. The use of patient-reported PS can result in almost half the patients excluding themselves from clinical trials where PS is a part of the inclusion criteria [8]. The use of PRPS as used in the present study can be a way around, wherein the patient-reported scores are used for determining prognosis, while physician-determined PS is used to determine clinical trial eligibility. Further use of specific questions as used in the EORTC QLQ C30 may reduce the possibility of patients rating their performance status artificially lower [35].

As Suh et al. have reported composite PS derived from a patient reported review of systems, can be obtained in a longitudinal fashion in the clinic and longitudinal changes in this influence prognosis [35]. Further, unlike the findings reported by Suh et al, we found that baseline PRPS was a significant and independent predictor of survival [35].

Our current study has several strengths in this regard. It is prospectively conducted in a cohort of patients with a single disease site. Consecutive patients were recruited to minimise selection bias. As all patients received the same radiotherapy treatment variability was also minimised which may have been a factor in previous studies [3, 5]. Follow-up was also complete and adequate. The final event (death) was observed in more than 80% of the patients. Baseline patient-reported QOL data were complete as was information regarding other prognostic variables. Further, all these information was collected in a standardised proforma, which further minimised interobserver variability. Hence, this data set was ideal to compare and contrast prognostic models that employ baseline predictive factors. The findings from the study lend further credence to the observation that patient-reported outcome measures are better predictors of survival as compared to traditional PS [14].

The limitations of the study include the fact that it was conducted in a single centre and most patients had an adenocarcinoma histology. Hence, applicability of this model to patients with other histologies and brain metastases from other sites may be limited. Nonetheless, performance status is a part of all diagnosis-specific GPA indices [33], which is an indicator of its importance in patients with brain metastases irrespective of the site of origin.

Further all patients in this study had received whole brain radiotherapy. Patients who received stereotactic radiotherapy for brain metastases, usually have a lower intracranial disease burden and hence likely to have better performance status. However, even in this setting, a score like the PRPS is likely to discriminate better between subtle grades of functional impairments as compared to the ECOG and KPS scales. As shown in the appendix, patients with better MMSE scores had a poorer correlation between the KPS and PRPS scale.

Patients’ assessment of their own HRQoL assessments is influenced by several factors like comorbidities, cognitive function, and response bias. However, the same issues affect assessment of a subjective domain like functional status when done by other observers. Eliminating other observers from the assessment process, has to potential to reduce inter-observer bias as the patient directly reports his/her functional status. In this regard, the finding that patients with poorer cognitive function had better agreement between the PRPS and the KPS score provides some insights. It is likely that the functional deficits produced in the presence of major cognitive deficits would have caught the attention of the treating physician resulting in better agreement in grading the functional deficits. In patients with better cognitive function (as assessed by the MMSE), the functional deficits would have been subtle and hence would have either not been noted by the physician or graded adequately using KPS.

Conclusion

Our study shows that patient functional score derived from the EORTC QLQ C30 (PRPS) was both a statistically significant and an independent predictor of mortality in patients with brain metastases. It also gave the same prognostic information as KPS in our prognostic model. There was significant disagreement between PRPS and KPS, although the same were measured at the same time point. The results of this study highlight the importance of patient reported outcome measures in patients with brain metastases and should spur further research in the use of patient-reported functional outcomes in this population.

Funding

The project was supported by an intramural grant from Tata Memorial Hospital.

Conflicts of interest

None of the authors have any conflicts of interest to declare.

Physical Function Scale

Min.	1st Qu.	Median	Mean	3rd Qu.	Max.
0	38.33	60	57.05	80	100

Role Function Scale Scores

Min.	1st Qu.	Median	Mean	3rd Qu.	Max.
0	33.33	66.67	61.31	100	100

Patient reported Performance Scale

Min.	1st Qu.	Median	Mean	3rd Qu.	Max.
0	41.67	63.33	59.18	83.33	100

FALSE	TRUE
0.4832	0.1742
Male	Female
0.4366	0.4679
1–3	> 3
0.5445	0.4117
No	Yes
0.2433	0.504
-ve	+ve	NA
0.4116	0.3498	0.5592
1	2	3
0.3271	0.3014	0.312
Mild	No	Severe
0.06933	0.5376	0.6451

Polychoric correlation coefficient for age groups

	FALSE	TRUE
type	polychoric	polychoric
rho	0.4902518	0.161044
var	0.004613493	0.05988673
n	121	19
chisq	139.7936	45.0513
df	224	44
ML	FALSE	FALSE

Polychoric correlation coefficient for gender

	Male	Female
type	polychoric	polychoric
rho	0.4564041	0.4849123
var	0.007279636	0.01055285
n	85	55
chisq	133.7854	97.89713
df	189	159
ML	FALSE	FALSE

Polychoric correlation coefficient for number of lesions

	1–3	> 3
type	polychoric	polychoric
rho	0.5270248	0.445308
var	0.01156839	0.006553012
n	43	97
chisq	82.61368	135.9663
df	111	204
ML	FALSE	FALSE
No	Yes
0.1745	0.5164

Polychoric correlation coefficient for EGFR receptor status

	-ve	+ve	NA
type	polychoric	polychoric	polychoric
rho	0.4578586	0.3712709	0.5545893
var	0.01312181	0.01860321	0.008961416
n	50	40	50
chisq	89.13907	74.80448	97.55861
df	103	119	154
ML	FALSE	FALSE	FALSE

Polychoric correlation coefficient for RPA class

	1	2	3
type	polychoric	polychoric	polychoric
rho	0.2853132	0.2990349	0.3232679
var	0.1476513	0.009558001	0.02769103
n	8	96	36
chisq	16.84401	126.7852	48.66904
df	13	199	87
ML	FALSE	FALSE	FALSE

Polychoric correlation coefficient for cognitive impairment defined by MMSE score

	Mild	No	Severe
type	polychoric	polychoric	polychoric
rho	−0.02262085	0.5264594	0.6680286
var	0.02782513	0.007320261	0.01047318
n	41	72	27
chisq	89.31905	84.63891	58.79614
df	107	119	109
ML	FALSE	FALSE	FALSE

C-index for the Full Model

original	corrected
0.685	0.658

C-index for the KPS Model

original	corrected
0.675	0.649

C-index for the PRPS Model

original	corrected
0.677	0.653

Integrated Discrimination Improvement

Variable	KPS versus full model	PRPS versus full model
IDI events	−0.001 (−0.0147 to 0.0127)	0.0041 (−0.0034 to 0.0117)
IDI non-events	0.0289 (0.01 to 0.0478)	0.0037 (−0.008 to 0.0154)
IDI	0.0279 (0.0049 to 0.051)	0.0079 (−0.0063 to 0.022)
Relative IDI (%)	17.61 (−2 to 37.21)	4.41 (−4.42 to 13.24)