Development of the ProPal-COPD tool to identify patients with COPD for proactive palliative care.

BACKGROUND: Our objective was to develop a tool to identify patients with COPD for proactive palliative care. Since palliative care needs increase during the disease course of COPD, the prediction of mortality within 1 year, measured during hospitalizations for acute exacerbation COPD (AECOPD), was used as a proxy for the need of proactive palliative care. PATIENTS AND METHODS: Patients were recruited from three general hospitals in the Netherlands in 2014. Data of 11 potential predictors, a priori selected based on literature, were collected during hospitalization for AECOPD. After 1 year, the medical files were explored for the date of death. An optimal prediction model was assessed by Lasso logistic regression, with 20-fold cross-validation for optimal shrinkage. Missing data were handled using complete case analysis.
RESULTS: Of 174 patients, 155 patients were included; of those 30 (19.4%) died within 1 year. The optimal prediction model was internally validated and had good discriminating power (AUC =0.82, 95% CI 0.81-0.82). This model relied on the following seven predictors: the surprise question, Medical Research Council dyspnea questionnaire (MRC dyspnea), Clinical COPD Questionnaire (CCQ), FEV1% of predicted value, body mass index, previous hospitalizations for AECOPD and specific comorbidities. To ensure minimal miss out of patients in need of proactive palliative care, we proposed a cutoff in the model that prioritized sensitivity over specificity (0.90 over 0.73, respectively). Our model (ProPal-COPD tool) was a stronger predictor of mortality within 1 year than the CODEX (comorbidity, age, obstruction, dyspnea, and previous severe exacerbations) index.
CONCLUSION: The ProPal-COPD tool is a promising multivariable prediction tool to identify patients with COPD for proactive palliative care.

Introduction

COPD is a progressive life-threatening lung disease.1 During the disease course, the need for disease-oriented care remains unchanged while the need for proactive palliative care increases for patients with COPD.2 Proactive palliative care, which includes advance care planning (ACP) conversations, is intended to improve the quality of life and quality of care.3,4 However, for patients with COPD, such care is not yet common.2,5 An important barrier is the identification of patients with COPD who might benefit from proactive palliative care.2

Identification for proactive palliative care is complicated by the unpredictable disease course of COPD.6 Stable phases in the disease course are interrupted by acute exacerbations; any one of which may be fatal.6 Although, general tools to identify patients for palliative care exist, they are not specifically intended or validated for the identification of patients with COPD.7,8 In addition, models of survival in stable COPD have been developed to be able to adjust and optimize care.9,10 Unfortunately, these population models are of limited value to predict survival for individual patients.11 This prognostic difficulty has led to a discussion among clinicians whether or not a transition point for the initiation of proactive palliative care exists.12–15 Some clinicians argue that more specific criteria of end-stage COPD need to be explored.13 Others argue that, since such criteria may not exist, searching further may lead to prognostic paralysis.14,15 Instead, they promote early integration of palliative care according to needs.15 The focus in this discussion has been on whether or not we can accurately predict mortality instead of the intended objective, identifying patients with COPD in need of proactive palliative care. Since palliative care needs increase during the disease course of COPD,2 the prediction of mortality can be used as a proxy for the need of palliative care. To ensure minimal miss out of patients in need of such care, this tool should have a sensitivity near 100 with specificity as high as possible.

Hospitalizations for an acute exacerbation COPD (AECOPD) are associated with significant mortality and therefore create an opportunity to identify patients with poor prognosis in need of palliative care.16 Recently, potential relevant variables have been identified to predict posthospital mortality in patients hospitalized for an AECOPD.16,17 This is the first prospective study to incorporate these variables with the objective to develop a multivariable prediction tool (the ProPal-COPD tool) to identify patients with COPD in need of proactive palliative care. Our objective was to assess the discriminating power of a set of indicators for proactive palliative care in predicting death for any cause within 1 year. This outcome was used as a proxy for the potential need of proactive palliative care in patients with COPD.

Patients and methods

Source of data

Data for prognostic accuracy testing in patients hospitalized for an AECOPD were obtained from the PROLONG study.18 This prospective pragmatic cluster controlled trial was performed in six general hospitals in the Netherlands; of those, three were in the intervention condition and three in the control condition. All recruited patients hospitalized for an AECOPD received usual care, while only patients in the intervention condition with poor prognosis additionally received proactive palliative care. In this trial, poor prognosis was defined as meeting two or more predictors of a set of 11 predictors of poor prognosis. For the prognostic accuracy study, described in this study, we only used data of those patients recruited from the three hospitals in the control condition.

Participants

We screened for potential participants in three general hospitals in the Netherlands from January 14, 2014, to December 29, 2014. All patients with the clinical diagnosis of COPD and hospitalized for an AECOPD who were 18 years or older were eligible. An AECOPD was defined as “an acute worsening of the patient’s condition from the stable state, which is sustained and may warrant the patient to seek additional treatment.”19 Since the severity of airway obstruction and spectrum of pathogens in pneumonic and non-pneumonic exacerbations are similar,20 patients with COPD who had been hospitalized for an acute pneumonia accompanied by increased sputum and dyspnea were also included. Patients who did not speak Dutch or had a severe cognitive disorder were excluded from participation. All patients received usual care for an AECOPD and the period thereafter.

The study was approved by the Medical Ethics Committee (METC) of the Radboud University Medical Center, Nijmegen (METC protocol number 2012/260). Thereafter, we requested and received research approval from all three participating hospitals. All participants gave written informed consent before enrollment.

Outcome

The outcome measure was death for any cause within 1 year. This outcome measure was used as a proxy for the potential need of proactive palliative care. To ensure minimal miss out of patients in need of such care, we were looking for a tool with a sensitivity near 100% and a specificity as high as possible.

Predictors

All 11 predictors are presented in Table 1. Each of them is a categorical variable indicating whether or not the participant meets the specific criteria for this predictor. References with the rationale to include the predictor in the multivariable prediction model are also presented in Table 1. Patients filled out demographics, the Medical Research Council dyspnea questionnaire (MRC dyspnea) and the Clinical COPD Questionnaire (CCQ) on the sixth day of hospital stay or in case the patient was discharged earlier on the day before leaving the hospital. The MRC dyspnea is a questionnaire used for grading the effect of breathlessness on daily activities.27 The patient’s dyspnea score is rated from 1 to 5, and higher scores represent a more severe dyspnea. The patient was asked to rate the dyspnea severity as it was 2 weeks before hospital admission. The CCQ day version was used to measure the health status of patients.28 The CCQ consists of 10 questions covering three domains (symptoms, functional status and mental state). The maximum total CCQ score is 6 and higher scores represent a worse health status. The treating pulmonologist thereafter filled out a checklist consisting of the remaining nine predictors. A year after the last patient was included, the medical files of all participants were explored. Variables looked for were as follows: if applicable, date of death and missing scores of predictors on the checklist. In case of uncertainty about death and/or date of death, we contacted the general practitioner (GP) of the patient for information.

Table 1

Set of 11 predictors of poor prognosis for patients hospitalized for an AECOPD and the references with the rational to include them in the prediction model

Predictors of poor prognosis	References
1. Hypoxemia (PaO2 <8 kPa) or hypercapnia (PaCO2 >6 kPa) at discharge	16,17
2. Treatment of the exacerbation with NIV	21
3. Patient needs professional home care service for personal care after discharge	22
4. Negative answer to the surprise question: “Would I (as pulmonologist) be surprised if this patient would die in the next year?”	23
5. The diagnosis of a severe comorbidity such as:
a. Non-curable malignancy or	17,24
b. Cor pulmonale (proven or non-proven) or	16
c. Proven CHF or	16,17
d. Diabetes mellitus with neuropathy or	16,17
e. Renal failure, clearance <40 (GFR, mL/min)	16,17
6. CCQ total, day version >3	25
7. MRC dyspnea =5	16,26
8. FEV1 (measured before AECOPD) <30% of predicted	16,17
9. BMI <21 or unplanned weight loss (>10% in the last 6 months or >5% in last month)	16,17
10. Previous hospital admissions for AECOPD (last 2 years ≥2 and/or last year ≥1)	16
11. Age >70 years	16,17

Notes: PaCO2, arterial partial pressure of carbon dioxide; PaO2, arterial partial pressure of oxygen.

Abbreviations: AECOPD, acute exacerbation COPD; BMI, body mass index; CCQ, Clinical COPD Questionnaire; CHF, chronic heart failure; FEV1, forced expiratory volume in 1 second; GFR, glomerular filtration rate; MRC dyspnea, Medical Research Council dyspnea questionnaire; NIV, noninvasive ventilation.

Sample size

In this prospective study, we included all eligible patients willing to participate during the year 2014 to maximize the power and generalizability of the results.

Missing data

Missing data were handled using complete case analysis. Participant data were excluded from the analysis if a participant had missing information on any of the potential predictors. Possible reasons for missing data were not or incomplete filled-out questionnaire by the patient and/or an incomplete filled out checklist by the pulmonologist.

Statistical analyses

Baseline characteristics were described using frequencies for noncontinuous data, mean with SD for continuous variables normally distributed and median with interquartile range (IQR) for continuous variables not normally distributed. The analysis plan consisted of two steps. The first step had a descriptive nature and was aimed at exploring the discriminating power of individual predictors in predicting death for any cause within 1 year. For all 11 predictors, the area under the curve (AUC), sensitivity and specificity were calculated. The goal of the second step was to build an optimal prediction model and to assess its predictive performance using internal validation. The model was built using a least absolute shrinkage and selection operator (Lasso) logistic regression model, with 20-fold cross-validation for optimal shrinkage.29,30 By using the Lasso method, the risk of over-fitting was reduced, and at the same time a relatively simple prediction model was obtained. Because the performance of our optimal prediction model was validated in the same population used to construct it, its predictive performance was overestimated (optimism). To estimate this optimism, the procedure proposed by Steyerberg30 was followed. In short, using bootstrapping, 500 new populations were created, and a prediction model was built for each population in a completely analogous way. The predictive performance of the model, as expressed by the AUC, in both the bootstrapped population and original population was calculated. The mean difference between these two AUC values is an estimate of the optimism of our prediction model.

Risk groups

No risk groups were created.

Results

The participant flow diagram is shown in Figure 1. Of all 174 participants, 155 participants were included in the analyses. The characteristics of the study populations are presented in Table 2.

Figure 1

Participant flow diagram.

Abbreviation: Pt, patient.

Table 2

Characteristics of the study population

Characteristics	Mean	±SD
Age, years	67.5	±9.6
Pack years	36.4	±32.8
	n	%
Sex
Male	68	43.9
Marital status*
Married	110	71.0
Unmarried	11	7.1
Divorced	13	8.4
Widow	18	11.6
Condition of living*
Single	34	21.9
Non-single	111	71.6
Place of living*
Home, independent of home care	103	66.5
Home, dependent of home care	45	29.0
Residential home	2	1.3
Highest level of education*
No education	1	0.6
Elementary school	21	13.5
Secondary school	47	30.3
Primary education	35	22.6
Secondary education	29	18.7
Higher/university education	15	9.7
GOLD stage*,•
0	8	5.2
I	10	6.5
II	41	26.5
III	52	33.5
IV	36	23.2
Predictors
Hypoxemia or hypercapnia	62	40.0
NIV	11	7.1
Professional home care	50	32.3
Negative answer SQ	76	49.0
Comorbidity	31	20.0
CCQ total, day version >3	96	61.9
MRC dyspnea =5	71	45.8
FEV1 <30% of predicted	45	29.0
BMI <21 or weight loss	26	16.8
Prev hosp AECOPD	62	40.0
Age >70 years	62	40.0

Notes: Study population (N=155).

Because of missing values, some numbers do not add up to 100%.

GOLD stage according to the last known data in medical file; inclusion in the study was based on the judgment by pulmonologist.

Abbreviations: AECOPD, acute exacerbation COPD; BMI, body mass index; CCQ, Clinical COPD Questionnaire; FEV1, forced expiratory volume in 1 second; GOLD, Global Initiative for Chronic Obstructive Lung Disease; MRC dyspnea, Medical Research Council dyspnea questionnaire; NIV, noninvasive ventilation; prev hosp, previous hospitalization; SQ, surprise question.

Model development

The numbers of participants who met the criteria of each predictor are presented in Table 2. In the study population, 19.4% of the participants (30 of 155) died within 1 year.

The discriminating power of individual predictors in predicting death within 1 year is presented in Table 3.

Table 3

The discriminating power of individual predictors in predicting death within 1 year

Predictor	Se	Sp	AUC
Hypoxemia or hypercapnia	0.53	0.63	0.583
NIV	0.93	0.07	0.503
Professional home care	0.47	0.71	0.589
Negative answer SQ	0.80	0.58	0.692
Comorbidity	0.43	0.86	0.645
CCQ total, day version >3	0.83	0.43	0.633
MRC dyspnea =5	0.80	0.62	0.712
FEV1 <30% of predicted	0.43	0.64	0.589
BMI <21 or weight loss	0.33	0.87	0.603
Previous hospitalization AECOPD	0.60	0.65	0.624
Age >70 years	0.43	0.61	0.521

Note: Study population (N=155).

Abbreviations: AECOPD, acute exacerbation COPD; AUC, area under the curve; BMI, body mass index; CCQ, Clinical COPD Questionnaire; FEV1, forced expiratory volume in 1 second; MRC dyspnea, Medical Research Council dyspnea questionnaire; NIV, noninvasive ventilation; Se, sensitivity; Sp, specificity; SQ, surprise question.

Model specification and performance

The optimal prediction model, build using the Lasso method, and its predictive performance are presented in Table 4. This logistic regression model includes an intercept and the weighted sum of the value of each predictor. The weights are the regression coefficients (B). The value of each predictor is 1 when present and 0 when absent.

Table 4

Optimal prediction model and its predictive performance of death within 1 year

Predictors in model	B	AUC	True AUC	95% CI
Intercept	−3.901	0.870	0.818	0.813–0.824
Hypoxemia or hypercapnia	/
NIV	/
Professional home care	/
Negative answer SQ	0.959
Comorbidity	1.479
CCQ total, day version >3	0.257
MRC dyspnea =5	1.475
FEV1 <30% of predicted	0.565
BMI <21 or weight loss	1.005
Prev hosp AECOPD	0.102
Age >70 years	/

Notes: Study population (N=155). B, weight in the model./, not incorporated in the optimal prediction model.

Abbreviations: AECOPD, acute exacerbation COPD; AUC, area under the curve; BMI, body mass index; CCQ, Clinical COPD Questionnaire; CI, confidence interval; FEV1, forced expiratory volume in 1 second; MRC dyspnea, Medical Research Council dyspnea questionnaire; NIV, noninvasive ventilation; prev hosp, previous hospitalization; SQ, surprise question.

A patient can be considered as in need of proactive palliative care when the total sum of the logistic regression models exceeds a certain cutoff point of the model. The choice of this cutoff is essentially a trade-off between sensitivity and specificity. A popular choice is to choose the cutoff that considers sensitivity and specificity to be of equal importance, so maximizing the sum of sensitivity and specificity. In the setting of this study, a cutoff that prioritizes sensitivity above specificity does make sense. Therefore, we also present the cutoff that maximizes the sum of specificity and 1.5 times the sensitivity. Possible cutoffs for both choices are presented in Table 5.

Table 5

Possible cutoffs considering different trade-offs between Se and Sp of the optimal prediction model

Cutoff	Se	Sp	Se + Sp	1.5 Se + Sp
−1.105	0.800	0.840	1.640	2.040
−2.896	0.967	0.328	1.295	1.778
−2.169	0.933	0.504	1.437	1.904
−1.362	0.900	0.728	1.628	2.078

Abbreviations: Se, sensitivity; Sp, specificity.

The receiver operating characteristic (ROC) curve for the prediction of death within 1 year with the optimal cutoffs considering different trade-offs between sensitivity and specificity is shown in Figure 2.

Figure 2

The ROC curve for the prediction of death within 1 year with the optimal cutoffs considering different trade-offs between Se and Sp.

Abbreviations: ROC, receiver operating characteristic; Se, sensitivity; Sp, specificity.

Discussion

This study shows that we were able to develop a promising multivariable prediction tool to identify patients with COPD in need of proactive palliative care, the ProPal-COPD tool. Since palliative care needs increase during the disease course,2 and hospitalizations for AECOPD are associated with significant mortality risk,16 these hospitalizations create an opportunity for identification. This is why the prediction of death was used as a proxy for the identification of patients in need of palliative care. In our study population, 19.4% of participants died within 1 year. This number is in line with the 1-year mortality rate in other studies investigating patients hospitalized for AECOPD.31,32 Our optimal prediction model of death within 1 year had good discriminating power; the true AUC was 0.82 (95% CI 0.81–0.82), and the estimated optimism of 0.05 indicated good internal validity. To ensure minimal miss out of patients in need of palliative care, we proposed a cutoff in the prediction model that prioritized sensitivity above specificity. This optimal cutoff was −1.36, with a sensitivity of 0.90 and a specificity of 0.73. This implies that, when used, only 10% of patients in need of palliative care would be missed out and that of all patients identified for such care 84% would die within 1 year.

The ProPal-COPD tool is the first which is using specific predictors of mortality during AECOPD requiring hospitalizations. This tool relies on seven predictors: the surprise question (a general predictor), five markers of COPD severity and the presence of specific comorbidities proven to independently predict posthospital mortality in AECOPD.24 Through the surprise question, the tool incorporates clinical judgment of severity. The surprise question has shown to be a proven predictor of mortality in patients with advanced chronic diseases33 and is recommended to be used in combination with other disease-specific predictors as screening tool for proactive palliative care.23 Those disease-specific predictors in the ProPal-COPD tool are five markers of COPD severity being the CCQ (health status), MRC dyspnea, FEV1% of predicted value, body mass index (BMI) and previous hospitalizations for an AECOPD. The presence of one of the following comorbidities, non-curable malignancy, cor pulmonale, chronic heart failure (CHF) or diabetes mellitus with neuropathy or renal failure, is also incorporated in our tool. These comorbidities, except non-curable malignancy, are conditions liable to acute decompensation especially during AECOPD, which may explain their association with an increased mortality risk.16

Most predictors of the ProPal-COPD tool are variables already used in clinical practice and do not need extra measurement.34 Only the surprise question, the CCQ and MRC dyspnea, which are short easy-to-use questionnaires, are often not documented and need to be specifically asked for.34 Besides, each predictor in our tool has an own weight (B). Those weights take into account the correlation with other predictors of the tool and therefore should not be interpreted as the individual importance of each predictor in the tool. Moreover, the Lasso method used for model development is looking for the most efficient model.30 In the trade-off between efficiency and effectiveness, four predictors (hypoxemia or hypercapnia, noninvasive ventilation [NIV], professional home care and age) were excluded from our optimal prediction model. This does not mean that those predictors were not predictive of death within 1 year. If we look at the predictor NIV, for example, it was noticeable that only 11 patients met the criteria of this predictor. Besides, NIV as an individual predictor had high sensitivity (0.93) and low specificity (0.07), meaning that almost all patients who were treated with NIV died within 1 year but patients who were not treated with NIV also had a considerable risk of dying. The exclusion of the predictor NIV can therefore be explained by the limited number of patients involved and the high correlation with predictors incorporated in our optimal prediction model. The other three predictors, hypoxemia or hypercapnia, professional home care and age, were merely excluded because of correlation.

The ProPal-COPD tool shows promise for the identification of patients with COPD in need of proactive palliative care. The ROC analysis suggests that it has good prognostic performance with a prognostic capacity superior to the CODEX index (comorbidity, age, obstruction, dyspnea and previous exacerbations).35 The CODEX index has an AUC of 0.68 in predicting death within 1 year and is the only other multivariable model developed to evaluate posthospital prognosis in patients with an AECOPD.35 However, the main objective of both tools is different. Our tool was developed to identify patients in need of proactive palliative care, whereas the CODEX was developed to predict mortality and/or read-missions. In addition, there are several substantive differences. First, our tool incorporates disease-specific predictors of mortality in AECOPD requiring hospitalization, whereas the CODEX index was developed using disease-specific predictors of mortality in stable COPD. Second, our tool contains specific comorbidities proven to independently predict posthospital mortality in AECOPD, whereas the evidence of the Charlson index (comorbidity burden) used in the CODEX index in predicting posthospital mortality in AECOPD is less consistent.16 Finally, each predictor in our tool has an own weight (B) in contrast to the predictors in the CODEX index. Those weights do not represent the individual importance of each predictor since they take into account the correlation with other predictors in the tool. Still, the own contribution of each predictor is better represented in our tool. It is possible that the abovementioned substantive differences between the ProPal tool and the CODEX index have contributed to the better prognostic performance of our tool.

Our study has several limitations that should be acknowledged. First, predicting posthospital mortality for patients with COPD is complex and this is reflected by the number of potential predictors incorporated in the original set of predictors. We only used already known predictors in this set and it is uncertain if all possible predictors were accounted for. In addition, it is possible that certain predictors were under-represented in the set of potential predictors. Anxiety and depression, for example, were represented as a subscale of the predictor CCQ. Second, to facilitate implementation in clinical practice, we used dichotomized variables. Each potential predictor was given a clinical cutoff value based on literature. Such dichotomization of continuous predictors may lead to loss of information and reduction in power.36 Third, the use of the ProPal-COPD tool may require collection of data that are not always captured routinely (surprise question, CCQ, MRC dyspnea). Fourth, the ProPal-COPD tool was developed in the Netherlands. International differences in hospital admission policy for patients with an AECOPD may affect the usefulness in other countries. Fifth, it is unknown whether the ProPal-COPD tool is also applicable in academic hospitals since it was developed using a population of patients in general hospitals. Finally, the ProPal-COPD tool was developed in a single population without validation in a different one. The external validity of our findings is therefore unknown and needs to be addressed in further research. Hence, other participant data need to be collected in a new prospective study performed in general and/or academic hospitals to evaluate the performance of the ProPal-COPD tool.

Conclusion

The ProPal-COPD tool is a promising tool to identify patients with COPD for proactive palliative care with good discriminating power (AUC =0.82). It relies on seven predictors: the surprise question (a general predictor), five markers of COPD severity and the presence of specific comorbidities. The prediction of death within 1 year was used as a proxy for the potential need of proactive palliative care. To ensure minimal miss out of patients in need of proactive palliative care, we proposed a cutoff in the prediction model that prioritized sensitivity over specificity. This optimal cutoff has a sensitivity of 0.90 and a specificity of 0.73. Each predictor in the tool has an own weight and the patient can be considered in need of proactive palliative care only when the total sum of the model exceeds the specific cutoff point. This prognostic equation can be easily integrated in the electronic patient record (EPR) system. However, before clinical use of the ProPal-COPD tool, the external validity should be addressed in further research.