Evaluation of clinicopathological factors in PD-1 response: derivation and validation of a prediction scale for response to PD-1 monotherapy.

BACKGROUND: Anti-PD-1 therapy has shown significant clinical activity in advanced melanoma. We developed and validated a clinical prediction scale for response to anti- PD-1 monotherapy.
METHODS: A total of 315 patients with advanced melanoma treated with pembrolizumab (2 or 10 mg kg-1 Q2W or Q3W) or nivolumab (3 mg kg-1 Q2W) at four cancer centres between 2011 to 2013 served as the setting for the present cohort study. Variables with significant association to response on a univariate analysis were entered into a forward stepwise logistic regression model and were given a score based on ORs to calculate a clinical prediction scale.
RESULTS: The developed clinical prediction scale included elevated LDH (1 point), age <65 years (1 point), female sex (1 point), history of ipilimumab treatment (2 points) and the presence of liver metastasis (2 points). The scale had an area under the receiver-operating curve (AUC) of 0.73 (95% CI 0.67, 0.80) in predicting response to therapy. The predictive performance of the score was maintained in the validation cohort (AUC 0.70 (95% CI 0.58, 0.81)) and the goodness-to-fit model demonstrated good calibration.
CONCLUSIONS: Based on a large cohort of patients, we developed and validated a simple five-factor prediction scale for the clinical activity of PD-1 antibodies in advanced melanoma patients. This scale can be used to stratify patients participating in clinical trials.

The treatment of melanoma has been transformed in recent years by novel targeted therapies and immunotherapy. Namely, molecularly targeted combinations for patients with BRAF mutations, such as dabrafenib plus trametinib or vemurafenib plus cobimetinib, and immunotherapies, such as the CTLA-4 blocking antibody ipilimumab (Robert ) and the PD-1 blocking antibodies nivolumab and pembrolizumab, have significantly improved survival for patients with advanced melanoma (Robert ; Ribas ).

The PD-1 receptor is an inhibitory receptor expressed on activated T and B cells. Its primary ligand, PD-L1, is frequently expressed within the tumour microenvironment such as cancer cells and tumour-infiltrating macrophages. The second ligand, PD-L2, is preferentially expressed by antigen presenting cells (Blank ; Pardoll, 2012; Sullivan ).

In the initial phase I clinical trial with nivolumab, Brahmer reported dramatic clinical activity with tumour regressions in several tumour types: colon, renal, lung cancers, and melanoma. Pembrolizumab was initially evaluated in a large phase I trial (KEYNOTE-001). In a pooled analysis of 411 advanced melanoma patients, the response rate was 34% after a median follow-up of 18 months and was maintained in 81% of those patients with a median overall survival of 25.9 months (Hamid ; Deeks, 2016; Ribas ). Results from two other trials, KEYNOTE-002 and KEYNOTE-006, have since been reported (Ribas ; Robert ). Two pembrolizumab dosages (2 or 10 mg kg−1 every 3 weeks) were compared with investigator's-choice chemotherapy in patients with ipilimumab-refractory advanced melanoma in KEYNOTE-002. Pembrolizumab demonstrated superior progression-free survival compared with chemotherapy (Ribas ). Pembrolizumab (10 mg kg−1 every 2 or every 3 weeks) was also compared with ipilimumab (3 mg kg−1 every 3 weeks) in 834 advanced melanoma patients in KEYNOTE-006 and found significantly improved overall survival compared with ipilimumab (Robert ). Additionally, nivolumab (3 mg kg−1 every 2 weeks) was compared with dacarbazine (1000 mg kg−1 every 3 weeks) in a phase 3, double-blind, randomised control trial in 418 untreated patients and was associated with significant improvement in overall survival and progression-free survival (Robert ). These trials have established PD-1 antibodies as an important treatment modality for melanoma.

Although PD-1 blockade represents a major step forward, many patients still fail to respond to this drug. The mechanisms underlying this clinical heterogeneity are currently unknown and may be associated with both tumour intrinsic and extrinsic factors (Blank ; Pitt ). Thus the identification of a clinical predictive model for response is critical due to the following reasons: (1) To allow patients who are unlikely to benefit from anti-PD-1 therapy to be spared from unnecessary risk of toxicity and to rationally select a combination that will better fit them (Hiniker ). (2) To spare those who are likely to respond to PD-1 monotherapy from unnecessary toxicities of a combination immunotherapy approach.

Biomarkers associated with both tumour intrinsic and extrinsic factors have been recently investigated, however, none have yet been established to clearly predict response to anti-PD-1 monotherapy (Weide ). Previous studies that assayed PD-L1 expression on tumour cells using traditional immunohistochemical approaches reported an association between high expression and clinical response to pembrolizumab (Kefford ) or nivolumab (Topalian ; Weber ; Taube ). The tumour immune phenotype has also been investigated, and CD8+ T cells have been suggested as a potential biomarker in several studies; Tumeh reported that preexisting CD8+ T cells in the tumour microenvironment were required for tumour regression after treatment with pembrolizumab. Additionally, our group has recently reported relative abundance of ‘exhausted' or PD-1/CTLA-4 dual-positive CD8 cells as a biomarker to predict response to anti-PD-1, using a multiparameter flow cytometry on freshly isolated melanoma samples (Daud ). Other clinical and pathological factors associated with favourable overall survival were reported recently by Weide and included: low pretreatment values of LDH, limited visceral tumour burden, high relative eosinophil count, and high relative lymphocyte count.

Although some preliminary biomarkers have been suggested, no clinical prediction scale has been created that can be used widely. To fill this gap, we sought to investigate readily available variables and develop and validate a simple clinical prediction scale for response to anti-PD-1 in advanced melanoma patients.

Materials and methods

Study design

Our study cohort consisted of 337 patients diagnosed with advanced melanoma and treated with either pembrolizumab (2 or 10 mg kg−1 Q2W or Q3W) or nivolumab (3 mg kg−1 Q2W) at four cancer centres: UCSF, UCLA, The Angeles Clinic and Research Institute (TACRI), and University Hospital of Zürich (USZ) between December 2011 to October 2013. Patients received immunotherapy as part of standard of care treatment or on the following clinical trials: KEYNOTE-001 (NCT01295827), KEYNOTE-002 (NCT01704287), KEYNOTE-006 (NCT01866319), or EAP (NCT02083484). All patients enrolled on these trials at any of the four centres were included in our study. Our final cohort consisted of 315 patients with advanced unresectable cutaneous or mucosal melanoma aged ⩾18 years treated with either pembrolizumab (N=304) or nivolumab (N=11). Patients with uveal melanoma were excluded (n=22). The final data set was divided into derivation and validation cohorts. The derivation cohort consisted of 228 patients from the following institutes: UCSF, UCLA, and USZ. The validation cohort consisted of 87 patients from TACRI (Figure 1). The study was approved by the Institutional Review Board at the University of California, San Francisco, CA, USA.

Figure 1

Study participant selection criteria.

Data collection

The electronic medical records of the 315 patients were reviewed. Data points collected included, but were not limited to, patient's demographics (sex, age), melanoma primary site, laboratory values (LDH, WBC), AJCC staging, metastatic status, Eastern Cooperative Oncology Group (ECOG) performance status, BRAF mutation status, previous therapies, and best overall response.

Treatment outcomes

The primary outcome measure was tumour response to treatment, which was evaluated using CT imaging at 12 and 16 weeks after the first infusion, and every 12 weeks thereafter. The Response Evaluation Criteria in Solid Tumors (RECIST) version 1.1 was used to define objective clinical responses by a investigator-adjudicated review. Sites of metastases were determined using CT imaging and were not limited to RECIST-defined target lesions. The primary site of melanoma was based on clinical documentation prior to enrollment. Two treatment outcome groups, responders and non-responders, were defined using CT imaging following the treatment. Responders included patients with tumour target lesion that met RECIST v1.1 criteria for complete response (>99% reduction in the tumour size) or partial response (⩾30% radiographic reduction in tumour size). Non-responders included patients with tumour target lesions that met RECIST 1.1 criteria for progression (⩾20% increase in the target lesion) or stable disease (<30% reduction or <20% increase in tumour target lesions).

Statistical analysis

Patient characteristics are summarised using means and s.d. for continuous variables and percentages for categorical variables. The final data set was divided into two parts: the derivation and the validation cohorts. The derivation cohort was used to develop the clinical prediction scale for tumour response, and the validation cohort was used to assess how well the scale predicted tumour response. LDH was analysed by the means of LDH-ratio (LDH value divided by the institutional upper limit of normal). Abnormal LDH was set at a cutoff of ‘1' (LDH ratio >1). Logistic regression was used to assess the possible association between response to anti-PD-1 and the following set of prespecified variables: age, sex, primary site, abnormal LDH, WBC, ECOG performance status, liver metastasis, lung metastasis, brain metastasis, history of ipilimumab treatment, BRAF mutation status, metastatic stage, and history of treatment with targeted therapy.

Variables with significant association on univariate analysis were entered into a stepwise logistic regression model; the P value threshold to enter/leave the model was set to 0.05. A simple and easy clinical tumour response prediction formula, which produces a score for each subject ranging from 0 to 7, was developed using the estimated coefficients of the variables in the final model.

To assess the performance of the score, the formula was used to calculate scores for the validation cohort, and the resulting scores were used to predict tumour response. Discrimination of performance of risk index was assessed using area under the receiver-operating curve (AUC). The Hosmer–Lemeshow goodness-of-fit statistic was used to test the calibration.

Results

Derivation of clinical scale

Demographic and clinical characteristics are summarised in Table 1. The majority of the derivation cohort patients (n=228) were aged <65 years (126 patients, 55.3%) and male (148 patients, 64.9%), with M1c metastatic stage (n=133, 58.3%) and ECOG performance status 0 (157 patients, 68.9%).

Table 1

Patient characteristics of the derivation and validation cohorts

	No. (%)a
Variable	Derivation (n=228)	Validation (n=87)
Age, years
Mean±s.d.	62.5±13.1	62.1±14.3
Age <65 years	126 (55.3)	51 (58.6)
Sex
Male	148 (64.9)	62 (71.3)
Female	80 (35.1)	25 (28.7)
Primary site
Cutaneous	200 (87.7)	68 (78.2)
Mucosal	13 (5.7)	11 (12.6)
Unknown	15 (6.6)	8 (9.2)
M category (AJCC 2009)
Unresectable stage 3	4 (1.8)	3 (3.4)
M1a	42 (18.4)	8 (9.2)
M1b	49 (21.5)	11 (12.6)
M1c	133 (58.3)	65 (74.7)
ECOG performance status
0	157 (68.9)	75(86.2)
1	65 (28.5)	12 (13.8)
2	5 (2.2)	0 (0.0)
3	1 (0.4)	0 (0.0)
LDH
Normalb	150 (65.8)	49 (56.3)
Elevatedc	78 (34.2)	38 (43.7)
WBC
Mean±s.d.	7.1±3.5	6.6±1.9
BRAF mutation status
Negative	162 (72.0)	56 (65.1)
Positive	63 (28.0)	30 (34.9)
Unknown	3 (1.3)	1 (1.1)
Liver metastasis
No	160 (70.2)	66 (75.9)
Yes	68 (29.8)	21 (24.1)
Lung metastasis
No	96 (42.1)	46 (52.9)
Yes	132 (57.9)	41 (47.1)
Brain metastasis
No	178 (78.1)	71 (81.6)
Yes	50 (21.9)	16 (18.4)
Previous ipilimumab treatment
No	81 (35.5)	46 (52.9)
Yes	147 (64.5)	41 (47.1)
Previous targeted therapy
No	174 (76.3)	68 (78.2)
Yes	54 (23.7)	19 (21.8)

Abbreviations: AJCC=American Joint Committee on Cancer; ECOG=Eastern Cooperative Oncology Group; LDH=lactate dehydrogenase; WBC=white blood cell.

Values are reported as no. (%) unless otherwise indicated.

Defined as LDH ratio ⩽1.

Defined as LDH ratio >1.

Sixty-eight patients had liver metastasis (29.8%), 132 patients had lung metastasis (57.9%) and only 50 patients had brain metastasis (21.9%). Less than 50% of patients were treated previously with targeted therapy (54 patients, 23.7%). However, the majority of patients were treated previously with ipilimumab (147 patients, 64.5%). Of the 228 derivation cohort patients, 44.3% responded to anti-PD-1 therapy (101 of 228) while 55.7% did not respond (127 of 228).

Table 2 presents all variables associated with response to anti-PD-1 monotherapy on univariate analysis. Age <65 years (OR=0.42 (0.24, 0.72), P<0.001), elevated LDH (OR=0.38 (0.21, 0.69), P<0.001), female sex (OR=0.50 (0.29, 0.89), P=0.01), previous ipilimumab treatment (OR=0.35 (0.20, 0.62), P<0.001) and liver metastasis (OR=0.34 (0.17, 0.62), P<0.001) were significantly associated with lower response to anti-PD-1 therapy. The following variables were tested and found not to be associated with response to anti-PD-1 on univariate analysis: BRAF status (OR=0.76 (0.42–1.37, P=0.37), brain metastasis (OR=0.88 (0.47–1.67), P=0.71), lung metastasis (OR=1.6 (0.94–2.80), P=0.08), WBC count (OR=0.99 (0.47–2.10), P=0.72) and metastatic status (M1b; OR=1.93 (0.83–4.5) P=0.12, M1c; OR=0.85 (0.42–1.73), P=0.66, 3c; OR=1.33 (0.17–01.4), P=0.78).

Table 2

Variables significantly associated with response to anti-PD-1 on univariate analysis

Source	Objective response rate (ORR),a %	Odds ratio (OR) (95% CI)	P value
Total cohort	44.3	NA	NA
Age <65 years	34.9	0.42 (0.24, 0.72)	<0.001
Elevated LDHb	29.5	0.38 (0.21, 0.69)	<0.001
Female sex	33.8	0.50 (0.29, 0.89)	0.01
Previous ipilimumab treatment	35.4	0.35 (0.20, 0.62)	<0.001
Liver metastasis	26.5	0.34 (0.17, 0.62)	<0.001

Abbreviations: CI=confidence interval; LDH=lactate dehydrogenase; NA=not applicable.

ORR was defined as the percentage of patients with a complete or partial response.

Defined as LDH ratio >1.

Table 3 presents the Cox regression analysis and the clinical scoring model for response to anti-PD-1 therapy. In the forward stepwise logistic regression model, female sex (OR=0.51 (0.27, 0.94), P=0.03), age <65 years (OR=0.55 (0.30, 0.98), P=0.04), previous ipilimumab treatment (OR=0.38 (0.20, 0.69), P<0.001), elevated LDH (OR=0.48 (0.25, 0.90), P=0.02) and liver metastasis (OR=0.34 (0.17, 0.66), P<0.001) were found to be significant.

Table 3

Cox regression analysis and clinical scoring model for response to anti-PD-1 therapy

Variable	Odds ratio (OR) (95% CI)	P value	Scoring
Female	0.51 (0.27, 0.94)	0.03	1
Age <65 years	0.55 (0.30, 0.98)	0.04	1
Previous ipilimumab treatment	0.38 (0.20, 0.69)	<0.001	2
Elevated LDHa	0.48 (0.25, 0.90)	0.02	1
Liver metastasis	0.34 (0.17, 0.66)	<0.001	2

Abbreviations: CI=confidence interval; LDH=lactate dehydrogenase.

Defined as LDH ratio >1.

A formula (clinical prediction scale) based on the estimated coefficients from the logistic model was derived so that a score ranging from 0 to 7 was calculated for each subject. An increase in the score on the scale corresponded to a significantly lower response to anti-PD-1 therapy (P<0.05). The final clinical prediction scale included 5 prognostic factors that were significantly correlated with lower response: female sex (1 point), age <65 years (1 point), previous ipilimumab treatment (2 points), elevated LDH (1 point), and the presence of liver metastasis (2 points). The developed scale had a fair performance in predicting response to anti-PD-1: AUC 0.73 (95% CI 0.67, 0.80) (Figure 2A).

Figure 2

Receiver operation characteristic curve to predict response to immunotherapy.(A) Derivation cohort, area under the curve (AUC) 0.73 (95% CI 0.67–0.80). (B) Validation cohort, AUC 0.70 (95% CI 0.58–0.81).

Validation of clinical prediction scale

Our scoring model was validated in a separate cohort of patients. The validation cohort included 87 patients diagnosed with advanced melanoma at TACRI. Of the 87 validation cohort patients, 63.3% responded to anti-PD-1 therapy (55 of 87), whereas 36.7% did not respond (32 of 87). The scale maintained a discrimination power, AUC 0.70 (95% CI 0.58, 0.81) (Figure 2B). Table 4 presents the validation of the clinical prediction scale by response probability. An increase in the score on the scale corresponded to a lower probability of response; the estimate probability of response for patients who scored ‘6' was 29% (95% CI 13, 52) and the estimate probability of response for patients who scored ‘0' was 87% (95% CI 71, 95). None of the patients in the validation cohort scored ‘7'.

Table 4

Validation of clinical prediction scale, response probability (n=87)

	Response, n (%)
Response score	No	Yes	Total	Estimated response probability (%) (95% CI)
0	1 (12.5)	7 (87.5)	8 (100)	87 (71, 95)
1	4 (23.5)	13 (76.4)	17 (100)	81 (66, 90)
2	3 (17.6)	14 (82.3)	17 (100)	72 (60, 82)
3	4 (40.0)	6 (60.0)	10 (100)	62 (51, 72)
4	14 (60.8)	9 (39.1)	23 (100)	51 (37, 64)
5	2 (28.5)	5 (71.4)	7 (100)	39 (24, 58)
6	4 (80.0)	1 (20.0)	5 (100)	29 (13, 52)
7a	NA	NA	NA	NA
Total	32 (36.7)	55 (63.2)	87 (100)	NA

Abbreviations: CI=confidence interval; NA=not applicable.

None of the patients in the validation cohort scored 7.

Discussion

We developed and validated a clinical prediction scale for response to anti-PD-1 monotherapy. The response index had good predictive performance that was maintained in the validation cohort with good calibration. The prediction scale to response to anti-PD-1 was developed using stepwise logistic regression and included five variables that were significantly associated with lower response to anti-PD-1: female sex, age <65 years, history of ipilimumab treatment, elevated LDH, and the presence of liver metastasis.

Although anti-PD-1 therapy has transformed the systemic treatment of advanced melanoma, the majority of patients still fail to respond to anti-PD-1 monotherapy; the clinical data indicates that the response rates for patients treated with anti-PD-1 immunotherapy rarely exceed 45% (Brahmer ; Hamid ; Ribas ). In this study, the response rate of the derivation cohort was about 44.3%, which is consistent with the literature.

As understanding and exploiting this clinical heterogeneity could result in better outcomes, biomarker research remains a very active area of clinical investigation, and characteristics of the tumour microenvironment have been shown to predict response to immunotherapy (Tumeh ; Blank ; Daud ; Pitt ). Assays for PD-L1 upregulation and characterisation of a partially exhausted T-cell phenotype, for example, have been strongly associated with effective response to anti-PD-1 (Herbst ; Taube ; Tumeh ; Daud ).

To date, however, there is less understanding of the impact of host factors on response to PD-1 blockade. A recent study investigating baseline biomarkers predicting outcomes of pembrolizumab-treated melanoma patients reported a better prognosis for patients with high relative eosinophil count, high relative lymphocyte count, low LDH and the absence of metastasis other than soft tissue/lung (Weide ). Additional characteristics, such as age, diet, germline polymorphisms, pre-existing immunosuppression and chronic infections, and the gut microbiome have been also proposed to influence response to immune checkpoint inhibition (Pitt ; Spranger ).

The effect of age, in particular, has been an area of debate. An aging, senescent immune system has been long thought to experience many alterations, including an increase in suppressive immune cell populations, such as MDSCs and Treg cells (Pawelec ; Goronzy and Weyand, 2013), and MDSCs have been reported as potential predictors of resistance to ipilimumab therapy (Meyer ; Martens ), which would suggest a similar effect in anti-PD-1-treated patients.

Interestingly, a recent meta-analysis (Nishijima ) comparing efficacy of immune checkpoint inhibition in younger and older patients showed overall survival benefit in both groups, although an exception was seen in a subgroup of older patients in four anti-PD-1 trials who did not demonstrate OS improvement. Although this is different than our findings that younger age is correlated with decreased anti-PD-1 efficacy, it is interesting to note that the age cutoff used for this analysis was ⩾75 years. Although there was no OS benefit observed in patients aged ⩾75 years, a consistent survival benefit was still observed in patients aged 65–75 years. Although this observation was made at the meta-analysis level and across a heterogeneous group of clinical trials and needs to be confirmed with adequate power, this preliminary data seems to suggest that patients over a certain age do just as well as their younger counterparts and, in light of our scoring model correlating greater anti-PD-1 efficacy in patients aged ⩾65 years, potentially may have even better response. This observation is certainly hypothesis generating and needs further prospective validation.

The role of sex in response to immunotherapy is also much debated. Although female sex was found to be a negative prognostic indicator in our model, the mechanisms of this remain poorly defined. Indeed, much of the current literature actually supports a survival benefit for females with advanced melanoma, although of note, much of this data preceded the advent of immune checkpoint inhibition (Manola ; Dronca ; Nosrati and Wei, 2014). Though it is speculated that the increased number and/or improved function of tumour-specific T-helper cells in women could underlie this clinical observation, data regarding the role of sex-hormone modulation of PD-1 are currently limited to preclinical models (Polanczyk , 2007). Further work of this nature needs to be carried out to identify and elucidate the nature of sex-dependent differences in immunoregulatory pathways in patients with advanced cancer (Dronca and Dong, 2015).

Given the relative novelty of this field, the influence of host factors on immunotherapy response requires further research. The factors elucidated in this clinical scale represent one further step in that direction, while also addressing the pressing clinical need for a clinical prediction-scoring model to help guide treatment choice. A model to identify those patients who are likely to respond to anti-PD-1 monotherapy would spare them the additional toxicity of combination CTLA-4/PD-1 blockade; conversely, the ability to identify patients who are unlikely to respond to PD-1 monotherapy would also be helpful in determining the most appropriate treatment option. Those patients, for example, may have improved outcomes with molecularly targeted agents (if their melanomas harbour BRAF V600 mutations) or with a more aggressive immunotherapy regimen (combination ipilimumab/nivolumab) or a clinical trial, rather than with an anti-PD-1 agent alone.

Although our study included a large cohort of patients from four different centres, considering the retrospective nature of this study, a patient selection bias cannot be excluded. Additionally, even though our scoring model had a fair prediction probability for response in the validation cohort, this is based on a relatively small sample of patients, and prospective validation with a larger sample is needed. Additionally, the high response rate in our validation cohort (63.3%) could have affected the ROC reported in our study. Furthermore, the effect of active brain metastasis on patients' response was not studied as the trials included in our study excluded patients with active brain metastasis. Finally, we could not study the effect of pretreatment with ipilimumab in a separate independent model owing to a low number of ipilimumab-naive patients. However, in the four centres included in our study, treatment with pembrolizumab was initiated in a minimum of 6 weeks following the last ipilimumab treatment.

To our knowledge, this is the first clinical scoring model developed to predict response to anti-PD-1 monotherapy. This scoring model was developed in a large cohort of patients who were treated with anti-PD-1 monotherapy and validated in a unique set of patients from a separate facility in an effort to strengthen its external validity. Although biomarker development in immunotherapeutics remains an active area of research, most biomarkers to date require accession of tumour tissue and complex laboratory diagnostics to predict response. With this scale, we sought to create a clinical tool that would be readily accessible in a wide range of clinical settings, without the need for additional biopsies or complex computing.

We acknowledge that our scale is only the beginning and additional data regarding host factors, and tumour characteristics from ongoing studies will greatly help in refining and modifying the current effort.