Patterns of response to anti-PD-1 treatment: an exploratory comparison of four radiological response criteria and associations with overall survival in metastatic melanoma patients.

BACKGROUND: Radiological assessment of response to checkpoint inhibitors remains imperfect. We evaluated individual lesion and inter-patient response by response evaluation (RECIST) 1.1, immune-related response criteria (irRC), CHOI and modified CHOI (mCHOI) and correlated response with overall survival (OS).
METHODS: Thirty-seven patients with 567 measurable lesions treated with pembrolizumab in the Keynote 001 trial were studied. Association of response with OS was determined.
RESULTS: Response varied according to site; lung lesions had the highest rate of complete response (69 out of 163 (42%) vs other sites 71 out of 404 (18%), P<0.0001). Delayed response post first scan was seen in 2 out of 37 (5%) deemed progressive (PD) by RECIST and 2 out of 14 (14%) deemed PD by irRC. Modified CHOI criteria showed response of 38% (14 out of 37). Change in tumour size and density on first follow-up assessment was associated with OS with each 1000 mm2 increase in tumour size from baseline increasing the hazard of dying by 25.9% (HR=1.259, (95% CI=1.116-1.420), P=0.0002). Similarly, each 20HU increase in density increased the HR by 15% (HR=1.15, (95% CI 1.045-1.260), P=0.004). Response defined by any criteria had superior OS (CHOI P=0.0084; mCHOI P=0.0183; irRC P<0.0001 and RECIST P=0.0003).
CONCLUSIONS: Response by any criterion was prognostic. Novel patterns of response and changes on treatment in tumour density suggest complex anti-tumour responses to immunotherapy.

The development of immune checkpoint inhibitors (ICIs) has heralded a new era in immuno-oncology (Couzin-Frankel, 2013). Therapies such as ipilimumab, pembrolizumab and nivolumab have shown impressive responses in metastatic melanoma and are FDA approved for this disease (Hodi ; Robert ; Hamid ; Robert ; Larkin ; Schadendorf ; Weber ). Numerous trials with these agents alone or in combination are underway in melanoma and a variety of other tumour types.

Clinical trials of ipilimumab, the first ICI to be licensed for metastatic melanoma, highlighted idiosyncratic patterns of response. Patients may have a standard RECIST radiological response upon treatment or progression during treatment, followed by a delayed response, possibly several months after treatment completion. This may be due to either transient tumour growth followed by an immune response or pseudo-progression, implying an inflammatory immune response leading to the increase in tumour size. Mixed responses or prolonged stable disease is additionally not uncommon. Such patterns of response provide particular challenges in managing patients on these therapies. Furthermore, traditional radiological clinical end points may not adequately capture benefit from these agents, where progression-free survival may be inaccurately assessed and overall survival (OS) is the most accurate measure of efficacy.

The immune-related response criteria (irRC) were formulated in 2009 as an alternative to response evaluation criteria in solid tumors (RECIST) to more accurately capture responders or benefit from ipilimumab (Wolchok ). All lesions are considered with overall tumour burden being assessed at each scan as opposed to defined target lesions. Progression on one scan requires confirmation on a follow-up scan during which patients may remain on treatment. Immune-related response criteria are being validated in clinical trials with ICI, where it is used along with RECIST as a corollary or exploratory end point. The CHOI criteria were proposed in 2004 (Choi ) to better capture responses in gastrointestinal stromal tumours treated with imatinib. Although decreases in tumour lesion size can be seen with this agent, the majority of patients have stable disease or tumour growth by RECIST in the initial phase of treatment. CHOI assesses tumour density where a decrease in tumour density of >15% indicates a response to treatment.

Patterns of response may differ between ICI agents. Anti-PD-1 and CTLA-4 antibodies have distinct mechanisms of action (Das ) and delayed responses or pseudo-progression appear to be less common with anti-PD-1 agents at least in melanoma (Chiou and Burotto, 2015; Wolchok ). It is unknown if patterns of response with ICI differ between solid tumour types.

We aimed to explore the different patterns of response to pembrolizumab an anti-PD-1 antibody, in melanoma patients receiving this drug on a phase I study. One of the aims was to assess and contrast response using RECIST and irRC. We also sought to explore the role of CHOI and modified CHOI (mCHOI) in assessing response with this agent.

Materials and methods

This retrospective study was conducted at the Princess Margaret Cancer Centre, Toronto, Ontario, Canada, using an institutional research ethics board-approved protocol (14–7328), in accordance with the Declaration of Helsinki. Patients enroled onto the Keynote 001 pembrolizumab phase I study in melanoma were identified and their clinical characteristics and scan(s) reviewed. Bi-dimensional tumour diameter and density measurements were obtained at baseline and in subsequent serial assessment CT scans performed at defined time points as per trial protocol. Where possible the scan performed before the baseline scan (pre-baseline scan) was also reviewed. Measurable lesions were defined as those lesions measuring ⩾5 mm in longest diameter or ⩾15 mm in short axis diameter for lymph nodes (LNs). Lesion-specific response was determined by change in the product of longest perpendicular diameters and classified as complete (CR, complete disappearance or ⩽10 mm short axis diameter for LN), partial (PR, ⩾50% reduction), PD (⩾25% increase) or stable (SD, neither CR or PR nor PD). Patient tumour response was determined using irRC (Wolchok ), RECIST 1.1 (Eisenhauer ), CHOI (Choi , 2007) and mCHOI (Nathan ) criteria. All radiological assessments were performed as previously described (Choi , 2007; Eisenhauer ; Wolchok ; Nathan ), see Table 1 and the number of lesions assessed is shown in Supplementary Table 1

Table 1

Comparison of radiological assessment criteria

	RECIST 1.1 (Eisenhaueret al, 2009)	irRC (Wolchoket al, 2009)	CHOI (Choiet al, 2004, 2007)	mCHOI (Nathanet al, 2010)
Method	Sum of the longest diameters (unidimensional) of index lesions	Sum of the products of the two largest perpendicular diameters (SPD) of all index lesions (5 lesions per organ, up to 10 visceral lesions and 5 cutaneous lesions) is calculated. Subsequently, on each scan, the SPD of the index lesions and of new, measurable lesions (⩾5 × 5 mm; up to 5 new lesions per organ: 5 new cutaneous lesions and 10 visceral lesions) are added to provide total tumour burden	Changes in tumour size (sum of the longest diameters) and changes in tumour density of index lesions	Changes in tumour size (sum of the longest diameters) and changes in tumour density of index lesions
Complete response	Disappearance of all target lesions or nodal short axis diameter <10 mm on follow-up	Disappearance of all lesions in two consecutive observations not <4 weeks apart	Disappearance of all lesions No new lesions	Disappearance of all target lesion
Partial response	>30% decrease in sum of longest diameter (LD)	>50% decrease in the burden of tumour compared with baseline in two observations at least 4 weeks apart	Decrease in size of >10% or a >15% decrease in tumour density (HU). No new lesions	Decrease in size of >10% and >15% decrease in tumour density (HU). No new lesions
Progressive disease	>20% increase in sum of LD taking as reference the smallest LD recorded and at least 5 mm increase or new lesion	Increase in tumour burden >25% relative to the minimum recorded tumour burden in two consecutive observations, at least 4 weeks apart	>10% increase in tumour size and does not meet criteria of PR by tumour density. New lesions	>10% increase in tumour size and does not meet criteria of PR by tumour density. New lesions
Stable disease	Neither PR nor PD	Neither of the above	Does not meet criteria of CR, PR or PD	Does not meet criteria of CR, PR or PD

Abbreviations: CR=complete response; irRC=immune-related response criteria; mCHOI=modified CHOI; PD=progressive disease; PR=partial response; RECIST=response evaluation criteria in solid tumors.

Overall response was assigned as per final CT scan assessment. Association of each radiological criterion with OS was determined.

Statistical analysis

Categorical variables, such as classification of tumour burden, site of occurrence, gender, stage, LDH level, BRAF and NRAS status, were summarised with counts and percentages. Continuous variables such as tumour burden from pre baseline to baseline and follow-up measures were summarised with mean, medians and/or ranges as appropriate. χ2-test was used to compare level of categorical covariates of interest with overall response (CR vs PR/SD/PD). Student's t-test was used to compare lesions size at baseline based on overall response (CR vs PR/SD/PD). Time to event was defined as the interval from the baseline CT scan until the date of death or date of last follow-up for those who were alive at the end of study. Overall survival estimates were obtained using the Kaplan–Meier product-limit method. Log-rank test was used as a test statistic to assess the impact of overall response on OS. Cox proportional hazards model was also used to assess the impact of increase in tumour size and increase in density from baseline on OS. All P-values were two-sided and for the statistical analyses, P<0.05 will be considered to indicate a significantly different result. Statistical analyses were performed using SAS Version 9.4 (SAS Institute Inc., Cary, NC, USA).

Results

Patient characteristics and distribution of lesions at baseline

Thirty-seven patients with 567 measurable lesions treated with pembrolizumab in a phase 1 trial were studied. The median age of patients was 56 years, 54% of patients were male, 95% were stage M1c (all patients had visceral metastases, five patients had a normal LDH level) and 84% were BRAF negative; patient characteristics are summarised in Table 2.

Table 2

Characteristics of patients at baseline prior to commencing treatment

Characteristics	No. of patients (%) (N=37)
Age (years)
Median (range)	56 (30–83)
Sex
Males	20 (54%)
Females	17 (46%)
Stage
M1b	2 (5%)
M1c	35 (95%)
LDH (ULN=220 U l−1)
<ULN	5 (14%)
⩾ULN	32 (86%)
BRAF
Positive	5 (14%)
Negative	31 (84%)
Unknown	1 (2%)
NRAS
Positive	5 (13%)
Negative	8 (22%)
Unknown	24 (65%)
Dose of pembrolizumab
10 mg kg−1 2 weekly	11 (30%)
10 mg kg−1 3 weekly	19 (51%)
2 mg kg−1 3 weekly	7 (19%)

Abbreviations: LDH=lactate dehydrogenase; ULN=upper limit of normal.

In total, there were 567 lesions assessed at baseline in all 37 patients, with a median product of diameter (POD), 154 mm2 (range 15–20 976). These multiple lesions were located at various sites, including visceral organs, skin, nodes and muscle. The most common site of occurrence was in the lungs with 163 of the 567 lesions (29%) occurring at this site. Nodal disease accounted for 25% (140) of all the lesions, see Table 3.

Table 3

Sum of the total tumour size and distribution of metastases in all patients

Product of diameters
Median (range)	154 mm2(15–20 976)
Site of metastases
Lung	163 (29%)
Liver	86 (15%)
Other solid organs	27 (5%)
Peritoneal	51 (9%)
Node	140 (25%)
Subcutaneous	53 (9%)
Other	47 (8%)

Response in lesions according to site of metastases and lesion size

Overall, most lesions remained stable in bi-dimensional size over the entire assessment period, whereas 25% (140 of the 562) demonstrated CR, which was defined as complete disappearance of a lesion or <10 mm in short axis diameter (Figure 1A). Complete response was most commonly seen in lung metastases with 42% of lung lesions undergoing complete disappearance, compared with 18% CR observed at other sites (69 of the 163 lung lesions vs 71 of the 399 other sites, P<0.0001; Figure 1B).

Figure 1

Response as assessed in individual lesions and according to location. (A) Overview of response of individual lesions, most lesions remained stable (34%) with CR evident in 25%. (B) Response differed according to location of metastatic disease with lung lesions showing the greatest response.

Lesions that underwent CR were smaller in size (defined as PODs) at baseline compared with those with PR/SD/PD; mean/s.d. POD of CR (568.8/879.6 mm2) vs mean/s.d. POD of PR/SD/PD (806.7/1166.7 mm2; P=0.015).

There was an association between clinical benefit and BRAF status (P<0.0001), but not NRAS mutation status. There was a higher rate of CR or PR (P<0.0001) when the LDH level was below 300 U l−1.

Changes in lesion size pre-baseline, at baseline and on treatment

On the whole, there was an increase in tumour burden from the pre-baseline to baseline study as assessed by change in the POD of all lesions with an average increase of 1467 mm2 (range −621 to 7074). When the POD was calculated per patient, the majority (84%, 31 out of 37) showed an increase in tumour size from pre-baseline to baseline. There was no evidence of associations between this pre-baseline to baseline change (increased or decreased) and subsequent response at the first follow-up study done (assessable in 37 patients, P=0.19). However, there was a significant association noted between this pre-baseline to baseline change (increased or decreased) and the second follow-up assessment study (assessable in 30 patients, P=0.03). Of patients whose disease was increasing before the baseline scan, 56% (irRC) and 48% (RECIST) demonstrated either complete or PR in the second follow-up study and 16% and 28% experienced SD as per irRC and RECIST criteria, respectively (P=0.03 and P=0.03).

Response according to irRC

When assessed using the irRC, 59% of patients (22 of 37 patients) had clinical benefit at the first assessment time point, described as CR (n=1), PR (n=14) or SD (n=7). Forty-one percentage of patients (15 out of 37) showed unconfirmed PD at the first assessment study (in the irRC, confirmation of PD by a repeat assessment in at least 4 weeks time is required). Among this group of patients, seven were confirmed as PD in the subsequent surveillance studies, two patients subsequently went on to show a delayed benefit with SD by irRC or PR by irRC and the other six patients did not have additional imaging assessment thereafter (Figure 2). One of the seven patients with confirmed PD at the second assessment showed an atypical delayed response in the subsequent assessments, with a resultant PR by irRC.

Figure 2

Patterns of response as assessed by (A) irRC and (B) Response evaluation criteria in solid tumors (RECIST) 1.1. Response patterns differed with greater variation seen on first scan as assessed by RECIST in comparison with irRC.

Response according to RECIST 1.1 criteria

When assessed using the RECIST criteria, 43% (16 out of 37 patients) were found to have PD on the initial scan. Among these RECIST PD patients, the irRC identified four of them as having stable disease in the initial assessment study, two of whom continued to demonstrate objective responses (irRC) in the subsequent assessment point, one had irRC PR and one irRC SD. Consequently, this analysis shows that 5% (2 out of 37) of treated patients, who were initially characterised as PD by RECIST criteria, did actually go on to demonstrate some treatment benefit (Figure 2).

Response according to CHOI and mCHOI criteria

Twenty-four (65%) patients met CHOI density and size criteria for assessment of clinical benefit (CR, PR or SD) at first follow-up. CHOI and mCHOI criteria showed benefit in 38% (14 out of 37). Change in tumour size and density on first follow-up assessment was associated with OS with each 1000 mm2 increase in tumour size from baseline increasing the hazard of dying by 26% (HR=1.26, (95% CI=1.12–1.42), P=0.0002). Similarly, each 20HU increase in density increased the HR by 15% (HR=1.15, (95% CI 1.045–1.260), P=0.004).

Differences in response assessment between criteria

The impact of density as opposed to change in size of lesions was examined by comparing where possible RECIST response to CHOI at each time point. There were differences in clinical benefit by each of these criteria, suggesting that density had added value in assessment of response (Table 4; Supplementary Table 2).

Table 4

Comparison of numbers of patients deriving clinical benefit vs progressive disease as assessed by RECIST vs CHOI

	CHOI FU1				CHOI FU2				CHOI FU3				CHOI FU4
RECIST FU1	CR/PR/SD	PD	Total	RECIST FU2	CR/PR/SD	PD	Total	RECIST FU3	CR/PR/SD	PD	Total	RECIST FU4	CR/PR/SD	PD	Total
CR/PR/SD	21	2	23	CR/PR/SD	8	1	9	CR/PR/SD	11	0	11	CR/PR/SD	7	0	7
PD	3	11	14	PD	0	21	21	PD	2	3	5	PD	1	2	3
Total	24	13	37	Total	8	22	30	Total	13	3	16	Total	8	2	10

Abbreviations: CR=complete response; PD=progressive disease; PR=partial response; SD=stable disease.

Examining the largest data sets, those at baseline, at first and at second follow-up, there were differences between criteria in the numbers of patients determined to be deriving clinical benefit on treatment (Table 5). It was not possible to determine which criterion performed optimally in assessing response during treatment or overall. The amplitude of response by each criterion across all assessable patients is shown in Supplementary Figure 1. Figure 3 illustrates the difference in assessment by different criteria in one lesion on serial CT scans on treatment.

Table 5

Comparison of the numbers of patients deemed to be deriving clinical benefit vs progressive disease by each radiological criteria

	Follow-up 1					Follow-up 2
	RECIST	irRC	CHOI	mCHOI		RECIST	irRC	CHOI	mCHOI
CR/PR/SD	23	24	24	14	CR/PR/SD	21		23	13
PD	14	13	13	23	PD	9		8	18

Abbreviations: CR=complete response; irRC=immune-related response criteria; mCHOI=modified CHOI; PR=partial response; SD=stable disease.

Figure 3

Serial images from a 62-year-old woman with metastatic melanoma to the subcutaneous tissues treated with anti-PD-1 antibody. (A) Baseline scan shows a subcutaneous deposit measuring 26 × 20 mm with a density of 57 HU. (B) The first follow-up scan performed 13 weeks after the first infusion and showed that the lesion had progressed by RECIST and irRC (measurements were 48 × 32 mm), but had decreased in density (to 38 HU). (C) On the second follow-up CT scan, the measurements were 60 × 31 mm, but the density had again decreased (to 35 HU). (D) On the final assessment scan, the measurements were 67 × 30 mm, but density was 25 HU.

Associations between response (by each criterion) and OS

At the time of reporting the median follow-up was 9.7 months (range 3–19). Responders (CR/PR/SD vs PD) defined by any criterion had superior OS (by log rank testing); irRC P<0.0001, RECIST P=0.0003, CHOI P=0.008 and mCHOI P=0.018, respectively.

Using irRC, the 6-month survival for responders vs non-responders was 100% vs 64% (95% CI 0.3–0.85) and 1-year survival was 85% (95% CI 0.48–0.97) vs 36% (95% CI 0.11–0.63), respectively. By RECIST 1.1, 6-month and 1-year OS for responders vs non-responders were 100% vs 78% (95% CI 0.51–0.91) and 100% vs 41% (95% CI 0.14–0.67), respectively. Using CHOI, the 6-month and 1-year OS were 95% (95% CI 71–99) vs 77% (95% CI 0.44–0.92) and 79% (95% CI 0.44–0.94) vs 52% (95% CI 0.22–0.75), respectively. Under mCHOI, 6-month and 1-year survival were 100% vs 82% (95% CI 0.59–0.93) and 86% (95% CI 0.33–0.98) vs 63% (95% CI 0.39–0.8), respectively. Thus, response by each response criterion assessed here had independent prognostic value.

Discussion

We performed a comparison of four radiological criteria in the assessment of response patterns in metastatic melanoma patients treated with pembrolizumab. There were differences in response according to location of metastatic disease and size of metastases. The majority of lesions showed stability over time with treatment rather than CR or PR. Nevertheless, there was a significant association with OS in all responders or benefiters from treatment (SD, PR and CR), by any response criterion, indicating a positive effect on overall disease control regardless of magnitude of tumour shrinkage. Responses tended to be early but 5% had a delayed response. Interestingly, there was an association between growth on pre-baseline scan and response on the second assessment scan possibly indicating a longer time to response in patients with disease progression pretreatment.

Ongoing and published clinical trials evaluating anti-PD-1/PDL1 agents have predominantly used RECIST criterion in assessing response (Chiou and Burotto, 2015). Some have also used irRC as a secondary response criterion. Where comparisons are possible, it does appear that irRC captures responses otherwise missed with RECIST, although the number of cases is small, between 3–12% depending on the study (Chiou and Burotto, 2015). The differential responses seen in our study according to the location of metastases and size have been reported in other studies. Twenty-seven patients treated on the pembrolizumab Keynote 001 phase I study at a single centre were evaluated for response using POD (as in our study) to assess individual lesions and irRC alone for overall response (Lyle ). Lung lesions and smaller lesions (median POD 80 vs 246 mm2(P<0.05) were noted to have the highest rate of CR. When the majority of Keynote 001 melanoma patients were assessed as a whole regardless of cohort (n=411), larger lesion size was the only independent factor associated with inferior response (Joseph ). Tumeh* reported (n=112 patients treated on keynote 001) liver metastases as being associated with treatment failure with lower CD8+T-cell tumour infiltration, T-cell PD-1 and tumour PDL1 expression. Lung lesions were again associated with higher rates of response. Given the differential responses seen in the liver but the consistent high rates of response in the lungs between these studies, it is likely that tumour microenvironments differ between metastatic sites and between individual patients. These differences likely account for the heterogeneity in responses and have implications for patient management (Herbst ; Tumeh ).

Comparing across radiological criteria in our study is difficult, as it was not possible to assess response by each criterion in every patient (due to patient drop out of trial consistent with this being a phase I study). Our study is also limited by its small size and its retrospective nature. Nonetheless, the use of CHOI criterion is novel and identified responses not otherwise captured by RECIST alone. The use of change in tumour density may take into account changes in tumour vasculature and or tumour necrosis that may not be perceptible using RECIST 1.1 yet be predictive of benefit. This is particularly important where the majority of lesions, as reported here, remain stable by size criteria over time. Future prospective studies are needed to evaluate the role of CHOI is assessing response to ICI particularly in cases of possible progression or heterogeneity of response.

The use of ICI is likely to extend beyond metastatic melanoma and non-small cell lung cancer to many other solid tumours. Although combination therapies of different ICI or ICI with targeted and anti-angiogenic agents are actively being explored, the toxicities from such combinations are likely to be significant. Moreover, it is clear that even with single agent ICI the assessment of response and determination of which patients are benefiting from their treatment is problematic. Future perspective studies are needed to determine a consensus on the best radiological criterion or combination of criteria to use in managing patients on ICI.