An indirect treatment comparison of the efficacy of pembrolizumab versus competing regimens for the adjuvant treatment of stage III melanoma.

Objective: To determine the efficacy of pembrolizumab relative to other treatments used in stage III melanoma by conducting a systematic literature review (SLR) and network meta-analysis (NMA).
Methods: A SLR was conducted to identify randomized clinical trials (RCTs) evaluating approved adjuvant treatments including interferon-containing regimens, BRAF-inhibitors, and PD-L1 inhibitors in stage III melanoma patients. Relative treatment effects for recurrence-free survival (RFS) were synthesized with Bayesian NMA models that allowed for hazard ratios (HRs) to vary over time.
Results: Included studies formed a connected network of evidence composed of eight trials. In high-risk stage III patients, the HR for pembrolizumab vs observation decreased significantly over time with the superiority of pembrolizumab over observation becoming statistically meaningful before 3 months. By 9 months, the HR for pembrolizumab vs observation was statistically significantly lower than the HR for most other treatments vs observation, with the exception of ipilimumab and biochemotherapy due to overlapping 95% credible intervals. In BRAF + patients, pembrolizumab was statistically significantly better than observation after 3 months. The HR for both BRAF-inhibitors vs observation increased significantly over time and pembrolizumab was statistically superior to both BRAF-inhibitors after 15 months. Conclusions: Pembrolizumab results in statistically significantly improved RFS compared to all competing regimens after 9 months, except ipilimumab and biochemotherapy, for the adjuvant treatment of stage III melanoma. However, point estimate HRs vs observation for pembrolizumab are much lower than those for ipilimumab. In BRAF + patients, the advantage of pembrolizumab versus competing interventions increases over time with respect to RFS.

Introduction

Malignant melanoma is one of the few remaining cancers that is increasing in incidence in developed countries worldwide. The rate of developing melanoma was 31.9 per 100,000 person years in men and 20.4 per 100,000 person years in women in the US in 2014. In 2015, the world regions with the greatest rates in both incidence and mortality were Australia, North America, Western Europe, Central Europe, and Eastern Europe. When diagnosed early, rates of survival are relatively high. However, Gershenwald and Scolyer state that the 5-year survival rates range from 93–69% for stage IIIA–stage IIIC melanoma compared to 99–97% for stage IA–IB and 94–87% for stage IIA–IIB melanoma. Similarly, 10-year survival rates decrease from 88% for stage IIIA to 69% for stage IIIC melanoma. Recurrence of stage III melanoma is moderate-to-high with 5-year recurrence-free survival ranging from 50–63% for stage IIIA and 11–12% for stage IIIC melanoma. Survival rates for stage III melanoma are significantly lower compared with those diagnosed with early stage melanoma, demonstrating a need for effective treatments in those diagnosed with high-risk melanoma. Until the results of the MSLT studies were released, complete lymphadenectomy with or without adjuvant therapy was the primary treatment for patients with stage III melanoma for patients with confirmed disease in the lymph nodes. Interferon-alpha (IFN-α) was the first treatment to provide meaningful improvement in relapse-free survival (RFS) and overall survival (OS) for these patients. A meta-analysis published in 2017 assessed clinical efficacy of IFN-α as adjuvant therapy and found that it significantly reduced the risk of relapse as well as improved overall survival regardless of dosage for patients with high-risk melanoma. In the several countries where adjuvant therapy for melanoma is routinely used, IFN-α was the only drug approved as adjuvant therapy for melanoma patients, until recently, when ipilimumab was approved in the US for adjuvant therapy.

In the past several years, clinical trials evaluating immune-checkpoint inhibitors, including PD-1 inhibitors and CTLA-4 inhibitors, for the adjuvant treatment of stage III melanoma have been published. EORTC 18071, conducted in high-risk stage III melanoma, demonstrated ipilimumab significantly improved RFS in this population compared to placebo (1-year rate of RFS = 63.5% [95% CI = 59.0–67.7%] vs 56.1% [51.5–60.5%]; hazard ratio (HR) for RFS = 0.75 [0.64–0.90]). CheckMate 238, a phase III double-blind RCT in patients undergoing resection stage IIIB–IV melanoma, found that adjuvant therapy with nivolumab monotherapy offered significantly longer RFS compared to ipilimumab (1-year rate of RFS = 70.5% [95% CI = 66.1–74.5%] vs 60.8% [56.0–65.2%]; HR for RFS = 0.65 [0.53–0.81]). In a randomized, phase III double-blind trial, KEYNOTE 054, in patients with resected high-risk stage III melanoma, pembrolizumab monotherapy was associated with significantly longer recurrence-free survival (RFS) compared with placebo (1-year rate of RFS = 76.4% [95% CI = 71.3–78.9%] vs 61.0% [56.5–65.1%]; HR for RFS = 0.57 [0.43–0.74]).

In addition to immune-checkpoint inhibitors, BRAF/MEK inhibitors such as vemurafenib and dabrafenib + trametinib combination therapy have been evaluated for the treatment of stage III BRAF V600 mutated melanoma. COMBI-AD, a phase III double-blind study, assessed dabrafenib in combination with trametinib vs placebo for the adjuvant treatment of stage III melanoma in BRAF V600E/K-mutant melanoma. Dabrafenib in combination with trametinib in the COMBI-AD trial was associated with significantly lower risk of recurrence vs placebo (1-year rate of RFS = 88% [95% CI = 85–91%] vs 56% [51–61%]; HR for RFS = 0.49 [0.40–0.59]). Finally, vemurafenib vs placebo was evaluated in BRAF V600 mutation positive patients for the adjuvant treatment of stage III melanoma in a phase III, double-blind trial, BRIM-8 cohort 2. Findings from BRIM-8 cohort 2 showed that vemurafenib did not statistically significantly reduce risk of recurrence vs placebo (HR for RFS = 0.80 [95% CI = 0.54–1.18]) in stage III BRAF V600 mutation positive patients,.

Given the relatively new development of these therapies and lack of evidence in comparative treatment efficacy of currently available adjuvant treatments for adult patients with high-risk stage III melanoma independent of BRAF-status as well as for the BRAF-mutated population, there is a need to formally assess relative treatment effects. In cases where a connected network of evidence is formed, network meta-analysis (NMA) provides a valid alternative for simultaneous comparison of all included interventions and estimates relative treatment effects between any pair of interventions in the network. The purpose of this study is to compare the relative efficacy, specifically RFS, given that this endpoint is available for all treatments of interest, of pembrolizumab to additional competing interventions for the adjuvant treatment of high-risk stage III melanoma patients, independent of BRAF-status, as well as for BRAF-mutated patients by means of network meta-analysis.

Methods

Systematic literature review

A systematic literature review (SLR) was performed using OVID MEDLINE, EMBASE, and the Cochrane Central Register of Controlled Trials. The final date for the literature searches was February 8, 2018. Pre-specified selection criteria regarding study population, interventions, comparators, outcomes, and study design (PICOS) are enumerated in Table 1. The PICOS captured RCTs that (1) were conducted in stage III melanoma patients, (2) evaluated recommended interventions based on the National Comprehensive Cancer Network (NCCN) guidelines, (3) reported outcomes of interest, and (4) were published in English. The Scottish Intercollegiate Guidelines Network (SIGN) filter for RCTs was used to limit study design. MeSH terms and keywords were used to identify studies by population, interventions, and comparators. In addition, 2016 and 2017 conference proceedings from the European Society of Medical Oncology (ESMO), The Society of Melanoma Research (SMR), Society for Immunotherapy of Cancer (SITC), American Association for Cancer Research (AACR), and the American Society of Clinical Oncology (ASCO) were searched. A hand search of the US National Institutes of Health Clinical Trials Registry was also performed. Full search strategies are available in Supplementary Tables S1–S5.

Table 1.

Study selection criteria to identify trials for the SLR.

Criteria	Description
Population	Stage III melanoma
	Sub-groups of interest: • BRAF mutation status
Interventions	Pembrolizumab
	Interferon (IFN)-α2a
	IFN-α2b
	Pegylated IFN-α2b
	Nivolumab
	Ipilimumab
	Dabrafenib in combination with trametinib
	Temozolomide in combination with cisplatin
	Vemurafenib
Comparisons	Placebo or best supportive care (BSC)
	Any intervention of interest as monotherapy or in combination
	Any treatment that facilitates an indirect treatment comparison
Outcomes	Recurrence-free (or relapse-free) survival (RFS)
Study design	Randomized controlled trials
	Systematic literature reviews
	Meta-analyses
Other	English language
	No time restriction

Two reviewers independently screened titles and abstracts of articles and conference proceedings for potentially eligible studies. Full-text publications corresponding to included studies were then retrieved and screened in duplicate. A PRISMA flow diagram for the SLR is provided in Supplementary Figure S1. Discrepancies were resolved by discussion between the pair of reviewers. Once the list of included studies was finalized, trial characteristics, patient characteristics, and study results were extracted in duplicate from all eligible publications.

Study quality was assessed using Cochrane Collaboration’s Risk of Bias tool to evaluate six key domains: sequence generation; allocation concealment; blinding of participants, personnel and outcome assessors; incomplete outcome data; selective outcome reporting; and other sources of bias.

Network meta-analysis

In the absence of head-to-head clinical trial data, NMA is a viable option to obtain relative treatment effects among interventions that have not been directly compared with each other. In order to synthesize multiple trials and data sources with minimal bias, NMA depends on careful consideration when defining prognostic factors and relative treatment effect modifiers. In NMA, only relative treatment effects from each trial are incorporated, thereby decreasing the risk that known and unknown prognostic factors will affect results. However, in order to decrease risk that potential relative treatment effect modifiers will bias NMA results, a thorough feasibility assessment is conducted on the evidence base. The feasibility assessment is conducted prior to the NMA and ensures that trials meeting PICOS selection criteria are reasonably similar with respect to: (1) whether RCT evidence for the interventions of interest form an evidence base for the target population and outcome of interest; and (2) assess that trial characteristics, interventions characteristics, and patient characteristics that may affect treatment effects of trials included in the evidence are reasonably distributed. Trial characteristics assessed include study phase, trial initiation and completion, eligibility criteria, and risk of bias. Patient characteristics of interest were those that may act as potential treatment effect modifiers, such as age, sex, disease stage, ECOG performance status, melanoma sub-type, PD-L1 expression, and BRAF mutation status. After assessing heterogeneity, and excluding trials that differ from the target population and overall evidence base, the evidence was synthesized by means of NMAs.

Traditional NMA results for time-to-event outcomes are based on HR estimates, which rely on the proportional hazards assumption. When the proportional hazards assumption is violated, such as if hazard functions of competing interventions cross or if the calculated time-varying parameter (d1) is statistically significant as determined by its associated credible interval (CrI), an NMA model allowing for time varying HRs is appropriate. Jansen and Ouwens et al. have presented methods for NMA of survival data using a multidimensional treatment effect as an alternative to the synthesis of the constant HRs. The hazard functions of the interventions in a trial are modeled using known parametric survival functions or fractional polynomials, and the difference in the parameters are considered the multidimensional treatment effect, which are synthesized (and indirectly compared) across studies,. With this approach, the treatment effects are represented by multiple parameters rather than a single parameter. By including additional parameters for treatment effects, the proportional hazards assumption is relaxed and the time-varying HR NMA model can more closely fit reported data.

This analysis was done for RFS, as it was the endpoint reported for all treatments of interest, which formed a connected network. For RFS, the following survival distributions were considered using a multivariate NMA framework, as proposed by Jansen: Weibull, Gompertz, and second-order fractional polynomials including p1 = 0 or 1 and p2 = 0 or 1. Second-order fractional polynomial models are extensions to the Weibull and Gompertz model, and allow for arc- and bathtub-shaped functions, which follow parametric distributions such as log-normal or log-logistic. For each treatment arm of each study in the NMA, the reported Kaplan-Meier curves were digitized and divided into intervals over the follow-up period. Within each interval, survival proportions are used to calculate patients at risk at the beginning of each interval as well as the incident number of deaths or recurrences, as a way to calculate the hazard rate corresponding to the underlying event probability, which is standardized by months.

Time-varying HR NMA results were presented as estimates for HRs of each intervention relative to the reference treatment, observation, up until 24 months, the maximum follow-up of KEYNOTE 054. Point estimates of HRs every 3 months, beginning at 3 months, are provided in tables for each treatment compared to observation. The best-fitting model for each analysis is presented, as determined by the lowest deviance information criterion (DIC). Statistically significant variations over time were justified by calculating a 95% credible interval (CrI) around the time-varying parameter, d1. Furthermore, if 95% CrIs of interventions vs observation no longer overlapped with each other, this determined statistical differentiation of treatments. All analyses were performed using R version 3.4.0 (R Project for Statistical Computing) with the package R2JAGS version 0.5.7 (OpenBUGS Project Management Group).

Results

Initial literature searches identified 4,857 references across all databases. After removal of duplicates, this number was reduced to 3,898 articles. After exclusion on the basis of titles and abstracts, 170 papers were retrieved for full-text review. Overall, a total of 18 full-text publications, three conference abstracts, one citation from the clinical trials registry, and one citation from an unpublished clinical study report met the inclusion criteria in full; these 23 included publications corresponded to 12 unique trials. The study identified through the clinical trials registry was BRIM-8 and the unpublished clinical study report was KEYNOTE 054, both of these trials were published after the date of the systematic database search. The included trials and their corresponding publications are listed in Table 2.

Table 2.

List of publications and key trial characteristics, arranged by trial.

Trial ID	Phase	Masking	Multi-center	Age (years)	Diseasestage	Performancescore	Trialnumber	Principalpublication	Subsequentpublications
BRIM-8*	III	Double-blind	Yes	≥18	IIC–IIIC	ECOG 0-1	NCT01667419	Maio et al.14	Lewis et al.13
Caraceni 1998	—	—	—	18–70	IIIB	Karnofsky 100	—	Caraceni et al.24	—
CheckMate 238*	III	Double-blind	Yes	≥15	IIIB–IV	ECOG 0-1	NCT02388906	Weber et al.10	Weber et al.25
COMBI-AD	III	Double-blind	Yes	≥18	IIIA–IIIC	ECOG 0-1	NCT01682083	Long et al.26	Hauschild et al.27
EORTC 18071	III	Double-blind	Yes	≥18	IIIA–IIIC	ECOG 0-1	EUCTR2007-001974-10	Eggermont et al.9	Coens et al.28
							NCT00636168		Eggermont et al.29
									Eggermont et al.30
EORTC 18952	III	Open-label	Yes	18–70	IIB–IIIC	—	NCT00002763	Eggermont et al.31	Eggermont et al.32
							CDR0000064718
EORTC 18991	III	Open-label	Yes	18–70	III	ECOG 0-1	NCT00006249	Eggermont et al.33	Fusi et al.34
									Bottomley et al.35
									Eggermont et al.36
									Herndon et al.37
KEYNOTE 054	III	Double-blind	Yes	≥18	III	ECOG 0-1	NCT02362594	Eggermont et al.11	—
Lian 2013	II	—	No	≥18	II–III	ECOG 0-1	ChiCTR-TRC-11001798	Lian et al.38	—
Nordic IFN trial*	III	Open-label	Yes	≥18	IIB–III	ECOG 0-1	NCT01259934	Hansson et al.39	Vihinen et al.40
SWOG S0008	II	Open-label	Yes	≥10	IIIA–IIIC	Zubrod 0-1	NCT00006237	Flaherty et al.41	—
WHO MPT 16	II	Open-label	Yes	18–70	III	—	—	Cascinelli et al.42	—

Note: Trials are listed in alphabetical order; * denotes trials that provide stage III sub-group data.

Feasibility assessment

Outcomes of interest

Relapse (or recurrence)-free survival (RFS) was reported in 11 trials identified in the SLR. Caraceni et al. did not report RFS as an outcome. RFS was defined in six trials: BRIM-8, CheckMate 238, COMBI-AD, KEYNOTE 054, Lian 2013, and the Nordic IFN trial,,,,,. The remaining five trials that reported RFS did not define how RFS was measured. Of the six trials defining RFS, five trials defined RFS as the time from randomization until date of the first recurrence (local, regional, or distant metastasis), death, or end of follow-up. Nordic IFN trial defined RFS as time until first verified relapse at any site.

Trial characteristics

Trial characteristics of included RCTs were reasonably similar (Table 2). Primary completion dates ranged from 2003–2018. Out of the 12 trials included from the SLR, five were double-blind, five were open-label, and two did not report masking status of the trial (Caraceni et al. and Lian et al.). According to the Cochrane Collaboration’s tool, most trials had low risk of bias, as shown in Supplementary Tables S6 and S7. Eligibility criteria for most trials included patients with ECOG performance scores of 0 or 1. Caraceni et al. measured performance score based on the Karnofsky scale, specifically enrolling patients with Karnofsky 100, which is the equivalent of ECOG 0. Additionally, two trials did not report ECOG performance score eligibility criteria (EORTC 18952 and WHO MPT 16),. No trial restricted eligibility based on PD-L1 immunohistochemistry status. Two trials, COMBI-AD and BRIM-8, only enrolled patients who were BRAF V600 mutation positive. All trials were multicenter, with the exception of Lian et al.. Of the 12 trials included after the SLR, the majority included only stage III patients. Four trials, however, allowed enrollment of stage II patients: BRIM-8 (stage IIC–IIIC), EORTC 18952 (stage IIB–IIIC), Lian (stage II–III), and Nordic IFN trial (stage IIB–III),,,. One trial, CheckMate 238, allowed enrollment of stage IV patients. All trials required patients with cutaneous melanoma, with the exception of Lian, which enrolled patients with confirmed mucosal melanoma only, and CheckMate 238, which did not specify specific melanoma sub-types in eligibility criteria,. Of trials reporting proportions of stage II and stage IV patients, all reported stage III sub-group data.

Intervention characteristics

The trials included in the feasibility assessment included the following interventions: BRAF-inhibitors (vemurafenib and dabrafenib + trametinib), interferon-containing regimens (IFN-α2a, IFN-α2b, pegylated IFN-α2b, and biochemotherapy), ipilimumab, nivolumab, pembrolizumab, and observation or placebo. Although biochemotherapy was not listed in the SLR search strategy, it was a combination of cisplatin + vinblastine + dacarbazine + interleukin-2 (IL-2) + IFNα + filgrastim (G-CSF), which included an intervention of interest, IFN. Intervention characteristics of included trials, including dosage and frequency and planned duration of treatment, were reasonably similar across trials. Treatments were administered via IV for five trials (nivolumab, ipilimumab, IFN-α2b, cisplatin, and biochemotherapy), oral tablets for four trials (temozolomide, vemurafenib, dabrafenib, and trametinib), and subcutaneously for eight trials (IFN-α2a, IFN-α2b, biochemotherapy, and pegylated IFN-α2b). Ipilimumab was administered in two trials, CheckMate 238 and EORTC 18071, at a dosage of 10 mg/kg every 3 weeks for 4 doses and then every 12 weeks for 1 year (CheckMate 238) or 3 years (EORTC 18071), or until disease recurrence, unacceptable toxicity, major protocol violation, or treatment refusal,. However, although initial ipilimumab treatment was similar in CheckMate 238 and EORTC 18071, maintenance treatment differed between the two trials. Specifically, in CheckMate 238, ipilimumab was administered every 12 weeks for up to 1 year, compared to EORTC 18071, which administered ipilimumab every 12 weeks for 3 years,. Although initial treatment of ipilimumab was the same for both CheckMate 238 and EORTC 18071, the duration of the ipilimumab maintenance period greatly differed. Maintenance treatment for ipilimumab began at week 24 for both CheckMate 238 and EORTC 18071, however, maintenance treatment was up to 1 year for CheckMate 238 and to 3 years for EORTC 18071. Differences in maintenance therapy duration, for the ipilimumab arms in CheckMate 238 and EORTC 18071, may lead to differences in treatment efficacy; therefore, the two ipilimumab arms cannot be considered equivalent. No background or concomitant therapies were reported by any trials included in the feasibility assessment. Crossover was not permitted in six trials; the remaining trials did not explicitly report whether they allowed crossover. Although KEYNOTE 054 allowed crossover in part 2 of the trial, this analysis reflects follow-up from part 1 of the trial, which did not allow crossover.

Patient characteristics

Differences identified in the ITT populations with respect to age, sex, ECOG status, BRAF status, and disease stage status are summarized in Table 3. Differences with respect to potential effect modifiers, such as BRAF status, disease stage, and melanoma sub-type were identified.

Table 3.

List of publications and key patient characteristics, arranged by trial.

Trial ID	Treatment	n	Median age (range)	Male (%)	ECOG 0 or 1 (%)	Stage IIIA (%)	Stage IIIB (%)	Stage IIIC (%)	Stage III (%)
BRIM-8 cohort 1	Vemurafenib	157	51.0 (43.0–60.0)**	84 (54%)	155 (100%)	36 (23%)	106 (68%)	—	—
	Placebo	157	49.0 (40.0–59.0)**	88 (56%)	157 (100%)	39 (25%)	106 (68%)	—	—
BRIM-8 cohort 2	Vemurafenib	93	55.0 (40.0–61.0)**	52 (56%)	92 (100%)	—	—	93 (100%)	—
	Placebo	91	50.0 (38.0–58.0)**	59 (65%)	91 (100%)	—	—	91 (100%)	—
Caraceni 1998	IFN-α2a	37	46.0 (39.0–53.0)**	22 (60%)	—	—	—	—	—
	Control	30	49.5 (41.0–58.0)**	20 (67%)	—	—	—	—	—
CheckMate 238	Nivolumab	453	56.0 (19.0–83.0)	258 (57%)	—	—	163 (36%)	204 (45%)	—
	Ipilimumab (1 year)	453	54.0 (18.0–86.0)	269 (59%)	—	—	148 (33%)	218 (48%)	—
COMBI-AD	Dabrafenib + trametinib	438	50.0 (18.0–89.0)	195 (45%)	—	83 (19%)	169 (39%)	181 (41%)	—
	Placebo	432	51.0 (20.0–85.0)	193 (45%)	—	71 (16%)	187 (43%)	166 (38%)	—
EORTC 18071	Ipilimumab (3 years)	475	50.7* (20.0–84.0)	296 (62%)	474 (99%)	98 (21%)	213 (45%)	164 (34%)	475 (100%)
	Placebo	476	51.5* (18.0–78.0)	293 (62%)	476 (100%)	88 (18%)	207 (44%)	181 (38%)	476 (100%)
EORTC 18952	IFN-α2b (13 months)	553	49.0 (17.0–74.0)	312 (56%)	—	412 (74%)	—	—	—
	IFN-α2b (25 months)	556	50.0 (16.0–75.0)	308 (55%)	—	414 (74%)	—	—	—
	Observation	279	47.0 (20.0–75.0)	152 (54%)	—	206 (74%)	—	—	—
EORTC 18991	Pegylated IFN-α2b	627	50.0 (19.0–70.0)	366 (58%)	627 (100%)	—	—	—	627 (100%)
	Observation	629	50.0 (18.0–70.0)	367 (58%)	629 (100%)	—	—	—	629 (100%)
KEYNOTE 054	Pembrolizumab	514	54.0 (19.0–88.0)	324 (63%)	514 (100%)	80 (16%)	237 (46%)	197 (38%)	514 (100%)
	Placebo	505	54.0 (19.0–83.0)	304 (60%)	505 (100%)	80 (16%)	230 (46%)	195 (39%)	505 (100%)
Lian 2013	Observation	63	57.0 (25.0–80.0)	28 (44%)	—	—	—	—	21 (33%)
	High-dose IFN-α2b	63	55.0 (26.0–84.0)	25 (40%)	—	—	—	—	16 (25%)
	Temozolomide + cisplatin	63	59.0 (18.0–75.0)	23 (37%)	—	—	—	—	19 (30%)
Nordic IFN trial	Observation	284	51.0 (18.0–76.0)	167 (59%)	284 (100%)	—	—	—	229 (81%)
	IFN-α2b (12 months)	285	53.0 (18.0–73.0)	177 (62%)	285 (100%)	—	—	—	227 (80%)
	IFN-α2b (24 months)	286	51.0 (22.0–77.0)	183 (64%)	286 (100%)	—	—	—	233 (82%)
SWOG S0008	IFN-α2b	203	48.0 (12.0–73.0)	141 (69%)	—	—	—	—	—
	Biochemotherapy†	199	46.0 (10.0–74.0)	141 (71%)	—	—	—	—	—
WHO MPT 16	IFN-α2a	225	—	131 (58%)	222 (99%)	—	—	—	—
	Surgery + observation	219	—	114 (52%)	212 (97%)	—	—	—	—

Mean age used in trials where median age is not reported.

IQR minimum and IQR maximum used in trials where age range is not reported.

Biochemotherapy is a combination of cisplatin + vinblastine + dacarbazine + interleukin-2 (IL-2) + IFN-α + filgrastim (G-CSF).

Ranging from 46–59 years, median age at baseline varied little between treatment arms. Similarly, sex distribution across trials had little variation, with all trials being majority male, except COMBI-AD and Lian, which consisted of 45% and 40% males, respectively,. In trials reporting ECOG performance status (PS), the proportion of patients with PS of 0 or 1 ranged from 98% to 100%. Two trials reported small proportions of unknown or missing ECOG status, WHO MPT 16 and COMBI-AD,. Few trials reported patients with stage II (n = 2 trials), stage IIB (n = 1 trial), and stage IIC (n = 1 trial) disease. Nordic IFN trial reported a small proportion of stage II patients (19%). EORTC 18952 included 26% stage IIB patients, and BRIM-8 cohort 1 included 9% stage IIC patients,. Among the four trials reporting stage II, IIB, and IIC patients, three trials were composed of less than 25% stage II patients, and all four trials provided stage III sub-group data. Stage IIIA, IIIB, and IIIC proportions were reported in six trials, and stage III proportions were reported in five trials. Stage III patients, including sub-stages stage IIIA–IIIC, ranged from 74–100%. However, Lian et al. reported only 29% of patients enrolled were stage III. Three trials did not report disease stage proportions: Caraceni, SWOG S0008, and WHO MPT 16,,. Only one trial, CheckMate 238, reported a population with stage IV patients, which was composed of 19% stage IV patients. Of note, although both SWOG S0008 and WHO MPT 16 did not report disease stage proportions, they both noted they only enrolled stage III patients,. BRAF mutation status was reported in four trials: BRIM-8, CheckMate 238, COMBI-AD, and KEYNOTE 054. Both BRIM-8 and COMBI-AD were conducted in BRAF + patients only, whereas 42–43% of patients enrolled in CheckMate 238 and KEYNOTE 054 were BRAF+,,,. Two trials allowed mucosal melanoma patients, Lian (100%) and CheckMate 238 (3%), compared to the remaining trials which only enrolled patients with cutaneous melanoma,. Furthermore, CheckMate 238 enrolled patients with acral and other rare sub-types of melanoma.

Feasibility assessment summary

Overall, the studies were determined to be of good quality and with minimal heterogeneity, although key differences were identified with respect to eligibility criteria, including melanoma sub-type and BRAF mutation status, intervention characteristics, and disease stage. Four trials were removed after the feasibility assessment, to ensure a homogenous evidence base: Caraceni 1998, CheckMate 238, Lian 2013, and EORTC 18952. Caraceni 1998 did not report any outcomes of interest, and therefore could not be included in the NMA. Lian 2013 was conducted exclusively in mucosal melanoma patients. Additionally, CheckMate 238 was largely conducted in a cutaneous melanoma patient population (85%), however, it did include small proportions of patients with acral, mucosal, and other melanoma sub-types. The rest of the evidence base was conducted in cutaneous melanoma patients. EORTC 18952 was conducted in stage IIB–IIIC patients and did not provide a stage III sub-group KM curve. The target population was stage III melanoma patients only. Therefore, trials conducted in stage III patients or that reported stage III sub-group analysis results were used in the NMA to ensure a homogenous evidence base.

Although CheckMate 238 provided sub-group data, reasons associated with both population and trial characteristics contributed to its exclusion from the evidence base. Specifically, CheckMate 238 was excluded due to differences in intervention administration of ipilimumab compared to EORTC 18071, which also administered ipilimumab, and patient population compared to the remaining trials in the evidence base. Differences in both patient population and intervention administration are described in the NICE appraisal consultation document for nivolumab in adjuvant treatment of resected stage III and IV melanoma. CheckMate 238 compared nivolumab to ipilimumab, rather than observation, and therefore was connected to the network by EORTC 18071, which compared ipilimumab to observation. Ipilimumab was administered every 3 months for up to 1 year or until disease progression in CheckMate 238, whereas, ipilimumab was administered every 3 months for up to 3 years or until disease progression, an unacceptable level of toxic effects, major protocol violation, or withdrawal of consent in EORTC 18071. Furthermore, CheckMate 238 only included stage IIIB, IIIC, and resected stage IV patients, whereas EORTC 18071 enrolled stage IIIA-IIIC patients, but did not enroll stage IV patients. Differences with respect to ipilimumab treatment duration and patient populations with respect to disease stage and melanoma sub-type in CheckMate 238 could not be adjusted, which would lead to biased estimates of nivolumab vs competing interventions, therefore, it was excluded from NMAs. Connected networks of evidence were constructed for RFS after completion of the feasibility assessment in stage III melanoma patients consisting of six trials (Figure 1) as well as for RFS in BRAF + patients consisting of three trials (Figure 2), respectively.

Figure 1.

Network of evidence for recurrence-free survival, Stage III.

Figure 2.

Network of evidence for recurrence-free survival, BRAF + sub-group.

A sub-group analysis for RFS was conducted in BRAF + patients only, as BRAF mutation status is a known treatment effect modifier. Of note, BRAF inhibitors were not included in the RFS stage III NMA, as these treatments are not considered relevant comparators for efficacy outcomes in a BRAF-unselected population.

Network meta-analysis

Networks of evidence were constructed for RFS in stage III and in BRAF + melanoma patients, as seen in Figures 1 and 2. The best-fitting second order fractional polynomial (FP) model as determined by the lowest DIC for each network of evidence as well as HRs by time-point are presented for stage III RFS (Figure 3(a) and Table 4) and the sub-group analysis for RFS in BRAF + patients (Figure 3(b) and Table 5). Time-varying HR NMAs are presented below, as the proportional hazards over time assumption was violated.

Figure 3.

Results of fixed-effects time-varying hazards network meta-analyses for recurrence-free survival (a) in Stage III melanoma patients with treatment effects as hazard ratio over time relative to observation under the best-fitting second order fractional polynomial model, (p1 = 0, p2 = 0) and (b) in BRAF + melanoma patients with treatment effects as hazard ratio over time relative to observation under the best-fitting 2nd order fractional polynomial model (p1 = 0, p2 = 0).

Table 4.

Time-varying hazard ratios of recurrence-free survival at select follow-up times for competing interventions vs observation, Stage III.

Months	HR vs observation (95% CrI)
	Pembrolizumab	Ipilimumab	Biochemotherapy	Interferon-α2a	Interferon-α2b (12 months)	Interferon-α2b (24 months)	Pegylated Interferon-α2b
3	0.62 (0.50–0.77)*	0.80 (0.64–1.00)	0.34 (0.18–0.65)*	0.74 (0.51–1.07)	0.46 (0.29–0.71)*	0.75 (0.51–1.13)	0.80 (0.65–1.00)
6	0.51 (0.41–0.65)*	0.76 (0.63–0.91)*	0.47 (0.30–0.73)*	0.85 (0.66–1.10)	0.61 (0.45–0.84)*	0.78 (0.59–1.03)	0.83 (0.71–0.98)*
9	0.46 (0.35–0.61)*	0.74 (0.61–0.89)*	0.56 (0.38–0.82)*	0.93 (0.73–1.17)	0.73 (0.56–0.96)*	0.79 (0.61–1.03)	0.85 (0.74–0.98)*
12	0.43 (0.32–0.58)*	0.72 (0.58–0.89)*	0.64 (0.44–0.92)*	0.98 (0.76–1.26)	0.83 (0.64–1.08)	0.80 (0.61–1.05)	0.87 (0.75–1.00)
15	0.40 (0.29–0.57)*	0.71 (0.56–0.90)*	0.70 (0.48–1.03)	1.03 (0.78–1.34)	0.91 (0.70–1.20)	0.81 (0.60–1.09)	0.88 (0.75–1.02)
18	0.39 (0.27–0.56)*	0.70 (0.54–0.91)*	0.76 (0.51–1.14)	1.06 (0.79–1.43)	0.99 (0.74–1.32)	0.82 (0.59–1.13)	0.88 (0.75–1.04)
21	0.37 (0.25–0.55)*	0.69 (0.52–0.92)*	0.81 (0.53–1.26)	1.10 (0.80–1.51)	1.05 (0.78–1.44)	0.82 (0.58–1.17)	0.89 (0.75–1.06)
24	0.36 (0.23–0.54)*	0.69 (0.51–0.93)*	0.86 (0.55–1.38)	1.13 (0.80–1.59)	1.12 (0.81–1.56)	0.83 (0.57–1.20)	0.90 (0.75–1.08)

Statistically significant results.

Table 5.

Time-varying hazard ratios of recurrence-free survival at select follow-up times for competing interventions versus observation, BRAF + sub-group analysis.

Months	HR vs observation (95% CrI)
	Pembrolizumab	Dabrafenib + trametinib	Vemurafenib
3	0.60 (0.41–0.89)*	0.09 (0.06–0.15)*	0.19 (0.08–0.40)*
6	0.48 (0.34–0.66)*	0.24 (0.17–0.32)*	0.48 (0.28–0.79)*
9	0.42 (0.28–0.63)*	0.41 (0.32–0.52)*	0.83 (0.53–1.31)
12	0.38 (0.23–0.63)*	0.60 (0.48–0.76)*	1.23 (0.78–2.00)
15	0.35 (0.19–0.63)*	0.82 (0.64–1.04)	1.66 (1.02–2.90)*
18	0.33 (0.17–0.64)*	1.04 (0.80–1.37)	2.13 (1.23–4.00)*
21	0.32 (0.15–0.65)*	1.28 (0.96–1.74)	2.63 (1.43–5.28)*
24	0.30 (0.14–0.65)*	1.54 (1.11–2.16)*	3.16 (1.62–6.77)*

Statistically significant results.

For RFS in stage III melanoma patients, the HR for pembrolizumab vs observation decreased significantly over time. The superiority of pembrolizumab vs observation became statistically meaningful by 3 months. The HRs for biochemotherapy and IFN-α2b (12 months) vs observation increased significantly over time based on the constructed 95% CrI for the d1 estimate, which does not cross zero. After 9 months of follow-up, pembrolizumab vs observation was statistically differentiated from all regimens in the network except biochemotherapy and ipilimumab as evidenced by no longer overlapping 95% CrIs. Although pembrolizumab was not statistically differentiated from ipilimumab, due to overlapping 95% CrIs throughout all follow-up, and point estimate HRs for both pembrolizumab and ipilimumab are statistically significant after 15 months, pembrolizumab had much lower HR point estimates compared with ipilimumab vs observation (Table 4). Furthermore, all IFN-containing regimens are no longer statistically significantly better than observation after 12 months as shown by their associated 95% CrIs in Table 4.

In BRAF + patients, HR point estimates for pembrolizumab were statistically superior to observation for the follow-up months shown in Table 5. Based on the constructed 95% CrI for the d1 estimate, which does not cross zero, HRs for pembrolizumab vs observation did not statistically vary over time in BRAF + patients. Therefore, pembrolizumab vs observation does not violate the proportional hazards assumption. Despite results for pembrolizumab over time, the increase in HRs of both BRAF-inhibitors vs observation over time was statistically important as confirmed by their associated d1 estimates and corresponding 95% CrIs. Because the HRs of both BRAF inhibitors increased significantly across all follow-up time, there was a statistical advantage for pembrolizumab vs the BRAF inhibitors after 15 months, as determined by no longer overlapping 95% CrIs between pembrolizumab and BRAF inhibitors vs observation (Figure 3(b)). Additionally, Table 5 provides supporting evidence that pembrolizumab offers improved RFS compared to both BRAF inhibitors. Specifically, HRs at time-points after 12 months show that pembrolizumab offers statistically significantly better RFS than observation, whereas both BRAF-inhibitors do not. By 24 months, both BRAF inhibitors are statistically inferior to observation, as evidenced by HRs and 95% CrIs vs observation in Table 5.

Discussion

The SLR identified 12 studies that met PICOS criteria, and were then assessed for heterogeneity in the feasibility assessment. Differences with respect to disease stage and trial characteristics led to the removal of four trials, thus eight trials were included in the final evidence base. Of these eight trials, six were included in the stage III analysis and three were included in the BRAF + sub-group analysis. Findings in the stage III analysis suggest that pembrolizumab had statistically better RFS compared to all interventions after 9 months, with the exception of biochemotherapy and ipilimumab. Throughout time, pembrolizumab was not statistically differentiated from biochemotherapy and ipilimumab, although pembrolizumab did produce numerically better HRs after 6 months compared with both biochemotherapy and ipilimumab. However, comparisons made with biochemotherapy must be made with caution as relative treatment effect estimates are based on one trial, SWOG S0008, which had a small sample size. Additionally, relative treatment effect estimates made with biochemotherapy were mediated by multiple treatments, thereby yielding large CrIs, which prevent statistical differentiation. Similarly, in BRAF + patients, pembrolizumab had statistically significantly improved RFS compared with BRAF inhibitors after 15 months. From these findings it may be inferred that pembrolizumab has better clinical efficacy than other treatments included in these analyses with respect to RFS in high-risk stage III melanoma patients with or without BRAF + mutation. However, in the absence of individual patient data to adjust for differences identified, there is a risk of confounding bias if these differences act as treatment effect modifiers. Thus, differences between the target population and the evidence base should be acknowledged when interpreting results of the NMAs conducted for RFS. Previous NMAs have been conducted assessing RFS for the adjuvant treatment of advanced, resected melanoma. Although findings were similar, all previous NMAs included CheckMate 238, which found that nivolumab was not statistically superior to pembrolizumab or dabrafenib in combination with trametinib in both time-varying and constant HR analyses,. Furthermore, these previous studies confirm that relative treatment efficacy differs among BRAF-inhibitors and that pembrolizumab had statistically better RFS than traditional therapies, such as IFN-containing regimens, based on constant HR analyses. Though our findings are consistent with previous analyses, this analysis relied solely on time-varying HR NMAs, separately assessed BRAF-inhibitors, and excluded CheckMate 238 based on trial and patient characteristic differences such as disease stage and melanoma sub-type, as outlined by NICE and in the feasibility assessment. The validity of an NMA depends on the quality of the RCTs and the extent of any violations in the similarity and consistency assumptions across studies. In an NMA of RCTs involving multiple treatment comparisons, randomization holds only within the individual trials and not across trials. If the different direct comparisons show systematic differences in study and patient characteristics, and these differences are treatment effect modifiers, then the estimates of any indirect comparison as obtained with the NMA will be biased. To assess these risks, a feasibility assessment examining heterogeneity in terms of treatment and outcome characteristics, as well as the study and patient characteristics, was performed. Trials included in the NMAs were largely similar in trial and patient characteristics. As outlined in the PICOS, stage III melanoma was of interest; however, some trials reported disease stages other than stage III. Therefore, in cases where stage II or stage IV patients were included, only stage III sub-group data was used to ensure a homogenous evidence base.

Notably, four trials differed with respect to eligibility criteria, disease stage, and treatment duration. These four trials were removed from the NMA as described in the feasibility assessment to ensure a close match between the analysis population and population of interest in differences that may modify relative treatment effects. Furthermore, because BRAF mutation status is a known treatment effect modifier for BRAF-targeted drugs, a sub-group analysis was conducted in this population to assess relative treatment effects in BRAF + patients only. All analyses were conducted using a time-varying HR model rather than a proportional hazards model, due to violations of the proportional hazards assumption in both the stage III and BRAF + sub-group analyses. Treatments that did not violate the proportional hazards in the stage III analyses were: ipilimumab, IFN-α2a, IFN-α2b (24 months), and pegylated IFN-α2b. Furthermore, pembrolizumab did not violate the proportional hazards assumption in the BRAF + sub-group analyses. However, given the observed statistically significant changes over time for a number of treatments in both the stage III and BRAF + NMA results, the proportional hazards assumption is violated. Consequently, the use of time-varying HR models are more appropriate.

Given the limited number of trials included in all analyses, there was insufficient data to reliably estimate between-study heterogeneity. Consequently, results are based on a fixed-effects model, despite a preference for a random-effects model, because of its assumption that between-study differences in treatment affect can arise from between-study heterogeneity. Because the fixed-effects model used implies that differences in treatment effects between studies can only arise from sampling differences, some credible intervals may be unrealistically narrow and should be interpreted with caution. Moreover, comparisons for pembrolizumab to all competing interventions were based on single trials. Given the structure of the network, comparisons to biochemotherapy were mediated by multiple treatment comparisons, and were, therefore, more uncertain. Additionally, sub-group data was not available for BRAF status by disease stage for BRIM-8, thus the all-comer population was used for the BRAF + sub-group analysis which may impact relative treatment effects. Finally, as stated in the feasibility assessment, CheckMate 238 was removed based on trial and patient characteristic differences. However, the removal of CheckMate 238 no longer allowed relative treatment effects to be assessed in all available treatments for the adjuvant treatment in stage III melanoma. Heterogeneity adjustments were not possible, therefore, this NMA does not represent a complete narrative with respect to all available treatments for adjuvant stage III melanoma.

Despite some limitations to this analysis, there are some recognizable strengths in our analysis. Guided by pre-defined eligibility criteria outlined by the PICOS, the SLR process involved highly sensitive searches of peer-reviewed literature, as well as searches of recent conferences and clinical trial registrations to identity any unpublished completed trials with results available. Although fixed-effects were used due to lack of trial data to generate a stable heterogeneity parameter, the thorough feasibility assessment acted as a way to decrease heterogeneity by removing trials that contained factors which would likely bias relative treatment effects. Traditional NMA reports pairwise HRs, which assumes that treatment effects do not vary over time. Although traditional NMAs were carried out, violations of the proportional hazards were observed for several treatments insinuating that pairwise HR results are not appropriate. The use of time-varying HR NMA rather than traditional NMA allowed for treatments to vary over time according to various fractional polynomial models. This is especially important for treatments that violated the proportional hazards assumption, such as the BRAF-inhibitors, which are known to rapidly lose efficacy after 6 months. With the time-varying HR approach, relative treatment effects are represented by multiple parameters rather than a single parameter, thus allowing more flexibility necessary to accurately depict treatment efficacy over time.

Conclusions

Standard treatment for patients with primary melanoma with or without regional metastases to lymph nodes is surgery followed by adjuvant therapy, but lack of direct evidence comparing standard of care treatment options with newer treatment options, such as immunotherapy, prevents adequate assessment of relative treatment efficacy in patients with higher-risk of recurrent melanoma. This analysis shows RFS benefit provided by pembrolizumab monotherapy over standard of care agents for the adjuvant treatment of stage III melanoma, overall, and for BRAF+, with the benefit over competing interventions increasing over time.