Multiple treatment comparison of seven new drugs for patients with advanced malignant melanoma: a systematic review and health economic decision model in a Norwegian setting.

OBJECTIVE: To assess the relative effectiveness and cost-effectiveness of seven new drugs (cobimetinib, dabrafenib, ipilimumab, nivolumab, pembrolizumab, trametinib and vemurafenib) used for treatment of patients with advanced malignant melanoma in the Norwegian setting.
DESIGN: A multiple technology assessment. PATIENTS: Patients with advanced malignant melanoma aged 18 or older. DATA SOURCES: A systematic search for randomised controlled trials in relevant bibliographic databases.
METHODS: We performed network meta-analyses using both direct and indirect evidence with dacarbazine as a common comparator. We ranked the different treatments in terms of their likelihood of leading to the best results for each endpoint. The cost-utility analysis was based on a probabilistic discrete-time Markov cohort model. The model calculated the costs and quality-adjusted life years (QALYs) with different treatment strategies from a healthcare perspective. Sensitivity analysis was performed by means of Monte Carlo simulation.
RESULTS: Monotherapies with a programmed cell death 1 (PD-1) immune-checkpoint-inhibitor had a higher probability of good performance for overall survival than monotherapies with ipilimumab or BRAF/MEK inhibitors. The combination treatments had all similar levels of effectiveness to the PD-1 immune-checkpoint-inhibitors.PD-1 immune-checkpoint-inhibitors are more effective and more costly compared with ipilimumab in monotherapy. Nivolumab in combination with ipilimumab had higher costs and the same level of effectiveness as the PD-1 immune-checkpoint-inhibitors in monotherapy.BRAF/MEK inhibitor combinations (dabrafenib and trametinib or vemurafenib and cobimetinib) had both similar effectiveness and cost-effectiveness; however, the combination therapies are more likely to give higher quality adjusted life year gains than BRAF or MEK inhibitor monotherapies, but to a higher cost.
CONCLUSIONS: None of the drugs investigated can be considered cost-effective at what has normally been considered a reasonable willingness-to-pay (WTP) in Norway. Price reductions (from the official list prices) in the region of 63%-84% would be necessary for these drugs to be cost-effective at a WTP of €55 850 per QALY. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

To our knowledge, a relative comparison of the different new drugs used for patients with advanced malignant melanoma has not been done by any other study, neither for effectiveness nor for cost-effectiveness.

Important differences between the drugs regarding patients’ health-related quality of life (HRQoL) may not have been captured in our analysis, due to very limited evidence on HRQoL for the alternative interventions.

There is uncertainty with regard to the correct treatment duration in the model, both for the new immunotherapies and the BRAF/MEK inhibitors.

There is lack of documentation regarding the long-term effect of the newer drugs. Further research could change current estimates and consequently the health economic results.

Introduction

Malignant melanoma is the most serious form of skin cancer.1 These tumours originate in the pigment-producing melanocytes in the basal layer of the epidermis.2 Malignant melanoma is divided into four stages, where stage I is the least severe and stage IV the most severe. Stage III includes locally advanced (inoperable, regional disease) and stage IV includes distant metastasis.3 4 The incidence of malignant melanoma in Norway is among the highest in the world5 with 1719 new cases in 2013.6 Malignant melanoma is the cancer type that increases most in Norway.1 For persons aged between 15 and 49 years, this is the second most frequent cancer type for both sexes together.7 A family history of malignant melanoma may be present in 5%–10% of the melanoma cases.3

Surgery is the primary treatment for malignant melanoma and currently also the only potentially curative treatment.5 Early diagnosis and appropriate surgical treatment cures 80%–90% of patients, while 10%–20% experience a relapse as local/regional recurrence or distant spreading.5 Patients with metastatic malignant melanoma have poor prognosis.7 The 5-year relative survival rate for distant melanoma (stage IV) for the period 2009–2013 in Norway was 12.3% for men and 24.5% for women.6

Dacarbazine has been the standard drug treatment for most patients.5 However, such chemotherapy has low response rates and has not been demonstrated to be life-extending.5 Recently, several new drugs have been under development for the treatment of malignant melanoma. As a result of growing clinical experience with these new drugs, treatment of advanced malignant melanoma has changed in the last 2–3 years.5 7

The new drugs have different mechanisms of action: (1) affect the immune system (ipilimumab, nivolumab and pembrolizumab)8; (2) inhibitors of mutated BRAF (serine-threonine protein kinase B-RAF) (dabrafenib and vemurafenib)9 10 or (3) MEK inhibitors (inhibit the mitogen-activated protein kinase pathway) (cobimetinib and trametinib).11 12

The drugs acting on the immune system do so by blocking mechanisms that limit activation of T cells. Activated T cells can be limited by CTLA-4 (cytotoxic T-lymphocyte-associated protein), a coinhibitory molecule of the immune system and by programmed cell death 1 (PD-1) with its ligands PD-L1 and PD-L2, which is expressed in peripheral tissues and cancers.8 Ipilimumab acts by blocking CTLA-4, whereas nivolumab and pembrolizumab block the interaction of the PD-1 receptor with its two ligands PD-L1 and PD-L2.8 13

Forty to fifty per cent of the patients with metastatic malignant melanoma have activated mutations in serine-threonine protein kinase B-RAF (BRAF).7 This knowledge has led to the development of the drugs, dabrafenib and vemurafenib, which are BRAF inhibitors. The use of a MEK inhibitor (cobimetinib or trametinib) together with a BRAF inhibitor may reduce the resistance seen to single agent BRAF inhibitors.14 The MEK inhibitors can also be used as single therapies.

The Norwegian Medicines Agency (NOMA), Canadian Agency for Drugs and Technologies in Health (CADTH) and National Institute for Health and Clinical Excellence (NICE) have performed single technology assessments on drugs used for metastatic melanoma, such as dabrafenib,15 16 ipilimumab,17–20 trametinib21 and vemurafenib.22 23 However, none of these assessments compared the different new drugs for patients with advanced malignant melanoma relatively to each other. We have, therefore, conducted this multiple technology assessment including all new drugs for these patients in a Norwegian setting.

Methods

Literature search and selection of studies

We included randomised controlled trials (RCTs) which examined overall survival (OS), progression-free survival (PFS), health-related quality of life (HRQoL) and serious adverse events (SAEs) in patients aged 18 or older with advanced malignant melanoma. We were interested in the following interventions: cobimetinib, dabrafenib, ipilimumab, nivolumab, pembrolizumab, trametinib and vemurafenib compared with placebo or any drug treatment. The drugs can be used as monotherapy or in combination with each other.

A systematic literature search was performed in February 2015 and updated in September 2015 and in May 2017. We limited the search to trials published in year 2000–2017 since the interventional drugs have entered the market recently. We searched the following databases: Ovid MEDLINE(R) In-Process and Other Non-Indexed Citations and Ovid MEDLINE(R) version 1946 to Present; Embase version 1974 to present; Cochrane Central Register of Controlled Trials (Central); Web of Science; PubMed (epub ahead of print). The complete search strategies are presented in online supplementary material 1. We also searched Google Scholar. The websites of selected health technology assessment agencies were searched in September 2015. We contacted the pharmaceutical companies that have marketing authorisation or represent the interventional drugs to obtain additional information as published articles, abstracts/posters that fulfil our inclusion criteria. We also checked for RCTs in the relevant systematic reviews, reviews or meta-analyses which we identified. We looked for ongoing trials in ClinicalTrials.gov and WHO International Clinical Trials Registry Platform in August 2015.

Two reviewers independently inspected all citations to identify potentially relevant articles based on title and/or abstract. Full text publications were obtained for articles appearing to meet our inclusion criteria. Two persons independently assessed whether the article was relevant or not according to our list of inclusion criteria. Disagreements were resolved by discussion or by consulting a third reviewer. One reviewer extracted data from the included articles and another reviewer checked these results for accuracy.

Statistical analyses and presentation of results

We conducted a systematic review and meta-analyses of existing literature, following the methods recommended in the PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions.24 The protocol for the systematic review can be seen in online supplementary material 2.

We expressed the comparative effectiveness of the treatments as the relative risk (RR) of dichotomous endpoints, HR for time-to-event endpoints and mean difference (MD) for continuous endpoints. If a continuous outcome had been measured/reported using different instruments/scales in the included RCTs, we would have calculated the standardised mean difference (SMD). For all endpoints 95% CIs (results from pairwise meta-analyses) or credible intervals (CrI, results from network meta-analyses) were calculated for the RR, HR, MD or SMD. The CrI is the Bayesian analogue to the CIs used in traditional frequentist statistical approaches. We considered a result ‘significant’ if the CrI did not include RR/HR=1 or MD/SMD=0.

We performed a network meta-analysis (NMA) for each clinical endpoint individually. We did this by combining direct and indirect effects of the interventions of interest for each endpoint. The analysis was based on multiple treatments meta-analysis as described by Salanti.25 We used the arm-based network meta-analysis method (a Bayesian method based on Markov Chain Monte Carlo simulation). All NMAs were performed using Winbugs version 1.4.3 (Imperial College and MRC, UK). The statistical analysis was based on binomial likelihoods (dichotomous endpoints) and normal likelihood (continuous endpoints), with vague priors for the trial baselines, basic parameters (normal distribution with mean 0 and SD 0.0001) and the random effects SD (uniformly distributed in the interval 0–2) and takes the correlation structure induced by multi arm trials into account. For time-to-event endpoints (OS and PFS), with HR as the measure of effect, we used the method described by Woods et al26 to combine HRs, cumulative number of events and median survival statistics. We used a random effects model. We checked for incoherence between direct and indirect evidence by ‘node-splitting’.27 We calculated the direct and indirect estimates of effect and the corresponding Bayesian ‘p values’ for incoherence.

We also ranked the different treatments in terms of their likelihood of leading to the best results for each endpoint. We based the rankings on the surface under the cumulative ranking curve (SUCRA).28 We interpreted the rankings cautiously taking into account the quality of the evidence.

The dose-comparison trials of Hamid et al29 and Robert et al30 showed that the effect of ipilimumab and pembrolizumab did not seem to depend on the doses given. As a consequence, we have combined different doses of ipilimumab and pembrolizumab into one group. We have treated different doses of the other interventions in a similar way; this was however, only relevant for trametinib.

For the endpoint PFS, we assumed that participants who dropped out experienced disease progression if a HR between intervention and control was not reported. For all other endpoints, we did not perform imputations for missing data. We based the statistical analyses on the intention-to-treat principle (all participants analysed in the group to which they were allocated and all available data included in the analyses).

Quality of the evidence

The quality of the included studies was assessed using the Cochrane Collaboration’s tool for Risk of Bias assessment.31 We assessed the overall quality of evidence for each endpoint using GRADE (Grading of Recommendations Assessment, Development and Evaluation). We followed the guidelines provided by the GRADE working group32 and categorised our confidence in the effect estimates into four levels: high, moderate, low and very low. The quality of the direct evidence, indirect evidence and the combined evidence from the NMAs was evaluated using the GRADE approach for network meta-analyses.33 We used the Guideline Development Tool,34 while evaluating the quality of the direct evidence.

Economic evaluation

In order to assess the cost-effectiveness of the interventions targeting patients with metastatic and/or unrespectable malignant melanoma, we made a probabilistic decision–analytic model.

The model has three mutually exclusive disease-related health states: (1) progression-free disease (PFS), (2) progressed disease (PD) and (3) death (see online supplementary material 3). All patients start in the progression-free disease state. For every model cycle, the arrows indicate that a proportion of the patients may remain in the same health state, another proportion may experience progression and another may die, determined by the transition probabilities.

Disease regression in the form of transition from progressed disease to PFS is not possible. The model does not include treatment sequences. The time horizon is 10 years with a monthly cycle-length. The analysis was carried out from a healthcare payer perspective. Costs and quality-adjusted life years (QALYs) were discounted at a rate of 4% per annum. We used R version 3.2.2 for the estimation of the cumulative density functions and baseline transition probabilities and TreeAge Pro 2015 for the decision modelling.

Model parameters

Table 1 shows the key parameters used in the base case analysis.

Table 1

Key parameters used in the base case analysis

	Definition	Value		Source/Note
Probabilities (Fitted distributions for dacarbazine arm)	Dacarbazine	Progression-free survival (Log-logistic)	Overall survival (Log-logistic)	Based on patient-level time-to-event data used to construct Kaplan-Meier curves in Robert et al35/Maio et al36
	Parameters	2.13, 3.16*	1.52, 9.03*
Efficacy estimate		Overall survival (95% CI)	Progression-free survival (95% CI)	Based on the results of our systematic review (table 3 and online supplementary material 16)
	Ipilimumab	0.69 (0.44 to 1.26)	0.84 (0.54 to 1.52)
	Dabrafenib	0.73 (0.49 to 1.10)	0.37 (0.22 to 0.63)
	Dabrafenib+trametinib	0.55 (0.37 to 0.84)	0.21 (0.12 to 0.37)
	Ipilimuab+dacarbazin	0.70 (0.47 to 0.99)	0.76 (0.45 to 1.33)
	Nivolumab	0.45 (0.30 to 0.71)	0.50 (0.36 to 0.82)
	Nivolumab+ipilimumab	0.48 (0.28 to 0.90)	0.35 (0.21 to 0.66)
	Pembrolizumab	0.46 (0.26 to 0.99)	0.47 (0.30 to 0.76)
	Trametinib	0.78 (0.49 to 1.22)	0.45 (0.25 to 0.82)
	Vemurafenib	0.77 (0.54 to 1.10)	0.38 (0.24 to 0.62)
	Vemurafenib+cobimetinib	0.50 (0.26 to 0.96)	0.22 (0.11 to 0.48)
QALY weight	Values that applies to the BRAF or MEK targeted monotherapies†	0.72	0.67	Grob et al41
	Values that applies to the BRAF+MEK targeted combination therapies‡	0.75	0.75	Grob et al41
	Values that applies to the immunotherapies§	0.80	0.70	Köping-Höggård et al42
Costs per patients per year in euros (€1.00≈Kr8.9530)	Drug costs per model cycle	First month	Months beyond the first	Based on Norwegian Medicines Agency 201217 and the information in Summary of Product Characteristics (see online supplementary material 6)
	Dabrafenib	9043	9043
	Dabrafenib + trametinib	18 982	18 982
	Dacarbazine	141	141
	Ipilimumab¶	82 750	0
	Ipilimumab* + dacarbazine	82 750 141	141
	Ipilimumab* + nivolumab	75 675 3219	3219
	Nivolumab	9656	9656
	Pembrolizumab	10 377	10 377
	Trametinib	9939	9939
	Vemurafenib	9331	9331
	Vemurafenib+cobimetinib ¶	19 270	19 270
	Serious adverse events costs	Probability of serious adverse events per cycle	Expected costs per model cycle	Based on the results of our systematic review (see online supplementary material 18) and DRG code for complication (https://helsedirektoratet.no/publikasjoner/innsatsstyrt-finansiering-regelverk)
	Dacarbazine	0.0164	38
	Dabrafenib	0.0167	39
	Dabrafenib+trametinib	0.0167	39
	Ipilimumab	0.0141	33
	Ipilimuab+dacarbazin	0.0244	57
	Nivolumab	0.0099	23
	Nivolumab+ipilimumab	0.0207	48
	Pembrolizumab	0.0080	19
	Trametinib**	0.0167	39
	Vemurafenib	0.0166	39
	Vemurafenib+cobimetinib	0.0178	41
	Monitoring costs in the progression-free health state (per model cycle)			Expert opinion and prices are based on official Norwegian unit prices (https://lovdata.no/dokument/SF/forskrift/20 03–06- 27–959) (for more details, see supplementary material 1)
	Dacarbazine	288
	Ipilimumab, ipilimumab +dacarbazine and pembrolizumab	308
	Nivolumab, nivolumab +ipilimumab	393
	Dabrafenib, dabrafenib +trametinib, vemurafenib, vemurafenib+cobimetinib, trametinib and cobimetinib	388
	Administration costs per medicine injection (per model cycle)††
	Dacarbazine	195		DRG code for injection treatment (https://helsedirektoratet.no/publikasjoner/innsatsstyrt-finansiering-regelverk)
	Ipilimumab	513 ‡‡
	Ipiimumab+ nivolumab	Ipilimumab: 469 ‡‡ Nivolumab: 293
	Ipilimumab+dacarbazine	Ipilimumab: 513‡‡ Dacarbazine: 195
	Nivolumab	293
	Pembrolizumab	195
	Monitoring costs model when treatment is discontinued (per model cycle)	1312		Expert opinion and prices are based on official Norwegian unit prices (for more details, see online supplementary material 6)
	Average BRAF gene mutation testing cost per patients	457		Based on the data from Oslo university hospital
	Dispensing costs per model cycle for oral medicines	46 (cobimetinb+vemurafenib, dabrafenib, dabrafenib+trametinib, vemurafenib) 23 (trametinib)		Expert opinion

*Parameters are defined as in R: shape and scale.

†Vemurafenib, dabrafenib, trametinib.

‡Dabrafenib+trametinb, vemurafenib+cobimetinib.

§Ipilimumab, ipilimumab+dacarbazin, ipilimumab+nivolimab, nivolumab, pembrolizumab.

¶At the time of conducting the analysis, we did not have a maximum pharmacy retail price for cobimetinib. Therefore, we assumed the cobimetinib had the same price as trametinib.

**We had no data to serious adverse events for trametinib and it was assumed that the relative risk for trametinib would be the same as for dabrafenib.

††Cost per treatment €147.

‡‡All doses were ‘administered’ during the first model cycle.

DRG, diagnosis related group; QALY, quality-adjusted life year.

Transition probabilities

To estimate transition probabilities for the models, we first needed to determine the cumulative density functions for OS and PFS for control-arm. For the baseline OS and PFS, we fitted cumulative density functions from the dacarbazine arm of a RCT, published by Robert et al35 and Maio et al.36 We used patient-level time to event data to construct Kaplan-Meier curves in these studies. The two trials by Robert et al35 and Maio et al36 have both only included patients that were not previously treated (table 1 in online supplementary material 4). The choice of cumulative density function was based on the best fit as evaluated by the Akaike information criterion.

We used the transition probability formula suggested by Briggs et al37 as the basis for calculating transition probability from alive to death (the transitions from health state 1 to 3 and from 2 to 3) and from PFS to progressed disease (1 to 2). From the fundamental relationships for probabilities of mutually exclusive events, it follows that the transition probability from PFS to progressive disease (1 to 2) is one minus the probability of death minus the probability of staying in the PFS health state.

In practice, however, trial results report only the OS rate, which includes survival from both the progression-free and progressed disease states. Using the fundamental relationship described above can lead to double counting. To avoid this possibility, we calculated the transition probability from PFS to PD in two steps (see online supplementary material 3), first with regard to death (OS) and then for progression, conditional on having survived. Online supplementary material 5 gives more explanation for the choices made.

Treatment effects

We used the HRs relative to dacarbazine from the network meta-analysis (table 1) to adjust the baseline transitions probabilities.

In the base case analysis, the HRs are applied up to 2 years, assuming no treatment effects past 2 years of treatment for any of the interventions. Any accumulated survival and progression benefits would, however, have an impact also after treatment discontinuation.

Costs

All costs were expressed in 2015 Norwegian kroner and converted into Euros based on the average exchange rate for Norwegian kroner to Euros in 2015 (€1.00≈Kr8.9530) reported by the Norwegian Central Bank. Table 1 summaries all costs included in the model.

BRAF gene mutation diagnostic testing

It has become current practice in Norway to test all patients with unresectable and/or metastatic malignant melanoma for the BRAF gene mutation. Information on the mutation status of the patient can be used to determine whether the patient should receive immunotherapy or BRAF gene mutation targeted therapy. In our model, the only treatment arm not tested for genetic mutations is the dacarbazine-arm. This arm was included as a comparator to reflect the situation as if the new therapies were not an option. In such a context, the results of the test would not add valuable information, as the treatment would be the same independently of the test result.

Based on expert’s opinion, we assumed that all patients are tested with a BRAF qPCR test and around 15% with a BRAFpyro. The BRAF gene mutation rate among Norwegian patients with malignant melanoma is around 50%. All patients who do not have the BRAF gene mutation are subsequently tested for neuroblastoma RAS viral oncogene homologue gene mutation. Our average cost estimate for the cost of testing is €457 per patient.

Medicine costs

The medicine costs depend on the acquisition price, the dosages and duration of treatment. Drug costs included in the model reflect the maximum pharmacy retail price, including VAT. We planned to conduct sensitivity analyses to determine the price at which each drug can be considered cost-effective. The dosages we used correspond to the information in the summary of Product Characteristics and an overview of the dosages used is provided in online supplementary material 6.

At the time of writing, we did not have a maximum pharmacy retail price for cobimetinib. In the base case of the model, we chose to assume that cobimetinib has the same price as trametinib, that is, the combination cost was the sum of the monthly cost of vemurafenib in monotherapy plus trametinib, resulting in a combined monthly drug cost of about €19 270 per month.

In clinical practice, the actual cumulative dose may, however, be lower than the planned cumulative dose, for instance due to drug intolerance. The relative dose intensity of dabrafenib in the Combi-D study was about 88% (personal communication). We used the same relative dose intensity for dabrafenib in combination with trametinib, trametinib, vemurafenib and vemurafenib in combination with cobimetinib in the model, based on expert advice.

Based on experience from clinical practice in Norway, the number of doses each patient receives on average with ipilimumab is likely to be less than 4. From the relative dose intensity information reported in Postow et al,38 we calculated a weighted average for ipilimumab in monotherapy of 3.5 doses and in combination therapy (with nivolumab) of 3.2 doses.

For the PD-1 immunotherapies, nivolumab and pembrolizumab, we were advised from our clinical experts to assume that treatment could be provided both in the progression-free and progressed health states. In addition, we assumed a gradual decline in the proportion being treated over time of those alive, according to information given in the appendix to Larkin et al39 at 12 months of follow-up and assuming no treatment after 2 years.

There is no accumulation of medicine costs past 2 years due to the assumption that treatment stops at this time. In online supplementary material 7, we have summarised all the assumptions we have made in the base case model.

In table 1, we have summarised the medicine costs estimates per model cycle (month), with the exception of ipilimumab, where the total drug cost is valid only for the first model cycle. For more details about the drug cost estimation and drug dosages, see online supplementary material 6.

Drug dispensing costs

We estimated that 80% of the patients receiving medicines in tablet form would prepare their doses alone, while the other 20% would get help from either a nurse (inpatients and homecare patients) or the hospital’s pharmacy (outpatients). We assumed that the dispensing costs would be about €46 per month (based on expert opinion).

Parenteral drugs administration costs

Dacarbazine, ipilimumab, nivolumab and pembrolizumab are intravenous therapies, administered by a nurse who supervises the whole procedure. For all therapies given intravenously at hospital, we assumed that administration costs per dose were reflected by 7% of the reimbursement, that is, €147 (included wages, material and overhead costs).

Drug-therapy related SAEs

We chose to include SAEs requiring hospitalisation, that is, adverse events grade 3 and 4. The monthly costs related to SAEs are determined by the cost of hospitalisation and the average monthly probability of such an event.

The costs of an adverse event-related hospitalisation were assumed to be equal to 100% of the reimbursement, that is, €2325. We estimated the average monthly rate of a SAE from patients included in the dacarbazine arm of the NCT00324155 trial published in Robert et al35,13 the only publications without crossover after progression where in addition dacarbazine was administered as monotherapy. The average monthly rate of 1.64% is the arithmetic average of the monthly frequency of experiencing SAEs.

We adjusted the baseline risk of an adverse event with the RR of adverse event versus dacarbazine identified in our meta-analysis (see online supplementary material 8). Table 1 summarises the expected cost per model cycle for each treatment arm and the associated cycle probability.

Any serious treatment related adverse events post progression was assumed to be included in the monitoring cost for the progressed disease state.

Monitoring costs

In the progression-free health state, patients are followed up during and after treatment stop in order to assess the course of the disease. While in treatment, the intensity and content of this follow-up schedule varies across interventions and includes outpatient visits to specialists (oncologists and/or dermatologists), blood analyses and diagnostic imaging (CT, ultrasound, bone scintigraphy, PET and/or MR). The resulting cost estimates are shown in table 1. table 2.

Table 2

Results when all incremental cost-effectiveness ratios refer to dacarbazine (time-horizon of 10 years)

Interventions	Costs (€)	Incremental cost	Effectiveness (QALYs)	Incremental effect	ICER (€/QALY)
Dacarbazine*	17 995		0.88
Trametinib†	100 708	82 714	1.16	0.28	295 405
Dabrafenib†	105 329	87 334	1.23	0.35	249 526
Vemurafenib†	105 394	87 399	1.19	0.31	281 932
Ipilimumab†	106 788	88 793	1.36	0.48	184 985
Ipilumab+dacarbazine†	107 071	89 077	1.28	0.40	222 692
Nivolumab*	118 792	100 798	1.70	0.82	122 924
Pembrolizumab†	121 325	103 330	1.68	0.80	129 162
Nivolumab+ipilimumab†	168 531	150 537	1.69	0.81	185 848
Vemurafenib+cobimetinib*	276 455	258 460	1.77	0.89	290 405
Dabrafenib+trametinib†	277 648	259 654	1.71	0.83	312 836

*Undominated.

†Dominated (including extended dominance).

QALY, quality-adjusted life year.

In the progressed health state the costs consists of a mix of surgery, radiotherapy and palliative treatment at a hospital centre and/or through day care. Although we assume that some patients may be treated with PD-1 immunotherapy post progression, we consider a mix of best supportive care to best reflect the per cycle costs in the progressed health state.

See online supplementary material 6 for more details regarding our monitoring costs estimates.

Health-related quality of life

HRQoL data are based on the systematic search. We conducted a systematic search for published utility weights that were relevant for our model population and treatment options. For consistency, and noting that different utility instruments can yield different utility weights for the same health state, we focused on values based on EQ-5D, the most commonly used instrument.40

We used EQ-5D values from Grob et al41 for vemurafenib in monotherapy and dabrafenib and trametinib in combination therapy to inform the PFS and progressed disease health states for those interventions. Due to lack of data, these values were also used for dabrafenib monotherapy and the combination vemurafenib and cobimetinib.

Grob et al41 indicated a slight decrease in HRQoL following progression for vemurafenib in monotherapy, but an increase for the BRAF and MEK combination therapy. We chose not to include this increase in the analysis for the combination therapies, instead fixing the HRQoL at the same level as in the progressive free health state.

The EQ-5D values for the interventions involving immunotherapies are derived from a published single technology assessment of pembrolizumab compared with ipilimumab.42

Table 1 shows the EQ-5D values we used in the model analysis for PFS and progressed disease. The immunotherapies have a slightly higher HRQoL in the PFS health states than the BRAF and MEK monotherapies and combination therapies, but the BRAF and MEK in combination therapies have a slightly higher HRQoL in the progressed disease state.

We used the BRAF or MEK monotherapies HRQoL values for the common comparator dacarbazine.

Sensitivity analysis

To explore the consequences of uncertainties in the base-case parameter values, we performed a probabilistic sensitivity analysis (PSA) in which all input parameters were randomly drawn from probability distributions and the model was run 10 000 times. For probabilities and utility estimates, the beta-distributions were constructed on plausible ranges for parameters. For cost, we modelled gamma-distributions. Efficacy and safety parameters were assumed to have a log-normal distribution where mean and CIs from our network meta-analysis were used to estimate the parameters. All the model parameters and distributions are presented in online supplementary material 9. Our suggestions about cost-effectiveness will reflect a range of potential willingness-to-pay (WTP) values per gained QALY. Separate scenario analysis will investigate the importance of drug pricing, the choice of time horizon and HRQoL weights.

We also performed an analysis of the expected value of perfect information (EVPI) in order to explore the uncertainty surrounding specific groups of parameters and to show which groups of input parameters it is most useful to conduct further research. The EVPI for the parameter or group of parameters is the difference between the expected net benefit with perfect (additional) information and the expected value with current information about the parameter(s).37

Results

Description of included studies

We identified 704 citations, additionally two citations were identified through manual search and 75 citations were received from the manufactures. Finally, 40 publications (24 full text publications and 16 abstracts/posters) met the prespecified inclusion criteria (figure 1). Those publications represent 17 unique clinical trials. The excluded publications, including reasons for the exclusions, are given in online supplementary material 10. Our searches in websites of sister health technology assessment agencies in August 2015 did not identify further trials; for details, see online supplementary material 1. Possible relevant ongoing trials are listed in online supplementary material 1.

Figure 1

Flowchart of identification and selection of documentation.

All the 17 RCTs were mostly of phase III, three trials were phase II43–45 and one was phase I.46 Most of the trials (10–17) were open-labelled, the other seven trials were double-blinded.11 13 14 35 47–49 All were multicentre trials, and the majority were performed in North America and Europe. A total of 7482 patients were included in the 17 trials, with a range from 59 to 945 patients in the respectively trials. The 17 trials (9–15, 37–46) were published from 2010 to 2015 in a total of 40 publications.8–14 35 36 41 43–71 Most of the publications were of new date, with 24 of the 40 publications published from 2014 and later. An overview of the included RCTs is presented in online supplementary material 4.

The trials included patients≥18 years, with advanced malignant melanoma. The median age ranged from 49 to 67, and the proportion of males ranged from 49% to 74%. The majority of the trials (10 of 17) included patients previously pharmacologically untreated. Four of the trials included both previously treated and untreated patients8 12 43 69 and three of the trials included only previously pharmacologically treated patients.44 47 72

Patients with identified BRAF V600 mutation, either specifically identified as BRAF V600E9 10 or BRAF V600E or V600K11 12 14 43 69 were included in the seven trials with BRAF and/or MEK inhibitors as the intervention. Patients who were included in the trials where the intervention acts on the immune system, had not identified any BRAF mutation, except in four trials44 48 49 54 that identified both patients with and without BRAF mutation.

More details can be seen in online supplementary materials 4 and 11.

Seven of the trials allowed patients in the control group to cross over to the intervention group after progression.44 45 49 57 63 64 71 McArthur et al63 reported results for OS and PFS both as censored at the time of crossover and without censoring at the time of crossover. In our analyses, we only included the data without censoring at the time of crossover, since this method was used in the other trials.

Dacarbazine was used as the comparator in seven of the trials,9 10 13 35 44 69 72 including three trials were the control groups were the investigator choice of chemotherapy: dacarbazine or paclitaxel,69 dacarbazine or paclitaxelin combination with carboplatin,72 or dacarbazine as one of five investigator-choice chemotherapies.44 72 The other comparators used in the included trials were ipilimumab for six of the trials,8 45–49 nivolumab in one trial,48 dabrafenib in two trials14 43 and vemurafenib in two trials.11 12

Since all the interventions, except cobimetinib, were compared with dacarbazine, directly or indirectly, this was used as our common comparator in our network meta-analyses. The three arms using investigator choice of chemotherapy44 54 69 were considered to be equivalent to dacarbazine. Consequently, these three arms were included as dacarbazine arms in the statistical analyses.

OS was measured with a follow-up time from 5 months10 to 5 years.36 When one study reported from more than one follow-up time points, we choose to extract from the first report measuring 2 years survival (if available) as well as from the latest available data. In our network meta-analyses we use the 2 years data.

We assessed the risk of bias for the endpoints in the included RCTs to be from low to high risk, mostly of low risk of bias; for more details, see online supplementary material 11.

Clinical effectiveness

Our results for the relative comparisons of the included drugs or combinations of drugs are based on the network meta-analyses. For OS, PFS and SAEs all the interventions/treatment strategies were included in the network; hence, they could all be compared relative to dacarbazine. Due to insufficient data we could not perform a network meta-analysis for HRQoL. Below, we present results from all the comparisons with available direct evidence. Estimate HRs and quality ratings for direct and indirect comparisons based on the network meta-analysis analyses can be found in online supplementary materials 12-14 for the outcomes OS, PFS and SAEs, respectively.

The evidence network for OS is shown in figure 2. The results of the random effects network meta-analysis for the comparisons between the interventions and the common comparator dacarbazine are presented in table 3. Table 3 also shows a ranking of the included treatments using the SUCRA, and a grading of the evidence from the network meta-analyses is presented.

Figure 2

Evidence network for overall survival.

Table 3

HRs for overall survival from network meta-analysis

Intervention	HR relative to dacarbazine (Network meta-analysis)	SUCRA	Quality of evidence from the network meta-analysis
Nivolumab	0.45 (0.30 to 0.71)	0.85	Moderate
Pembrolizumab	0.46 (0.26 to 0.99)	0.81	Very low
Nivolumab+ipilimumab	0.48 (0.28 to 0.90)	0.78	Low
Vemurafenib+cobimetinib	0.50 (0.26 to 0.96)	0.73	Moderate
Dabrafenib+trametinib	0.55 (0.37 to 0.84)	0.68	Low
Ipilimumab+dacarbazine	0.70 (0.47 to 0.99)	0.41	High
Ipilimumab	0.69 (0.44 to 1.26)	0.40	Very low
Ipilimumab+gp100	0.72 (0.40 to 1.55)	0.36	Very low
Dabrafenib	0.73 (0.49 to 1.10)	0.35	Moderate
Trametinib	0.78 (0.49 to 1.22)	0.30	Low
Vemurafenib	0.77 (0.54 to 1.10)	0.29	Moderate
Dacarbazine	1	0.05	–

SUCRA, surface under the cumulative ranking curve.

We found that nivolumab, pembrolizumab, nivolumab combined with ipilimumab, vemurafenib combined with cobimetinib and dabrafenib combined with trametinib all have better OS than dacarbazine. The ranking as measured by the SUCRA suggests that nivolumab, pembrolizumab, nivolumab combined with ipilimumab, vemurafenib combined with cobimetinib and dabrafenib combined with trametinib have a higher probability of good performance than the other available treatment strategies (table 3).

The evidence network for PFS is shown in online supplementary material 15. In addition, online supplementary material 16 shows a ranking of the included treatments using the SUCRA together with the quality of evidence.

The ranking as measured by the SUCRA suggests that dabrafenib combination with trametinib and vemurafenib combined with cobimetinib has a higher probability of better performance for PFS than the other available treatment strategies (see online supplementary material 16).

The evidence network for SAEs is shown in online supplementary material 17.

In addition, online supplementary material 8 shows a ranking of the included treatments using the SUCRA together with the quality of evidence.

Based on the results of the network meta-analysis, we could not establish any differences between the available treatment strategies. However, the ranking as measured by the SUCRA suggests that pembrolizumab and nivolumab has a higher probability of fewer SAEs than the other available treatment strategies, even though we could not establish any differences (see online supplementary material 8).

The full network-meta-analysis results comparing all available treatment strategies for OS, PFS and SAEs are presented in online supplementary material 18.

More details about the estimates of OS, PFS and SAEs with quality rating (GRADE) for direct and indirect evidences are shown in the Summary of Finding tables in online supplementary material 19.

Health-related quality of life

Due to insufficient data, we have not performed network meta-analyses for HRQoL. Online supplementary material 20 presents a descriptive overview of the evidence for HRQoL. More details of the studies are found in online supplementary material 11.

Cost-effectiveness analysis

The results of the incremental analysis are presented in table 2. Nivolumab and the combination of vemurafenib and cobimetinib were two undominated treatment strategies. Nivolumab had an incremental cost-effectiveness ratio against dacarbazine of about EUR

123 000 per QALY gained. Vemurafenib in combination with cobimetinib had an incremental cost-effectiveness ratio against nivolumab of about €2 million per QALY gained.

Figure 3A shows the scatter plot for 10 000 Monte Carlo simulations. From the top right corner, are the combinations dabrafenib in combination with trametinib and vemurafenib in combination with cobimetinib and the combination nivolumab and ipilimumab. The largest cloud of scatter consists of the BRAF inhibitors dabrafenib, vemurafenib and the MEK inhibitor trametinib, the immunotherapies ipilimumab, nivolumab and pembrolizumab.

Figure 3

(A) Scatter plot for 10 000 Monte Carlo simulations. (B) Cost-effectiveness acceptability curve with dacarbazine. QALY, quality-adjusted life year.

The cost-effectiveness acceptability curves are presented in figure 3B. The results showed that nivolumab was the most cost-effective strategy for WTP values above €123 000. However, both nivolumab and pembrolizumab had quite similar percentages of iterations being cost effective.

The results of EVPI indicated that the treatment efficacy data was the most influential source of uncertainty, followed by the HRQoL data, costs and SAEs data (HR for SAE) (the figure is presented in online supplementary material 21).

Results for the immunotherapies

Nivolumab was the undominated strategy and had an incremental effect of 0.82 QALYs and an incremental cost-effectiveness ratio compared with dacarbazine of about €123 000 per QALY gained.

When dacarbazine is excluded from the comparison nivolumab and pembrolizumab had very similar costs and effectiveness. The combination therapy nivolumab and ipilimumab had higher costs and the same level of effectiveness as nivolumab or pembrolizumab monotherapy. Online supplementary material 22 shows the incremental analysis when dacarbazine is excluded from the comparison.

When dacarbazine is included as a comparator (figure 3B), the PSA shows that nivolumab is more likely to be cost effective than the alternatives for WTP values exceeding about €123 000. Pembrolizumab is however not far behind with respect to the percentage of iterations being cost-effective. When dacarbazine is excluded from the analysis (see online supplementary material 23), ipilimumab in monotherapy is more likely to be cost effective than the alternatives for WTP values below €34 000, where after nivolumab is more likely to be cost effective than the alternatives for increasing WTP values.

Results for the BRAF and MEK monotherapies and combination therapies

Dabrafenib and vemurafenib in combination with cobimetinib were the two undominated strategies. Dabrafenib had an incremental effect of 0.35 QALYs and a cost-effectiveness ratio compared with dacarbazine of approximately €250 000 per QALY gained. The combination vemurafenib in combination with cobimetinib have an incremental effect of 0.54 QALYs and incremental cost-effectiveness ratio compared with dabrafenib of about €320 000 per QALY gained.

The results of PSA showed that dacarbazine is more likely to be cost effective than the alternatives for WTP values up to about €223 000. The monotherapies showed a decreasing trend for further increasing WTP values, contrary to the combination strategies. When dacarbazine is excluded, the BRAF and MEK inhibitors in monotherapy had the highest probability of being cost-effective for the lowest WTP values and the combination strategies showing an increasing trend for very high WTP values.

Scenario/sensitivity analyses

The results of the PSA showed that in the BRAF and MEK group, the combination therapies were more effective (in form of QALY) but more expensive than monotherapies. For the immunotherapies, the new available treatment alternatives (nivolumab and pembrolizumab) was more effective but more costly than ipilimumab monotherapy and ipilimumab in combination with dacarbazine. The results showed that the combination of nivolumab and ipilimumab had about the same level effectiveness as nivolumab and pembrolizumab in monotherapy, but was clearly more expensive than the other immunotherapies alternatives (figure 3A).

In addition to the PSA, we performed some scenario analyses to test the uncertainty around some of the model assumptions and the input parameters.

Alternative drug prices

All the prices for the new drug interventions in our analysis are maximum pharmacy retail prices. To gain a clearer idea of cost-effectiveness, we conducted one-way sensitivity analyses to determine at what price each treatment would be considered cost-effective for a WTP of €55 850.

The maximum pharmacy retail price would have to be reduced by approximately 79% for dabrafenib, 83% for trametinib, 84% for dabrafenib in combination with trametinib, 81% for vemurafenib, 84% for vemurafenib in combination with cobimetinib, 75% for ipilimumab, 63% for nivolumab and 64% for pembrolizumab. For the combination ipilimumab and dacarbazine, the drug cost of ipilimumab would need to be reduced by approximately 82%. For the combination nivolumab and ipilimumab, a combined price reduction of about 76% would be necessary. Table 4 presents the results of the analysis.

Table 4

Results of drug price scenario when the interventions refer to an incremental cost-effectiveness ratio of €55 850 per QALY gained relative to dacarbazine

Interventions	Drug price at which treatment is cost-effective (WTP = €55 850)	Drug cost per month in model	Required rebate for cost-effectiveness at WTP = €55 850 (%)
Dabrafenib	1899	9040	79
Trametinib	1675	9939	83
Dabrafenib+trametinib	3127	18 982	84
Vemurafenib	1787	9331	81
Vemurafenib+cobimetinib	3127	19 270	84
Ipilimumab*	20 328	82 750	75
Ipilimumab†+Dacarbazine	15 190	82 750	82
Nivolumab	3574	9656	63
Nivolumab +Ipilimumab‡	33 508	138 473	76
Pembrolizumab	3686	10 377	64

*The ipilimumab cost is not per month, but the full treatment drug cost.

†Reduction applies to ipilimumab only.

‡The price reduction was estimated from the total cumulative drugs cost per patient, which combines the cost of ipilimumab and nivolumab.

WTP, willingness-to-pay.

Treatment until progression also for the PD-1 immunotherapies and different assumptions about treatment extension

We changed the structural assumption regarding treatment with nivolumab and pembrolizumab, from treatment independent of progression status and gradual reduction in the proportion being treated of those alive over time, to treatment in the progression free health state only, for 2 years. We also extended the treatment duration from 24 months to 36 months in two scenarios, assuming further treatment for 50% and 100% of those in the progression free health state at 2 years following treatment. The changes were only applied to the cost side of the model.

The results showed that costs are likely to be underestimated for nivolumab and pembrolizumab in monotherapy when the treatment duration is limited to 24 months and to the progression free health state only. The scenario that assumes that 100% of those in PFS at 24 months will be treated for another 12 months is the closest to our base case scenario. Our base case cost estimates for the immunotherapies are independent of the transition probability from PFS to progressed disease.

Time-horizon (5 years vs 10 years)

A time-horizon of 10 years was considered in the base-case analysis. We performed a scenario analysis where a time-horizon of 5 years was taken. The results showed that the time-horizon of 5 years influenced both costs and effects. However, the ranking of the interventions in the incremental analysis is unchanged when we change the time-horizon to 5 years instead of 10 years as in the base case (see online supplementary material 24). All other assumptions are the same as in the base case.

Quality of life weights

We performed a scenario analysis where we used the standard gamble weights from Beusterien et al73 for all interventions, which were 0.80 for PFS and 0.52 in progressed disease. Although some changes in the results can be observed, the choice of HRQoL weights and assumptions about different quality of life weights across the interventions does not seem to be decisive for the results of our cost-effectiveness analysis.

Discussion

We have assessed the comparative effectiveness and cost-effectiveness of seven new drugs used for patients with advanced malignant melanoma in the Norwegian setting. The results are based on 17 RCTs. Our conclusions for the relative effectiveness of the included drugs or combinations of drugs rest on network meta-analyses using both direct and indirect evidence with dacarbazine as a common comparator. We ranked the different treatments in terms of their likelihood of leading to the best results for each clinical endpoint. The rankings were interpreted cautiously taking the quality of the evidence into account. The cost-utility analysis was based on a probabilistic discrete-time Markov cohort model.

When all drug interventions were compared, nivolumab, pembrolizumab, nivolumab in combination with ipilimumab, vemurafenib in combination with cobimetinib and dabrafenib in combination with trametinib had a higher probability of good performance for OS than monotherapies with ipilimumab or BRAF/MEK inhibitors.

Dabrafenib in combination with trametinib and vemurafenib in combination with cobimetinib had a higher probability of good performance for PFS than the immunotherapies, as well as the monotherapies of BRAF and MEK inhibitors. Ipilimumab had poorer PFS than monotherapy with the BRAF inhibitors as well as the BRAF inhibitors in combination with a MEK inhibitor.

Pembrolizumab and nivolumab seem to have a higher probability of fewer SAEs than the other treatment strategies, even though we could not establish any significant differences. However, since the quality of the evidence for SAEs were low or very low in most of our assessments (due to very few events and CrIs that included both benefit and harm), we have little confidence in these results. We have no results from network meta-analyses for HRQoL.

We find it difficult to separate the new immunotherapies (PD-1) nivolumab and pembrolizumab with respect to cost-effectiveness. They are both more effective and at the same time more costly compared with ipilimumab. Based on the cost-effectiveness results, we cannot argue that any of the BRAF or MEK inhibitor monotherapies (dabrafenib, vemurafenib, trametinib) should be preferred over another or that any BRAF/MEK combination (dabrafenib and trametinib or vemurafenib and cobimetinib) should be preferred over another. However, the combination therapies are more likely to give the highest quality adjusted life year gains.

None of the drugs investigated can be considered cost-effective at what has normally been considered a reasonable WTP in Norway. Drug price reductions (from official list prices) in the region of 63% –84% would be necessary for these drugs to be cost-effective at a WTP of €55 850 per QALY.

Comparison with other studies

The Norwegian Medicines Agency (NOMA), CADTH, The Swedish Dental and Pharmaceutical Benefits Agency (TLV) and NICE, UK have performed single technology assessments on drugs used for metastatic melanoma. This has been done for dabrafenib versus dacarbazine16 74 (NICE, TLV), dabrafenib in combination with trametinib versus dabrafenib15 75 (CADTH, TLV), dabrafenib in combination with trametinib versus vemurafenib15 (CADTH), ipilimumab in combination with gp100 versus ipilimumab in previous treated patients17–19 76 (NMA, CADTH, NICE, TLV), ipilimumab in previous untreated patients (indirect comparisons to dacarbazine, vemurafenib and dabrafenib20 77 78 (NICE, CADTH and TLV), trametinib versus dacarbazine21 (CADTH) and vemurafenib versus dacarbazine 22 23 79 (NICE, CADTH and TLV). The results of these single technology assessments have been presented in online supplementary material 25.

In our multiple technology assessment, we have included all the clinical trials used in these single technology assessments. None of these assessments has compared all the different new drugs for patients with advanced metastatic malignant melanoma relatively to each other. We have chosen not to explicitly compare our incremental cost-effectiveness results with the results of other published single technology assessments. Any such comparison would be highly dependent on how data on clinical effectiveness was used in the model, structural assumptions in the decision model and differences in cost data.

Strengths and limitations of the study

We found only two head to head comparison for the included drugs as monotherapies and five direct comparisons of combination treatment versus monotherapy. None of the included trials compared a BRAF inhibitor (dabrafenib or vemurafenib) head to head with a drug acting on the immune system. The best available comparisons are the indirect evidences via dacarbazine as a common comparator. All the interventions could be included in the network meta-analyses for OS, PFS and SAEs. HRQoL and SAEs are of importance for the patients. However, from the available literature we were not able to find data usable for our network meta-analysis for HRQoL, and the quality of the evidence for SAEs were low or very low in most of our assessments.

We only included RCTs and clinical endpoints were all defined and harmonised in their definitions across the trials.

The number of available interventions for patients with advanced malignant melanoma is evolving rapidly. Many of the interventions in this multiple technology assessment have just reached marketing authorisation in Norway, and the available evidence from RCTs is quite limited. Hence, clinical effectiveness data have the uncertainty that the majority of the evidence for the included comparisons were based on a single study. It cannot be ruled out that new evidence from future RCTs may have the potential to change the ranking of the interventions both with regard to effectiveness and cost-effectiveness.

Seven of the trials allowed patients in the control group to cross over to the intervention group after progression. The statistical analyses of the endpoint OS was based on the intention to treat population in all seven trials, that is, the analyses were done without taking the crossover into account. The assumption underlying the statistical analysis is that crossover does not alter mortality patterns seriously. However, we decided to include these data since the results for the intervention will not be favoured.

One of the trials that allowed crossover of patients from the control group to the intervention group did, however, present results from both analyses where participants who crossed over were censored at the time of crossover and analyses without censoring.61 They reported a slightly lower HR for death from the analysis with censoring at the time of crossover than from the analysis without censoring at the time of crossover. In our network meta-analyses we only included the data without censoring at time of crossover, since this was what we had for all the other trials.

Latimer et al80 found that adjusting for switching resulted in lower HRs than those obtained from standard ITT analyses, however, CI are wide and results are sensitive to the assumptions associated with each adjustment method.

We believe that the economic model distinguished the interventions fairly well with regard to costs and OS, but not so well with regard to HRQoL.

The use of different quality of life data resulted in different QALYs gained. However, the choice of HRQoL weights and assumptions about different quality of life weights across the interventions does not seem to be decisive for the results of our cost-effectiveness analysis.

Due to paucity in data, the network meta-analyses were not performed separately for the subgroups of patients with respect to those who were previously treated and those who were previously not treated. Therefore, we did not perform separate cost-effectiveness analyses for these two subgroups of patients. However, for cost-effectiveness, we performed three separate subgroup analyses: (1) including all interventions, (2) including only the BRAF and MEK inhibitors; monotherapies and combination therapies and (3) including only the immunotherapies.

There is lack of documentation regarding the long-term effect of the newer drugs. Further research could have the potential to change current estimates and consequently the health economic results.

None of the drugs investigated can be considered cost-effective at what has normally been considered a reasonable WTP in Norway. Drug price reductions (from the official list prices) in the region of 63%–84% would be necessary for these drugs to be cost-effective at a WTP of €55 850 per QALY.

Monotherapies with a PD-1 immune-checkpoint-inhibitor had a higher probability of good performance for OS than monotherapies with ipilimumab or BRAF/MEK inhibitors. The combination treatments had all similar levels of effectiveness to the PD-1 immune-checkpoint-inhibitors.

PD-1 immune-checkpoint-inhibitors are more effective and more costly compared with ipilimumab in monotherapy. Nivolumab in combination with ipilimumab had higher costs and the same level of effectiveness as the PD-1 immune-checkpoint-inhibitors in monotherapy.

BRAF/MEK inhibitor combinations (dabrafenib and trametinib or vemurafenib and cobimetinib) had both similar effectiveness and cost-effectiveness; however, the combination therapies are more likely to give higher quality adjusted life year gains than BRAF or MEK inhibitor monotherapies, but to a higher cost.