Survival, cost and added therapeutic benefit of drugs granted early access through the French temporary authorization for use program in solid tumors from 2009 to 2019.

Decisions on market authorization (MA) and reimbursement have different durations across countries because of health technology assessment (HTA) procedures and negotiations between manufacturers and national authorities. To overcome this delay, France has implemented a Temporary Authorization for Use (ATU) program that allows early access to drugs before MA, in order to treat patients with unmet medical needs. The objectives of our study were to establish the added therapeutic benefit (ATB) of ATUs for solid tumors and to investigate the correlations between three tools evaluating ATB and survival outcomes and drug costs. Data on ATUs granted from January 2009 to December 2019 to treat solid tumors were analyzed. An assessment of their ATB was conducted using the American Society of Clinical Oncology-Value Framework (ASCO-VF), the European Society for Medical Oncology-Magnitude Clinical Benefit Scale (ESMO-MCBS) and the French HTA criterion, clinical added value (CAV). The latter score determines reimbursement and national market access. Thirty-five drugs in 39 indications were granted ATUs. All of them obtained MA and derived a clinical benefit to be reimbursed by the Social Security. Twenty-eight (71.8%) had CAV compared to preexisting therapies. 24/38 (63.2%) had a 4-5 ESMO-MCBS score and 19/33 (57.6%) had an ASCO-VF score over 45. No correlations were found between cost, PFS, OS, CAV and ASCO-VF score, while high ESMO-MCBS scores were correlated to OS. In conclusion, many patients were treated with innovations before MA thanks to ATU, although there are discrepancies between ATB scales, hence the importance of international collaboration in the evaluation of innovative therapies.

actual clinical benefit

French National Agency for Safety of Medicines and Health Products

American Society of Clinical Oncology—Value Framework

added therapeutic benefit

temporary authorization for use

clinical added value

Economic Committee for Health Products

disease‐free survival

Early Access Program

European Medicines Agency

European Society for Medical Oncology—Magnitude of Clinical Benefit Scale

Food and Drug Administration

French National Authority for Health

hazard ratio

health technology assessment

intraclass correlation coefficient

marketing authorization

nonsmall cell lung cancer

overall survival

progression‐free survival

quality of life

response rate

small cell lung cancer

Tarification À l'Activité

Transparency Committee

INTRODUCTION

In recent decades, considerable improvements in the effectiveness of anticancer agents have been made with the development of precision medicine. Patient outcomes have significantly improved , but the cost of these therapies raises the issue of durable access for all patients. , , To speed up access, drug assessment is based on preliminary data and subsequently involves early data evaluation via a health technology assessment (HTA) that can significantly influence reimbursement and price negotiations. , Unlike the United States, which regulates access to the market without harmonizing reimbursement at the federal level (this being the responsibility of private insurance companies or public programs such as Medicaid and the Veterans Health Administration), the French healthcare system, through the French National Authority for Health (HAS), also regulates the coverage of healthcare products. The regulation of marketing authorization (MA) is based on an absolute benefit/risk analysis, whereas the regulation of reimbursement is based on a relative benefit/risk analysis, which is therefore more stringent. While there are considerable variations across countries, , reimbursement and price negotiations can delay access to drugs because of disagreements between MA holders and national health authorities. A study on time to availability of drugs in 28 European countries between 2015 and 2018 revealed that France was ranked 21 out of 28, with a mean of 566 days before patients could access drugs. ,

To overcome this delay, France has implemented a specific Early Access Program (EAP) since 1994, allowing patients with unmet medical needs to be treated with an innovative drug before MA. , , The Temporary Authorization for Use (ATU) is granted by the French National Agency for Safety of Medicines and Health Products (ANSM) if the assessment of preliminary data presents a positive benefit/risk balance and if the patient has no therapeutic alternatives among the existing therapies/clinical trials. Two types of ATU are available: the nominative ATU, granted for a specific patient at his or her clinician's request; and the cohort ATU, granted for a group of patients who meet specific criteria, for a specific indication, at the pharmaceutical company's request. Requests for cohort ATUs are frequently submitted after the nominative ATU. If the cohort ATU is granted, the pharmaceutical company commits to gathering efficacy and safety data from patients treated in this program in order to obtain “real life” data. Then, when an ATU drug obtains an MA, its process is closed, and the drug enters a “post‐ATU” phase to ensure continuity of coverage by the Social Security. This latter stage lasts as long as it takes for the pharmaceutical company and the French Ministry of Health to negotiate and settle on a price and reimbursement, which can be a significant amount of time. The ANSM recently published data showing that, thanks to the ATU program, between 2007 and 2019, 25 out of the 36 oncology drugs (69.4%) that obtained MA had early access with an average of 203 days before Food and Drug Administration (FDA) approval, 428 days before European Medicines Agency (EMA) approval, and 566 days before national public reimbursement.

The HAS is an independent public body whose tasks include HTAs, carried out by its Transparency Committee (TC). If an MA holder wants its drug to be reimbursed and available in France, it has to submit an application to the TC, which assesses the actual clinical benefit (ACB) and the clinical added value (CAV) of drugs. The French Ministry of Health, the Social Security and the Economic Committee for Health Products (CEPS) then take the TC's recommendations into consideration in their drug assessment, reimbursement, and price decisions. ACB and CAV are evaluated for each indication of a drug. ACB is an assessment of the actual benefit of the drug and whether its cost is worth being covered by the Social Security. Reimbursement rates are based on disease severity and the impact on morbidity and mortality, the drug's clinical efficacy and safety, its place in the therapeutic guidelines and its significance for public health (Supplementary Table 3). CAV is an assessment of the clinical benefit compared to therapeutic alternatives and has a role in pricing decisions. Its five levels (I to V) are defined according to the quality and methodology of the drug application, drug efficacy and safety, compared to those already on the market. A score of I, II, III or IV indicates a significant added therapeutic benefit (ATB) and a possibility for the drug to be registered on the “costly drugs” list; and premium prices are guaranteed for a score of I to III (Supplementary Table 4). The HAS' assessments are determined by a vote of the TC regarding the criteria mentioned above and are definitive, unless a reassessment is requested by the MA holder.

Leading professional organization for medical oncology such as the European Society for Medical Oncology (ESMO) and the American Society of Clinical Oncology (ASCO) have also independently published a validated and reproducible tool to assess the clinical benefit of anticancer drugs: the European Society for Medical Oncology—Magnitude of Clinical Benefit Scale (ESMO‐MCBS) and the American Society of Clinical Oncology—Value Framework (ASCO‐VF). The purpose of the ESMO‐MCBS is to assess the magnitude of the clinical benefits of anticancer interventions and to allow health authorities to prioritize drugs with great clinical added value so that they may be rapidly endorsed, whereas the goal of ASCO‐VF is to provide a tool to guide clinicians and their patients toward the choice of high‐value treatment. Both frameworks calculate a preliminary score based on survival data, which is then adjusted according to toxicities, quality of life (QoL) and bonus points. ESMO‐MCBS and ASCO‐VF both consider overall survival (OS), progression‐free survival (PFS) and response rate (RR) to obtain the preliminary clinical benefit score, but ASCO‐VF applies a prioritization approach that first uses the Hazard Ratio (HR) of OS data. If not available, the difference between median OS reported for both study arms is examined; if not available, HR of PFS is used, and so on. ESMO‐MCBS prioritizes the primary endpoint of the clinical trial and considers both HR and gains in median survival. The scale depends on the disease setting: in the case of a curative setting, the ESMO‐MCBS is rated from C to A with B and A representing a substantial magnitude of clinical benefit. In a noncurative setting, the ESMO‐MCBS is rated from 1 to 5 with 4‐5 representing a high level of proven clinical benefit. The maximum score of 5 can only be obtained if the primary endpoint is OS. On the other hand, ASCO‐VF uses continuous scoring from 0 to 130. ASCO has not determined a threshold for meaningful ATB, but Cherny et al calculated a threshold of 45 through an evaluation of receiver operating characteristic (ROC) curves based on 102 clinical studies. In contrast to HTA assessments, the ASCO and ESMO tools allow for repeated assessments as the drug is being investigated, with updated data and prolonged follow‐up.

The costs of novel anticancer drugs have risen this past decade, but the correlation between costs and clinical benefits is unclear. Furthermore, in EAPs, the data analyzed usually come from clinical trials set up for the MA application. They are frequently nonmature, with results obtained from analyses of preliminary studies, and often lack the follow‐up required for a reliable positive benefit/risk balance. Besides, the urgent need to improve and to speed up access to new treatments for patients with fatal diseases has alleviated regulatory and methodological approaches. Therefore, the ATB of drugs granted EAPs is yet to be evaluated.

We previously reported a synthesis of the ATB of drugs granted ATUs according to ESMO‐MCBS and HAS assessments, which indicated that ATU was a successful program in giving early drug access to patients in medical need. In continuity with this previous work, the objective of the present study was to assess the ATB of drugs granted the ATU program in oncology over the past 11 years, according to the French HTA criteria (ACB and CAV), the ESMO‐MCBS and the ASCO‐VF. The correlations between ATB, survival and cost were also evaluated.

MATERIAL AND METHODS

Drugs granted ATUs in solid oncology between January 2009 and December 2019, the number of patients treated with an ATU, and the ATU duration were collected from the ANSM's internal software. MA dates were collected from the EMA website and post‐ATU program lengths were identified on the French Ministry of Health website.

Two kinds of prices are described in our study: Pharmaceutical companies' freely set ATU prices, and afterwards the prices negotiated with the CEPS for market access. ATU prices were collected from the national website legifrance.fr and the CEPS kindly provided unpublished data. Monthly costs of reimbursed drugs were calculated based on the updated prices (cut‐off: 31 December 2019) published in the Official Journal of the French Republic and were calculated for a 28‐day cycle.

Two independent reviewers calculated the ESMO‐MCBS (version 1.1) and ASCO‐VF (version 2) scores for each ATU drug and each indication (Supplementary Table 5). Interrater reliability was assessed using the intraclass correlation coefficient (ICC) and 95% confidence intervals. Poor reliability was indicated by values of less than 0.5, moderate between 0.5 and 0.75, good between 0.75 and 0.9, and excellent reliability over 0.9. A two‐way mixed‐effects model with multiple raters, assessing absolute agreement, was used. Unlike the ESMO and ASCO scores, the CAV score is established by a vote of the TC and published on the HAS website. The CAV scores were therefore retrieved directly from the HAS website and could not be recalculated.

A significant ATB corresponded to an ESMO‐MCBS score of 4‐5 or A, an ASCO‐VF score of 45 and a CAV score of I to IV.

Each ATB score was compared to OS, PFS and prices by correlation analysis. Next, the three ATB scores were compared to each other. Finally, the correlation between prices and survival endpoints was assessed. All these correlation analyses were done with Pearson correlation tests, as all these variables are quantitative.

Survival benefit (OS and PFS) could obviously not be calculated for ATU drugs granted based on data from a single‐arm trial, so no comparison with ATB could be done.

Statistical analyses were calculated using R (version 4.0.2) and SPSS23.0 (IBM, Paris France).

RESULTS

Drugs evaluated in our study

Thirty‐nine indications for 35 drugs were granted an ATU, all obtained an MA (Table 1). The HAS attributed a CAV score for the 39 indications, all ESMO‐MCBS scores were calculated, while ASCO‐VF scores were established only in 33 indications, since the latter scale does not allow single‐arm study evaluation (six drugs obtained an ATU and an MA based on single‐arm trials while the other studies were randomized trials with superiority design). Monthly costs were available for 31 drugs out of 35; price negotiations were still ongoing between the French health authorities and the pharmaceutical company for four drugs at the time of the study. Between 2009 and 2019, ATUs were mainly used to treat patients with lung cancer (25.6%), melanoma (25.6%) or breast cancer (10.3%) and they were mostly tyrosine kinase inhibitors (TKI) (41.0%) or immune‐checkpoint inhibitors (23.1%), and to a lesser extent, PARP inhibitors (10.3%). The ATU program has enabled 22 554 patients with cancer to be treated with innovative drugs prior to MA over the last 11 years. The median duration for nominative ATUs was 6 months [0‐47 months] and 4 months for cohort ATUs [0‐11 months]. The median post‐ATU phase length was 15 months [4‐41 months], which is notably longer than the 180‐day statutory deadline for HTA assessment and a CEPS decision. As the three phases of this program (nominative, cohort and post‐ATU) usually follow one another, the median sum of these phases indicates that patients have benefited from this specific authorization for a median of 27.7 months [8‐66 months].

TABLE 1

List of drugs granted with ATU in solid oncology between 2009 and 2019 (more detailed indications in Supplementary Table 2)

Drug	Indication
Abiraterone	Metastatic castration‐resistant prostate cancer (mCRPC) previously treated
Alectinib	Anaplastic lymphoma kinase (ALK)‐positive advanced nonsmall cell lung cancer (NSCLC) previously treated
Apalutamide	Nonmetastatic castration‐resistant prostate cancer (nmCRPC)
Atezolizumab	Extensive‐stage small cell lung cancer (in combination with carboplatin and etoposide) in first line
Avelumab	Metastatic Merkel cell carcinoma previously treated
Binimetinib	Metastatic melanoma with a BRAF V600 mutation (in combination with encorafenib)
Brigatinib	ALK‐positive NSCLC previously treated
Cabozantinib	Advanced renal cell carcinoma previously treated
Cemiplimab	Metastatic cutaneous squamous cell carcinoma (mCSCC or laCSCC) previously treated
Ceritinib	ALK‐positive NSCLC previously treated
Cobimetinib	Metastatic melanoma with a BRAF V600 mutation (in combination with vemurafenib)
Crizotinib	ALK‐positive NSCLC previously treated
Durvalumab	Maintenance treatment of locally advanced, unresectable NSCLC
Encorafenib	Metastatic melanoma with a BRAF V600 mutation (in combination with binimetinib)
Enzalutamide	mCRPC previously treated
Everolimus	Advanced renal cell carcinoma (RCC) previously treated
Ipilimumab	Metastatic melanoma previously treated
Lapatinib	Metastatic breast cancer previously treated (in combination with capecitabine)
Larotrectinib	Solid metastatic tumors that display a neurotrophic tyrosine receptor kinase (NTRK) gene fusion with no satisfactory treatment options
Lenvatinib	Metastatic, differentiated thyroid carcinoma (DTC), refractory to radioactive iodine and previously treated
Lorlatinib	ALK‐positive NSCLC previously treated
Niraparib	Maintenance treatment of relapsed high grade serous epithelial ovarian, fallopian tube or primary peritoneal cancer
Nivolumab	Stage III or IV melanoma
Nivolumab	Squamous stage IIIb or IV NSCLC previously treated
Nivolumab	Nonsquamous stage IIIb or IV NSCLC previously treated
Olaparib	Maintenance treatment of advanced high‐grade epithelial ovarian, fallopian tube or primary peritoneal cancer with BRCA1/2 mutation
Olaparib	Maintenance treatment of relapsed high‐grade epithelial ovarian, fallopian tube, or primary peritoneal cancer with BRCA1/2 mutation
Osimertinib	EGFR mutation positive metastatic T790M NSCLC previously treated
Palbociclib	Hormone receptor (HR)‐positive, human epidermal growth factor receptor 2 (HER2)‐negative metastatic breast cancer previously treated (in combination with fulvestrant)
Pembrolizumab	Stage III or IV melanoma
Ramucirumab	Advanced gastric cancer or gastro‐esophageal junction adenocarcinoma Cyramza previously treated (in combination with paclitaxel)
Regorafenib	Metastatic colorectal cancer (CRC) previously treated
Regorafenib	Metastatic gastrointestinal stromal tumors (GIST) previously treated
Talazoparib	Metastatic breast cancer previously treated and with BRCA1/2‐mutations
Temsirolimus	Advanced RCC
Trastuzumab emtansine	Adjuvant treatment for early HER2+ breast cancer
Trifluiridine‐Tipiracil	Metastatic CRC previously treated
Vemurafenib	Metastatic melanoma with a BRAF V600 mutation
Vismodegib	Metastatic basal cell carcinoma or locally advanced basal cell carcinoma

Abbreviations: mCRPC, metastatic castration‐resistant prostate cancer; nmCRPC, nonmetastatic castration‐resistant prostate cancer; NSCLC, nonsquamous cell lung carcinoma.

The median OS difference between experimental arm and control arm calculated in our study was 3.9 months [−2 to 26.1 months] (primary and secondary endpoints together) for drugs granted an ATU. The median HR for OS was 0.73 [0.50‐1.02]. The median PFS difference was 3.85 months [−2 to 25.8 months] and the median HR for PFS was 0.495 [0.15‐0.92] (Figure 1).

FIGURE 1

ATB scores (ASCO‐VF, ESMO‐MCBS and CAV), survival benefit and monthly price of drugs granted with ATU in oncology between 2009 and 2019. Histogram distribution represents HR of OS on the left, HR of PFS in the middle and monthly costs on the right. Color distribution follows the same pattern for each ATB score: red for low ATB to dark green for high ATB. ASCO‐VF scores are presented on the left, ESMO‐MCBS in the middle and CAV on the right. The ASCO scale ranges from 0 to 100 as there were no indications with a score above 100. adj BC, adjuvant breast cancer; ASCO‐VF, American Society of Clinical Oncology—Value Framework; ATB, added therapeutic benefit; ATU, temporary authorization for use; C, cancer; CAV, clinical added value; CC, cell carcinoma; Cut Squamous CC, cutaneous squamous cell carcinoma; ESMO‐MCBS, European Society for Medical Oncology—Magnitude of Clinical Benefit Scale; GIST, gastrointestinal stromal tumor; HR, hazard ratio; mBC, metastatic breast cancer; mCRC, metastatic colorectal cancer; mCRPC, metastatic castration‐resistant prostate cancer; mM, metastatic melanoma; nmCRPC, nonmetastatic castration‐resistant prostate cancer; NSCLC, nonsquamous cell lung carcinoma; nsq, nonsquamous; OS, overall survival; ovarian C maint, ovarian cancer maintenance; PFS, progression‐free survival [Color figure can be viewed at wileyonlinelibrary.com]

ATB assessed by French HTA criteria, ESMO‐MCBS and ASCO‐VF

Of the 39 indications assessed in our study, the HAS gave 30 (76.9%) substantial ACBs, 10.3% of the indications received a moderate ACB, and 12.8% a low ACB. No indication received an insufficient ACB and all drugs were registered to be reimbursed in France for these indications. Concerning ATBs with the HAS criteria: none of the 39 indications were given the best level, CAV I. Twenty‐eight (71.8%) were evaluated as bringing an improvement in actual benefit compared to existing drugs (level II to IV): 1 (2.6%) level II, 12 (30.8%) level III, 15/39 (38.5%) level IV (Table 2). The evaluation concluded upon no improvement in actual benefit in 11/39 (28.2%), which corresponds to a level V CAV; 5/11 (45.5%) of these indications were assessed in a single‐arm trial for ATU and MA application (Supplementary Table 1). Two drugs and their indications have not met the criteria for hospital use, as they have not been registered on the “costly drugs list”: ramucirumab for the treatment of patients with advanced gastric cancer and cemiplimab for the treatment of patients with cutaneous squamous cell carcinoma.

TABLE 2

Distribution of actual clinical benefit (ACB), clinical added value (CAV), European Society for Medical Oncology—Magnitude of Clinical Benefit Scale (ESMO‐MCBS) and American Society of Clinical Oncology—Value Framework (ASCO‐VF) scores

ACB (n = 39)	Substantial	30 (76.9%)
	Moderate	4 (10.3%)
	Low	5 (12.8%)
CAV (n = 39)	I	0
	II	1 (2.6%)
	III	12 (30.8%)
	IV	15 (38.5%)
	V	11 (28.2%)
ESMO‐MCBS (n = 38)	5	4 (10.5%)
	4	20 (52.6%)
	3	9 (23.7%)
	2	2 (5.3%)
	1	3 (7.9%)
ASCO‐VF (n = 33)	80‐100	3 (9.1%)
	60‐80	6 (18.2%)
	40‐60	16 (48.5%)
	20‐40	7 (21.2%)
	0–20	1 (3.0%)

Note: For the ESMO‐MCBS score, 1/39 indication was for adjuvant therapy, which was not covered by the 1 to 5 scale. For the ASCO‐VF score, 6/39 indications were not applicable because they were based on single‐arm trials.

Concerning ESMO‐MCBS, the ICC for both reviewers was 1, meaning that both reviewers found exactly the same result for all 39 ESMO‐scores. Only one drug and its indication were assessed with the curative framework in our study, and obtained the highest score, A: trastuzumab emtansine for the adjuvant treatment of patients with HER2‐positive breast cancer after neoadjuvant taxane‐based and HER2‐targeted therapy. Of the 38 other indications, 24 obtained an ESMO score of 4 or 5 (63.2%), representing a meaningful clinical benefit, 9 (23.7%) had a score of 3, 2 (5.3%) had a score of 2, and 3 (7.9%) had a score of 1 (Table 2, Figure 1). Among the 6 ATUs granted based on a single‐arm trial, 1 had an ESMO score of 4, 4 had an ESMO score of 3 and 1 had an ESMO score of 1, the lowest grade. Twenty‐three out of 38 (60.5%) of the calculated ESMO scores were upgraded due to favorable safety and QoL data. Only cemiplimab for the treatment of patients with cutaneous squamous carcinoma was downgraded by a point, because 30% more adverse events impacting QoL were reported in the experimental arm (Figure 2B).

FIGURE 2

(A) ESMO‐MCBS and (B) ASCO‐VF score upgraded with QoL and safety bonus points for drugs and indications granted with an ATU in solid oncology between 2009 and 2019. adj BC, adjuvant breast cancer; ASCO‐VF, American Society of Clinical Oncology—Value Framework; ATU, temporary authorization for use; C, cancer; CC, cell carcinoma; ESMO‐MCBS, European Society for Medical Oncology—Magnitude of Clinical Benefit Scale; GIST, gastrointestinal stromal tumor; QoL, quality of life; mCRC, metastatic colorectal cancer; mCRPC, metastatic castration‐resistant prostate cancer; mM, metastatic melanoma; nmCRPC, nonmetastatic castration‐resistant prostate cancer; NSCLC, nonsquamous cell lung carcinoma; ovarian C maint, ovarian cancer maintenance; SCLC, small cell lung cancer [Color figure can be viewed at wileyonlinelibrary.com]

Regarding ASCO‐VF, the ICC for both reviewers was ICC = 0.93 IC95% [0.88‐0.96]. Excellent reliability was found using the mean value of two raters on a two‐way mixed‐effects model assessing absolute consistency. Three out of 33 (9.1%) indications obtained a score between 80 and 130, 6 (18.2%) had a score between 60 and 80, 16 (48.5%) had a score between 40 and 60, 7 (21.2%) had a score between 20 and 40 and 1 (3.0%) had a score between 0 and 20 (Table 2). The only drug evaluated with the curative framework (trastuzumab emtansine for the adjuvant treatment of patients with HER2‐positive breast cancer after neoadjuvant taxane‐based and HER2‐targeted therapy) had a score of 43.4. The median ASCO score was 50.67 and 19/33 (57.6%) had a meaningful ATB according to the threshold of 45 defined by Cherny et al (Table 2, Figure 1). Twenty‐four out of 33 (72.7%) of the calculated ASCO scores were upgraded with 10 to 40 bonus points (Figure 2A). None were downgraded.

Correlation between price of the drugs in France, ATB and survival gain

There was no statistically significant correlation between OS absolute gain or HR and CAV level (Pearson correlation r = .232, P = .244 for absolute gain and r = .116, P = .551 for HR). There was no statistically significant correlation between PFS absolute gain or HR and CAV level (r = .127, P = .694 for absolute gain and r = − .027, P = .883 for HR). OS absolute gain was significantly higher in ESMO‐MCBS scores 4 and 5 (r = .410, P = .034), whereas there was no statistically significant correlation for OS HR (r = −.230, P = .239), for PFS absolute gain (r = .198, P = .536) and for HR (r = −.057, P = .756). There were no statistically significant correlations between ASCO‐VF scores and OS absolute gain (r = .161, P = .433), OS HR (r = .003, P = .987), PFS absolute gain (r = .434, P = .182) and PFS HR (r = −.253, P = .155).

ESMO‐MCBS was correlated to CAV and ASCO‐VF (r = −.490, P = .002, and r = .689, P = .0001, respectively), whereas there was no statistically significant correlation between ASCO‐VF and CAV (Supplementary Figure 1).

Of the 35 drugs in our study, 31 (88.6%) had a price negotiated between pharmaceutical companies and CEPS. The median monthly cost calculated was €3755.42. No statistically significant correlations were found between monthly cost and OS absolute gain (r = .119, P = .572), OS HR (r = −.169, P = .398), PFS absolute gain (r = .184, P = .589) and PFS HR (r = .249, P = .176) (Figure 3).

FIGURE 3

Pearson correlation coefficient plots for HR of OS (A) and HR of PFS (B) vs monthly price. Blue dots represent significant HR and orange dots represent nonsignificant HR. HR, hazard ratio; OS, overall survival; PFS, progression‐free survival [Color figure can be viewed at wileyonlinelibrary.com]

All the correlation tests performed are detailed in Supplementary Figure 2 and Supplementary Table 6.

DISCUSSION

Of the 39 indications evaluated in our study, 76.9% had a substantial ACB. 71.8% demonstrated a significant ATB according to the French HTA (CAV level I to IV), 63.2% according to ESMO‐MCBS (score 4 to 5) and 57.6% according to ASCO‐VF (score above 45). As a comparison, a systematic evaluation of EMAs approval in oncology from 2009 to 2013 has shown that, of the 68 indications analyzed, only 51% presented a significant improvement in OS or QoL, with a median of 5.4 years of follow‐up. Thus, based on the HAS, ESMO and ASCO assessments, most of the indications and drugs concerned by an ATU provide a clinical benefit, meaning that this EAP has been beneficial. Additionally, this process has allowed the early use of new drugs on average 27.7 months (ATU and post‐ATU) before their access to the market with reimbursement. It should be noted that a score of I to IV was used in our study to define a meaningful ATB for CAV. However, the TC did not define meaningful ATB in its doctrine, so a score of I to III could also have been retained. Indeed, although a CAV IV corresponds to therapeutic progress, it does not provide the assurance of reimbursement, unlike a CAV III. If we consider the range of I to III for the definition of a relevant benefit, the rate drops to 33.4%, with the vast majority of indications having a score of IV and none having a score of I. This calls into question the relevance of this I‐V scale for the evaluation of cancer treatments.

Postponing treatment for patients, especially those with cancer, is a major ethics issue. Countries like France have developed an EAP to afford patients faster access to innovative therapies before market approval: fast track FDA, prime EMA, priority review FDA, accelerated approval FDA, breakthrough therapy FDA and conditional approval EMA. In addition, the United States implemented the federal “Right to Try” act into law on May 30th, 2018 to create an additional way, different from the FDA's expanded access, for patients to gain access to an investigational, out‐of‐study drug. However, the counterpart of EAP is that endpoints (PFS, OS) measuring the beneficial effects of those new drugs may be nonmature at the time of the decision on early access.

Our study highlights the fact that it is the price negotiation phase in particular that delays the availability of new drugs on the market: The mean post‐ATU phase length of 475 days calculated in our study is consistent with the study by Les Entreprises du médicament (LEEM) presenting a median time of 566 days between MA and drug availability between 2015 and 2018, and of 515 days in the IQVIA study. This long period of HTA assessment may be explained by the way the HAS evaluates innovative medicines, which may involve stricter criteria than ESMO's and ASCO's. For example, the HAS has only granted the lowest level, CAV level V, for indications and drugs assessed in phase II noncomparative clinical trials because of the methodological approach (except for one drug which was granted a level IV CAV). This is debatable in studies evaluating oral drugs if the comparator is an intravenous drug. The arrival of immunotherapy associated with new types of side effects, with specific management guidelines, may raise questions about double‐blind designs because of the risk caused to patients. ASCO‐VF cannot be used to assess a drug's ATB in a study with no comparator, but ESMO‐MCBS v1.1 has implemented a new framework to do so for single‐arm studies on “orphan diseases” and for diseases with “high unmet needs.” This has allowed some therapies to obtain a better ESMO score compared to CAV. Nevertheless, it is important to bear in mind that these scales were developed with different goals: ASCO‐VF's aim is to provide a tool to help and guide clinicians and patients in their choice of the most suitable treatment for the patient (efficacy vs toxicity vs price). ESMO‐MCBS proposes to facilitate the identification of treatments with high clinical benefit through public health policy, so that they can obtain an MA and be available on the market quickly. These two scales were created with different methodologies and perspectives to evaluate the ATB of a drug based on one study, usually the pivotal clinical trial. Conversely, the HAS assessment has a different purpose, as it plays a role in reimbursement and pricing decisions, and it takes into account data from several clinical trials submitted by the pharmaceutical company. Its recommendations are crucial to decisions about which drug is worth being covered by the public sector. However, HAS' assessment only covers data provided at submission unless a reevaluation is requested, and these can be incomplete or lacking in long‐term follow‐up. QoL data and improved symptoms are not often considered by the HAS because of short follow‐up data. In contrast, ESMO and ASCO scores can be assessed whenever updated data are presented, and our study shows that 60% to 75% of ESMO and ASCO scores were upgraded thanks to new safety data. This may be one explanation for the absence of correlation or only moderate correlation between ESMO, ASCO and HAS assessments.

ESMO and ASCO have undertaken a cooperative project to assess the concordance between the two scales (ESMO‐MCBS V1.1 and ASCO‐VF v2) and have estimated, for 102 pairs of scores, a Spearman's rank correlation coefficient of .68, which converges with our result of .689. Other studies have assessed this correlation and found it to be weaker (r = .40, P = .06, with ASCO‐VF v2 and ESMO‐MCBS v1) based on 44 clinical trials, and even lower (r = .17) based on 83 clinical trials (with ASCO‐VF v2 and ESMO‐MCBS v1). ESMO‐MCBS v1.1 slightly modifies scores calculated with the previous version, as only 12 out of 118 clinical trial scorings were modified. Discrepancies between these studies might find their explanation in a different analysis of published data by different reviewers and in a heterogeneous interpretation of the ESMO and ASCO tools. The application of these frameworks is operator‐dependent and has a degree of subjectivity. Cheng et al showed a concordance among raters' scores of 18% for ASCO‐VF V2 and 69.4% for ESMO‐MCBS V1.1, although our study shows a 100% concordance for ESMO‐MCBS scores, and an ICC of 0.93 for ASCO‐VF scores. Fewer publications have evaluated the correlation between HAS criteria and ESMO and ASCO scales. A study on 17 clinical trials calculated a Spearman's rank coefficient of 0.34 (P = .181) between CAV and ESMO‐MCBS v1, and of 0.27 (P = .286) between CAV and ASCO‐VF v2. Additionally, a statistically significant and moderate correlation (r = .51, P = .035) has been shown between ESMO‐MCBS v1 and ASCO‐VF v2. A study on 59 clinical trials to compare HAS and ESMO clinical benefit assessment has shown that only 40% are correlated, with HAS being the more stringent. A recent study on 36 drugs and 68 indications has shown a weak correlation between CAV and ESMO‐MCBS (r = .28), between CAV and monthly costs (|ρ| = .35, P = .004) and between ESMO‐MCBS and costs (|ρ| = .33, P = .005), in line with our study (r = .43 for ESMO‐MCBS and prices).

According to our results, ATB tools are poorly correlated to price and survival outcomes. Furthermore, several discrepancies were found between the different tools in their functioning and in the results obtained, which raises questions about the relevance of these tools and the need for harmonization of drug assessments. It is of note that ESMO‐MCBS was the only tool correlated to OS benefit. Strikingly, our study shows that OS and PFS (survival advantage and HR) are not correlated with drug costs. Patients' outcomes have currently improved, but the drugs' efficacy does not translate to cost effectiveness so well.

Our study has several limitations. In contrast to the ASCO and ESMO scores, the HAS score could not be calculated prospectively. As a result, the data used to establish the ASCO and ESMO scores were more current and complete, which may have biased the results of our study in favor of these tools. Second, nominative, cohort and MA indications can differ slightly and result in discrepancies in the number of patients granted ATUs, or in the length of ATUs. Finally, the monthly costs calculated only present the costs of the drug. Associated costs like hospital expenses, transportation costs, consulting fees or the financial impact on the patient's life and that of his or her entourage have not been considered.

CONCLUSION

The ATU program has been a successful gamble since all of the indications obtained FDA and EMA approval and a positive reimbursement decision based on ATB analysis. The second main objective of this research was to compare three tools to evaluate the ATB of a drug receiving EAP, although they differed significantly in their conception and methodology. In conclusion, it remains challenging to make a clear statement about the best way to measure a substantial benefit regarding these three tools and international harmonization of clinical benefit measurement is recommended. This could probably be achieved in Europe with the EuNetHTA program, which may help to create a consensus among scientific societies for an optimal evaluation of the magnitude benefit in the context of granting MA. In addition, the increase in new anticancer drugs with rising costs is of great concern, since a high price is not correlated with outcomes in terms of clinical benefit such as OS and PFS. A reform of the French healthcare system is being implemented, with the aim of better correlating decisions on reimbursement with the actual clinical data obtained from the pivotal clinical trials.

AUTHOR CONTRIBUTIONS

Conception and design: Fiona Y.‐V. Pham, Nicolas Albin. Provision of study material or patients: Adrien Monard, Ghania Kerouani‐Lafaye, Florence Turcry, Liora Brunel, Françoise Grudé, Isabelle Yoldjian, Isabelle Sainte‐Marie, Lotfi Boudali, Nicolas Albin. Collection and assembly of data: Fiona Y.‐V. Pham, Emmanuelle Jacquet, Adrien Monard, Jean‐Yves Blay, Nicolas Albin. Data analysis and interpretation: Fiona Y.‐V. Pham, Emmanuelle Jacquet, Amina Taleb, Adrien Monard, Jean‐Yves Blay, Nicolas Albin. Manuscript writing: All authors. Final approval of manuscript: All authors. Accountable for all aspects of the work: All authors. The work reported in the paper has been performed by the authors, unless clearly specified in the text.

CONFLICT OF INTEREST

Jean‐Yves Blay: Faculty member of the ESMO. Research support and honoraria from GSK, Novartis, Bayer, Roche, MSD, BMS, Deciphera, Pharmamune. All remaining authors have declared no conflicts of interest.

Supplementary Figure 1 Comparison between reversed CAV, ESMO‐MCBS score and ASCO‐VF score for drugs granted with ATU in solid oncology between 2009 and 2019. Reversed CAV has been used instead of CAV to ease comparison with other scores, as the highest CAV level is 1, as opposed to ESMO‐MCBS and ACO‐VF

Supplementary Figure 2 Pearson correlation coefficient plots for CAV vs HR of OS (A), CAV vs HR of PFS (B), ESMO‐MCBS vs HR of OS (C), ESMO‐MCBS vs HR of PFS (D), ASCO‐VF vs HR of OS (E), ASCO‐VS vs HR of PFS (F)

Supplementary Table 1 List of indications and drugs assessed through single‐arm trial: treatment of rare cancers except for nonsmall cell lung cancer (NSCLC) ALK+ but indicated for patients previously treated and/or not considered candidates for all available therapy. To assess ATU's relevance, ACB and CAV were collected from HAS' website only for indications of drugs granted with an ATU in solid tumor. ACB, actual clinical benefit; CAV, clinical added value; ESMO‐MCBS, European Society for Medical Oncology—Magnitude of Clinical Benefit Scale

Supplementary Table 2 List of drugs granted with ATU in solid oncology between 2009 and 2019

Supplementary Table 3 Actual clinical benefit (ACB) of drugs and reimbursement rate

Supplementary Table 4 Clinical added value and drug prices

Supplementary Table 5 List of clinical trials used for the calculation of ESMO‐MCBS and ASCO‐VF scores

Supplementary Table 6 Correlation analysis between OS (overall survival) gain, OS hazard ratio (HR), progression‐free survival (PFS) gain, PFS HR, American Society of Clinical Oncology‐Value Framework (ASCO‐VF), added clinical benefit (ACB), clinical added value (CAV), drug costs and European Society for Medical Oncology—Magnitude of Clinical Benefit Scale (ESMO‐MCBS)

Click here for additional data file.