Methodological issues in the choice among different drugs approved for the same therapeutic indication: a position paper by the Italian Association of Medical Oncology (AIOM).

In oncology, as in other clinical fields, different treatments are often approved for the same therapeutic indication. In many cases, no direct comparisons are available to inform the choice in clinical practice. In 2015, the Italian Association of Medical Oncology (AIOM) instructed a working group, including both clinicians and methodologists, to discuss the issue of the best choice among different treatments available for the same indication. The working group discussed 3 different scenarios: (1) biosimilar drugs; (2) different drugs with same mechanism of action; (3) different drugs with different mechanism of action. For each scenario, methodological issues were discussed, along with the priority for investment of resources in the conduct of clinical trials testing direct comparison. As for biosimilar drugs, the panel recommended that, following comparability exercise and approval by regulatory agencies, they should be widely used, considered that their use allows financial savings. As for different drugs (with either the same or a different mechanism of action), the panel agreed that indirect comparisons and network meta-analyses are associated with relevant risk of bias and imprecision, and direct comparisons should be encouraged. The priority of these direct comparisons should be higher when the potential differences in efficacy and/or toxicity are clinically relevant. The choice of the study design (superiority vs non-inferiority) depends on the toxicity profiles and also on the presumed difference in efficacy. Scientific societies should put pressure on public bodies to identify all the administrative and financial mechanisms useful to facilitate the conduct of trials testing direct comparisons, when needed. Decision about therapeutic equivalence can have important consequences on innovation: the availability of drugs characterised by the same effectiveness, but at a lower cost, could enable non-negligible savings of economic resources that could be used to guarantee access to innovative, high-cost drugs.

Introduction

In recent years, due to economic difficulties (not limited to a single country but substantially global, although with different severity among countries), there has been a rapidly growing awareness of the limitation of financial resources. As a consequence, the concept of sustainability has gained a crucial role in the scientific debate about the efficacy of anticancer treatments.1 2

Testing a new treatment in a clinical trial is primarily focused on demonstrating a favourable ratio between risks and benefits associated with its administration. However, a great deal of methodological rigour is needed in the conduct and interpretation of the clinical trials, in order to derive a more complete picture of the value of the new treatment for the patient and the society.3–5 In fact, the introduction of a treatment into clinical practice involves the determination of a price, and adopting the new treatment represents a direct cost for the community.

Many opinions have recently invited to ‘raise the bar’ in the assessment of the efficacy of experimental treatments in clinical trials.3–5 This invitation, and the consequent decisions, should fall on the central regulatory authority level, and not on the individual peripheral administrative centres.6 7 In fact, if economic choices about drug reimbursement and availability in clinical practice are taken at the highest decisional level, unacceptable disparities could be avoided, and the single physician, when discussing with an individual patient, could avoid to put financial considerations over and above clinical arguments. Of course, this does not mean that doctors should not be aware of the cost of the drugs: the economic implications of treatment decisions should be always taken into account. Indeed, due to the relevance of these topics, decision-making should involve the contribution of all the stakeholders, the doctors and even the patients themselves. The involvement of patients in the debate should be greatly encouraged by the scientific societies, inasmuch as it would impose on all involved parties the need to address the critical issues related to the balance between treatment advantages and costs. This would probably create a stronger alliance between healthcare professionals and patients, and would—in any case—increase awareness among all parties.

In this complex debate, the concept of therapeutic innovation should be clearly distinguished from the issues related to therapeutic equivalence. In the case of innovation, when a new therapeutic strategy has demonstrated a greater efficacy compared with the previous standard, the debate should centre on both the benefit–risk ratio and on the size of the increase in efficacy.8–10 A shared and reproducible procedure should be identified to define how much the community is willing to pay for benefits of various types and sizes. In any case, very small benefits may be negligible, irrespective of the cost. However, shared and reproducible procedures should be also identified to define this minimal level of efficacy, below which the incremental benefit could be considered as clinically not relevant. On the other hand, in the case of therapeutic equivalence, the scientific debate is centred on the choice between therapeutic options of similar efficacy, for the same clinical indication. If the tolerability profile roughly differs between these options, toxicity will obviously play a determining role in the choice. However, when the tolerability profile is also similar, the cost may reasonably play a role in the choice of one treatment over another. Of course, the equivalence margin of efficacy must be sufficiently small so that the cost-based choice can be scientifically and ethically transparent, and shareable with the patients. Of note, decisions in the field of therapeutic equivalence can have important consequences also on innovation: the availability of drugs characterised by similar efficacy, but at lower cost (typically, the case of biosimilar drugs), could enable a non-negligible saving of economic resources that could be used to guarantee access to innovative, high-cost drugs.

Therefore, how to help the choice among different treatments available for the same therapeutic indication? In 2015, the Italian Association of Medical Oncology (Associazione Italiana di Oncologia Medica, AIOM) appointed a dedicated working group, including both clinicians and methodologists, with the aim of discussing the clinical and methodological aspects of this topic. The panel produced a document, approved by the AIOM National Directory and freely available on the AIOM website.11 This paper summarises the conclusions of the working group.

Methods

The AIOM working group on therapeutic equivalence was coordinated by a member of AIOM National Board (MDM), and composed of three clinicians, with specific expertise in different solid tumours (FM, MT and EV) and three methodologists, with specific expertise in clinical cancer research (PB, FP and VT). The group worked by physical and virtual meetings, between January and April 2015. The final document, endorsed by the AIOM National Board, was published on the AIOM website in April 2015. In September 2016, for the present publication, the original material was adapted, updated as appropriate and revised by all the authors.

As a preliminary step, four potential scenarios of therapeutic equivalence were identified, as reported in table 1. Of the scenarios outlined in table 1, the AIOM working group did not address scenario 1 (ie, the issue of generic drugs). Coherently with issues discussed by the working group, this paper is structured to address the remaining three scenarios, that is, the issue of biosimilar drugs (table 1, scenario 2) and the issue of drugs with a different active principle but approved for the same therapeutic indication (table 1, scenario 3: drugs with the same mechanism of action; scenario 4: drugs with different mechanisms of action).

Table 1

Different scenarios of therapeutic equivalence

	Therapeutic indication	Mechanism of action	Active principle	Example
Scenario 1	Same	Same	Same	Generic imatinib vs brand
Scenario 2	Same	Same	Different, biosimilar	Biosimilar filgrastim vs originator; biosimilar trastuzumab vs originator
Scenario 3	Same	Same	Different	Erlotinib vs gefitinib vs afatinib as first-line treatment for EGFR-mutation positive advanced NSCLC
Scenario 4	Same	Different	Different	Axitinib vs everolimus as second-line treatment for clear-cell renal cancer

EGFR, epidermal growth factor receptor; NSCLC, non-small cell lung cancer.

Same therapeutic indication, same mechanism of action, biosimilar drugs

A biosimilar drug is a ‘copy’ version of an already authorised drug (usually called ‘originator’) with similar biological activity, physicochemical characteristics, efficacy and safety.12 Not only in oncology, but in many clinical fields, the development of biosimilar drugs has been considered an opportunity in terms of resource savings.13 14 The first setting where biosimilar drugs have been available in cancer clinical practice is the ‘supportive care’ for patients undergoing chemotherapy (ie, granulocyte colony-stimulating factors,15 erythropoiesis-stimulating agents).16 However, as of 2016, biosimilar drugs are soon awaited for several anticancer drugs (monoclonal antibodies) that are currently indicated in the treatment of patients with advanced disease (eg, trastuzumab, bevacizumab, rituximab) or as adjuvant treatment (eg, trastuzumab in the adjuvant treatment of early breast cancer).17

The introduction of a biosimilar drug into clinical practice is based on the successful outcome of an examination conducted by the regulatory agencies, sufficient to permit its use for the registered indications. In detail, the so-called ‘comparability exercise’ is an experimental procedure, requested for approval by regulatory agencies.12 In the comparability exercise, the biosimilar drug is compared with the originator drug, at a physicochemical level, a preclinical level and a clinical level (usually with the conduction of randomised trials). Clinical trials are usually designed with a study population and clinical end points which are considered to be the most sensitive at highlighting any differences there may exist between the originator and the biosimilar drug. Of note, the aim of the comparability exercise is not the demonstration of the efficacy in itself of the biosimilar drug; rather, its aim is the demonstration of the comparability of the biosimilar, with respect to the originator, in terms of quality, efficacy and safety, given that the efficacy of the drug has been already demonstrated by the originator. Incidentally, it must be said that the comparability exercise is also requested to the originator, even after marketing, in order to authorise subsequent ‘versions’ of the drug, should the production process undergo changes.

The AIOM working group agreed that the positive outcome of the comparability exercise should imply confidence in the substantial therapeutic equivalence between the biosimilar drug and the originator drug, and the subsequent application of the biosimilar drug, to allow cost-savings. Regulatory agencies underline that the comparability exercise is sufficient for the demonstration of equivalent activity, efficacy and toxicity with respect to the originator drug. Of course, it should be recognised that the ‘extrapolation’ from one indication to another (in terms of different stages, different types of disease or different association with other drugs) between the clinical trial performed in the comparability exercise and the use in clinical practice may be particularly critical in terms of acceptability by the scientific community.18 19

Should public bodies (regulatory authorities, scientific societies, cooperative groups) consider a priority to invest resources in direct comparisons (postmarketing) between biosimilars and originators?

The AIOM working group considered that, once the comparability exercise has led to the approval of a biosimilar drug for use in clinical practice, a wide use of biosimilars in clinical practice should be recommended. Previous experience with supportive drugs (the first category of biosimilar drugs introduced into clinical practice) has shown that, despite regulatory decisions, ‘trust’ by doctors in the equal efficacy of the biosimilar drug and the originator—and the consequent use of biosimilar drugs in clinical practice—may be scarce.20 21 At least in principle, postmarketing trials designed to compare biosimilar drugs with originator drugs would have the merit of increasing the level of trust in biosimilars by the oncological community. This could be particularly true at the beginning, when biosimilars are not readily and widely accepted, unless there are no administrative directives. As explained above, a particularly critical condition is the extrapolation from one indication to another. However, the panel acknowledges that these trials could not be realistically designed following the strict methodological criteria used for testing a new drug, where a formal demonstration of efficacy compared with the standard treatment is required. In fact, this would imply a very large number of patients and a very large amount of financial resources that would represent a disappointing obstacle to the development of biosimilars. Consequently, the priority in the investment of resources by public bodies (regulatory authorities, scientific societies, cooperative groups) in postmarketing trials designed to directly compare a biosimilar drug and the originator drug is rather low compared with the priority of trials comparing different drugs (table 2).

Table 2

Priority of resource investment by public bodies in clinical trials testing direct comparison between different drugs approved for the same indication

Scenario	Priority of investment of resources in direct comparison trials	Comment
Biosimilar drugs	Low	The conduction of postmarketing trials comparing biosimilar and originator is a low priority, although it would increase the degree of ‘trust’ of the scientific community in the efficacy of the biosimilar drug.However, the panel believes that the positive results of the comparability exercise are sufficient to recommend a wide use of biosimilar drugs in clinical practice.
Same indication, same mechanism of action	Variable	The smaller is the plausible difference between the treatments in question, the lower the priority of resource investment in direct comparisons.
Same indication, different mechanism of action	High	Priority depends on the clinical relevance and on the economic relevance (price difference between the drugs) of the unresolved clinical problems, and on the cost that would be involved in conducting such studies.

Same therapeutic indication, different drug with same or different mechanism of action

The availability in clinical practice of drugs with different mechanisms of action, even in the absence of a direct comparison, is often useful from a clinical perspective (due to the possibility of sequential use in different lines of treatment, as well as to different toxicity profiles). On the other hand, in most cases, the panellists agreed that the co-presence on the market of different drugs with the same mechanism of action is less justified, unless there are radical differences in the toxicity profile (which permit the choice of one treatment or another, depending on the patient). In certain cases, however, even if the drugs belong to the same class and therefore have the same mechanism of action, they have pharmacological differences which, at least in principle, could lead to a different efficacy. Conversely, the plausibility of a different efficacy between the treatments under consideration becomes lower as the pharmacological differences diminish.

How to choose among different drugs approved in the same setting, in the absence of a direct comparison?

In principle, the AIOM working group emphasised that randomised trials in which the control arm is not the best treatment available in clinical practice should never be conducted. However, it is a fact that the regulatory authorities can be in the position of approving a new drug that has not been compared with the other drug (or drugs) currently available for the same indication, but it has been compared with the previous standard (which, for instance, may have been the standard at the time of trial design and/or conduction). Such situations of ‘parallel’ development of new drugs have arisen and probably will continue to arise, in some cases ‘justified’ by reasons linked to ‘time frames’ preceding the introduction of a drug into clinical practice and, in other cases, as a result of geographical differences in the availability of the drugs.

In principle, a coordination of different programmes of clinical research, instead of the parallel development of similar drugs, by means of a project shared among several pharmaceutical companies, and ‘flexible’ over time (in order to optimise the inclusion of new drugs and the process of selection), would be optimal. Of course, such a scenario would imply a robust strategic and coordinating role played by the regulatory authorities. For instance, the model being developed by the US Food and Drug Administration and the National Cancer Institute, with the involvement of several pharmaceutical companies, in the so-called master protocol, specifically designed for lung cancer, could be ‘exportable’ into other settings.22 Such coordinating programmes can be designed to allow the parallel development of drugs with different targets, but they could ideally allow also an early comparison of drugs for the same therapeutic indication, in order to select the best treatment to introduce in clinical practice. Unfortunately, this ideal coordination among development programmes for different drugs seems quite difficult to accomplish in practice, and drugs will probably continue to be introduced in clinical practice without direct comparison with the possible alternatives.

In the absence of trials designed to compare the different therapeutic strategies, it is necessary to perform indirect assessments, which however are highly arbitrary, lack recognised methodological standards for the assessment of their quality and consequently produce evidence of questionable reliability.

How reliable are indirect comparisons (eg, network meta-analyses), which are often the only available comparison among different treatments for the same indication?

When more than one treatment has been used and compared, either head-to-head or indirectly, in the same clinical setting, network meta-analyses (also known as multiple treatment comparison meta-analyses or mixed treatment meta-analyses) offer a technical methodology to compare the relative effectiveness of all included interventions, allowing to synthesise and interpret all the available evidence.23–25

The AIOM working group considered that indirect comparisons are, in many cases, unreliable. As a general rule, when assessing the ratio between benefits and risks of the individual treatments available, the results of simple mathematical exercises (such as network meta-analyses), even if conducted with a technically correct procedure, should be less relevant compared with a detailed methodological and clinical analysis of existing evidence.

Differently from ‘traditional’ meta-analyses, network meta-analyses, by definition, are not restricted to combining the results of studies which directly compared treatment A with treatment B, but rather they compare the results of two or more treatments, combining studies which have a common treatment arm.24 From a methodological point of view, such meta-analyses represent a potentially dangerous tool, because their conduction (and publication) risks to dress with methodological rigour and objectivity comparisons that remain weak and scientifically debatable. In particular, if a network meta-analysis combines the HRs from different studies, such HRs will be strongly conditioned by the duration of enrolment and of follow-up in single studies, and also by the mechanism of action of one treatment with respect to the other. For example, a network meta-analysis could be conducted to formally quantify the difference in efficacy between drug A, which produces a median survival advantage roughly distributed across the entire study population, and drug B, which although not significantly altering the median survival, determines a relevant increase in the percentage of long-term survivors. It is clear that such a network meta-analysis would produce distorted results and would be difficult to interpret. Therefore, the panel emphasised that indirect comparisons, albeit useful in hypothesis generation, should not be mere and misused mechanical exercises which, though ‘formally’ correct, may be misleading from a clinical point of view. Instead, indirect comparisons between different drugs available for the same indication should be an exercise at the highest scientific level, in which statistical and clinical skills are integrated in an attempt to examine the distribution of the effects of the various treatments. Therefore, the difference in medial survival, the percentage of participants alive at a specific time point and the HR itself may be insufficient, especially if taken singly, while more complex assessments are necessary. In general, the ‘network’ meta-analysis should serve as support, and not as a decisional tool to sanction on the superiority of one treatment rather than another.

Considering the above described limitations, the interpretation of the indirect comparisons should be limited to the discussion of any medium–large differences, ignoring the statistical significance and differences of modest clinical relevance, in which the weight of comparison bias may be greater than the real difference between the treatments under study.

Should public bodies (regulatory authorities, scientific societies, cooperative groups) consider a priority to invest resources in direct comparison (postmarketing) between different drugs available for the same indication?

In general, given the substantial limitation of indirect comparisons (as described in the previous paragraph), the AIOM working group judged the conduction of clinical trials directly comparing different drugs for the same indication as a priority. Such trials aim to increase scientific evidence, and optimise therapeutic choices in clinical practice. The priority for resource investment in direct comparisons depends on the clinical relevance and on the economic relevance (difference in price between the drugs) of any unresolved clinical issues (table 2). The larger is the possible or actual difference between the treatments, the higher the priority of resource allocation for direct comparison. The priority for investing resources in a direct comparison between drugs with the same mechanism of action is necessarily lower. However, even if the drugs belong to the same class, if the difference that can be assumed (on the basis of indirect comparisons and of pharmacological differences) is potentially relevant, then the priority could be higher.

After the approval of a new drug for use in clinical practice, it is obviously not realistic that pharmaceutical companies will promote further trials that would challenge the indication already obtained. Consequently, the conduction of these trials should be a priority for academic research that has the aim of optimising therapeutic decisions in clinical practice. Although we are aware that the availability of funding for this research can be really difficult, the panel strongly believes that public bodies (regulatory agencies, scientific societies, cooperative groups) should encourage and support, with dedicated funds and calls for projects, the conduction of these trials.

How should a direct comparison be designed?

The general answer is relatively simple: when several drugs have been approved for the same indication, but not yet directly compared, the choice of the study design (superiority or non-inferiority design) depends on the comparison between the toxicity profiles (which is grossly possible a priori, even in the absence of a direct comparison) and also depends on the differential efficacy (which, in the absence of a direct comparison, is necessarily only ‘presumed’; figure 1).

Figure 1

Choice of the study design (superiority vs non-inferiority design) for postregistration trials comparing different treatments for the same therapeutic indication, based on the presumed difference in efficacy and on the toxicity profiles.

When it is plausible to assume similar efficacy (or small differences in efficacy between the treatments), it will be the more toxic drug that needs to demonstrate, within a direct comparison, a greater efficacy, that will justify its use despite the excess of toxicity. In this case, it is correct to consider a superiority design: the less toxic treatment will be the reference (control) arm, while the more toxic treatment will be the experimental arm. Necessarily, the greater the difference in toxicity, the greater will be the threshold of the increase in efficacy which must be demonstrated to declare the superiority of the experimental arm. In the case of a negative result, the less toxic drug will remain preferable as the standard in clinical practice. As for other superiority trials, the threshold of increased efficacy—either as a relative or an absolute benefit—that would determine the preference for the more toxic treatment, is clearly dependent from the specific clinical setting, in terms of patients' prognosis and toxicity profile of the drugs, and should be clearly discussed in the study protocol. When the toxicity is high, even a potentially relevant difference in survival could not be enough to convince clinicians and patients to use the more toxic strategy. On the contrary, the same difference could be sufficient if the toxicity, although more relevant, is considered manageable.

On the other hand, if it is plausible that the more toxic drug is also associated with a greater efficacy, the aim of a direct comparison will be to demonstrate that the use of the less toxic drug is not associated with a ‘clinically relevant’ loss of efficacy. Therefore, in this case, it will be correct to consider a non-inferiority design, in which the more toxic treatment will be the reference arm, and the less toxic treatment will be the experimental arm. If the study is negative (ie, non-inferiority not demonstrated), the more toxic drug will remain preferable as the standard in clinical practice. However, it is useful to underline that, as a general rule, non-inferiority trials should only be conducted when the experimental treatment presents clear advantages for the patient (eg, a lower toxicity, or a more suitable administration route) or for the community (as in the case of biosimilars) without any predictable damage for the patients. In the absence of such advantages, a non-inferiority trial loses its rationale, and it becomes ethically unacceptable. Of note, the extent of such advantages is relevant for the study design, in that it enables to determine the acceptable margin of non-inferiority. It is particularly difficult to propose a general rule for the definition of margins of non-inferiority, because this definition is necessarily related to each specific clinical scenario. Clinical reasons for the definition of margins of non-inferiority should be clearly discussed in the study protocol, based on the prognosis of patients, toxicity of treatments considered and the absolute benefit that the more toxic treatment has demonstrated compared with the previous standard. A potential loss of 2 months in median survival with the less toxic of two treatments can be clinically acceptable, if the difference in toxicity is large and if the absolute benefit previously demonstrated by the more toxic treatment is 5–6 months. On the contrary, the same potential loss of 2 months in median survival would be obviously unacceptable, even in the presence of a large difference in toxicity, if the absolute benefit of the more toxic treatment is lower than 3 months.

If the drugs being compared are both approved for clinical practice, there could be confusion in the interpretation of results, related to cross-over. In fact, many of these studies should be designed (and interpreted) as studies of treatment sequences rather than ‘simple’ comparisons of two different single lines of treatment.

In particular cases, especially with rare diseases, when the toxicity profiles are qualitatively very different while the efficacy appears, at least on the basis of indirect comparisons, roughly superimposable, an observational study could be more feasible than a randomised comparison. In this case, the choice of treatment could be left to the patient, on the basis of the toxicity profile (while randomisation could be performed just for those patients who are uncertain about the choice), and the analysis of clinical outcomes in the different treatment groups would add evidence useful for future patients.

Conclusions

In many clinical settings, clinical trials comparing different treatment options for the same therapeutic indication would produce important evidence, allowing better decisions in clinical practice. As expected, most of these trials are not part of the development of drugs aimed at their registration, and their conduction could be part of the postregistration phase.

With this position paper, the panel of clinicians and methodologists of the Italian Association of Medical Oncology emphasise the need for supporting this aspect of clinical research. In addition, the panel suggests priorities in the allocation of financial resources for the conduction of these non-profit trials by public bodies (regulatory agencies, scientific societies, cooperative groups). In the absence of previous direct comparisons, the larger the potential difference between the existing therapeutic options, the higher is the priority for the conduction of these trials. Of course, the feasibility of these trials will depend on the cost involved in their conduction. From this point of view, scientific societies should put pressure on the regulatory authorities, to identify all the administrative and financial mechanisms useful to facilitate the conduction of such trials, limiting costs as much as possible.