Level of evidence used in recommendations by the National Comprehensive Cancer Network (NCCN) guidelines beyond Food and Drug Administration approvals.

BACKGROUND: A previous analysis of 113 National Comprehensive Cancer Network® (NCCN®) recommendations reported that NCCN frequently recommends beyond Food and Drug Administration (FDA)-approved indications (44 off-label recommendations) and claimed that the evidence for these recommendations was weak.
METHODS: In order to determine the strength of the evidence, we carried out an in-depth re-analysis of the 44 off-label recommendations listed in the NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines®).
RESULTS: Of the 44 off-label recommendations, 14 were later approved by the FDA and/or are supported by randomized controlled trial (RCT) data. In addition, 13 recommendations were either very minor extrapolations from the FDA label (n = 8) or were actually on-label (n = 5). Of the 17 remaining extrapolations, 8 were for mechanism-based agents applied in rare cancers or subsets with few available treatment options (median response rate = 43%), 7 were based on non-RCT data showing significant efficacy (>50% response rates), and 2 were later removed from the NCCN Guidelines because newer therapies with better activity and/or safety became available.
CONCLUSION: Off-label drug use is a frequent component of care for patients with cancer in the United States. Our findings indicate that when the NCCN recommends beyond the FDA-approved indications, the strength of the evidence supporting such recommendations is robust, with a significant subset of these drugs later becoming FDA approved or supported by RCT. Recommendations without RCT data are often for mechanism-based drugs with high response rates in rare cancers or subsets without effective therapies.

Introduction

The National Comprehensive Cancer Network® (NCCN®) is a not-for-profit alliance of 28 leading cancer centers in the United States devoted to patient care, research, and education. The NCCN mission includes improving the quality, effectiveness, and efficiency of cancer care so that patients can live better lives, and, in order to do so, NCCN promotes continuous quality improvement and creates clinical practice guidelines appropriate for use by patients, clinicians, and other health care decision-makers. The NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines®) comprised recommendations for the prevention, diagnosis, and management of malignancies across the continuum of care. The guidelines currently apply to over 97% of patients living with cancer in the United States and incorporate real-time updates of the rapid advances in cancer research [1]. The NCCN Guidelines are used worldwide, with nearly half (46.6%) of registered users being from outside the United States and guideline downloads from over 180 countries [1].

The NCCN Guidelines are developed by panels of experts from NCCN Member Institutions who diagnose and treat patients with a broad spectrum of cancers. In addition to common cancers, the guidelines deal with complex, aggressive, or rare cancers (e.g., inflammatory myofibroblastic tumor, hairy cell leukemia, malignant thymoma, thymic carcinoma), which often reflect an unmet need for evidence-based therapy approaches. At least yearly, clinical cancer experts at the NCCN Member Institutions identify areas where new data, obtained via multiple modalities including literature review or external submission requests, have changed the standard of care, and these areas are discussed at the annual panel meeting, serving as a basis for possible changes to the recommendations contained in the guidelines. The NCCN Guidelines are subject to frequent rounds of review and revision by disease experts in order to optimize the treatment recommendations. For some guidelines, this results in multiple updates per year as new data becomes available.

In order to safeguard the objectivity and integrity of the NCCN Guidelines development and update process, the NCCN has put in place a comprehensive policy for disclosure of financial relationships and for management of potential conflicts of interest. Individuals are disqualified from panel membership if they receive non-research support from industry of $20,000 or more from a single entity or more than $50,000 in aggregate from all entities in a 12-month period. Individual panel members must recuse themselves from deliberations and/or votes when there is a meaningful conflict of interest of any level. In addition, the NCCN does not accept any form of industry or external financial support for the development of the guidelines. Development of the NCCN Guidelines is funded exclusively by the Member Institutions’ dues [1].

In the United States, the Food and Drug Administration (FDA) is responsible for assuring the safety, efficacy, quality, and security of human drug and biologic therapies. When the FDA approves a therapy, it means that the agency has reviewed available data to determine that the benefits of the therapy outweigh the risks for the approved indication [2]. Off-label use is the practice of using an approved drug or biologic for a disease or setting in which the therapy is not FDA labelled. This practice is common, particularly for cancer treatment, where off-label use of therapies is estimated to be around 30% in the United States [3, 4]. The reasons behind the prevalence of off-label use are multifaceted, stemming from the relatively narrow indications often specified on the FDA label, a lack of FDA-approved drugs available for a certain disease or setting, and the desire to provide a promising new drug to a patient who might not have access through an FDA approval or a clinical trial [5]. In addition, even when there are positive trials supporting the use of an FDA-approved drug in new settings, the pharmaceutical company may not pursue FDA approval for these expanded indications due to the cost of doing so. As a result, off-label drug use is often, but not always, based on prospective clinical trials, sometimes including randomized phase III studies. Insurance coverage of off-label drug prescriptions is guided through the use of compendia. The NCCN Drugs & Biologics Compendium (NCCN Compendium) is widely recognized by public and private insurers alike as an authoritative reference to guide oncology coverage decisions [6].

Our study was conducted in response to the article titled ‘Frequency and Level of Evidence Used in Recommendations by the National Comprehensive Cancer Network Guidelines Beyond Approvals of the US Food and Drug Administration: Retrospective Observational Study’ published in the March 2018 issue of BMJ [7]. Based on an analysis of off-label oncology drug indications recommended in the NCCN Guidelines, this article came to the conclusion that ‘The NCCN frequently recommends beyond the FDA approved indications even for newer, branded drugs. The strength of the evidence cited by the NCCN supporting such recommendations is weak’. Our study sought to determine the evidence supporting the recommended off-label indications and the rationale behind these recommendations.

Methods

We determined the versions of the NCCN Guidelines that would have been available when Wagner et al. downloaded the library of guidelines on 25 March 2016 and used these versions for our analysis. Working from the supplementary Table in their paper, which provided the basis for the results described in the original article [7], we confirmed whether or not the extrapolation claimed in the original article was accurate, located the specific setting for the Guideline recommended therapy, noted any relevant Guideline footnotes that clarified the setting, recorded the NCCN Category of Evidence and Consensus, and specified the algorithm page containing the extrapolation (see supplementary Table S1, available at Annals of Oncology online).

Next, we used these versions of the NCCN Guidelines to record what data were cited as supporting the extrapolation. Most often, these data were cited either within the algorithm pages (as a footnote or reference) or within the discussion text. In some cases, generally when the algorithm had been recently updated and the discussion text was still under development, we looked to the Transparency Document. Transparency Documents are posted on the NCCN website to document all changes made in response to external requests to the recommendation category or indication of drugs or biologics, a short summary of the panel discussion, and the rationale for the change. They include references (if applicable), and the results of the panel vote for each of the changes listed (indicated as a subfolder for each guideline entitled ‘Minutes/Evidence’ that can be found in the ‘Transparency: Process and Recommendations’ tab under the NCCN Compendium section).

We furthered our analysis by determining whether the therapy was subsequently FDA-approved for the Guideline indication and searched for subsequent published randomized controlled trial (RCT) data in settings where there was no published RCT data supporting the extrapolation at the time of initial analysis.

The above analysis was carried out by two senior NCCN investigators (RK and ABB) together with an NCCN Oncology Scientist (LAG) and discussed on several teleconferences in order to carefully examine the data. We also consulted panel experts in specialties where the NCCN Guidelines had been questioned in order to validate and crosscheck our information.

Results

The results of our analysis are shown in Figure 1. As described by Wagner et al. [7], the 47 drugs that were approved by the FDA between 1 January 2011 and 31 December 2015 were recommended for 113 indications in the NCCN Guidelines. Of these 113 recommended uses, 69 were aligned with the FDA-approved indications for the therapies. Therefore, the NCCN Guidelines recommended these therapies in an additional 44 settings beyond the FDA indications (see Figure 1 and supplementary Table S1, available at Annals of Oncology online). As indicated in the supplementary Table S1, available at Annals of Oncology online, 100% of these off-label recommendations were supported by data, which were most often referenced in the NCCN Guidelines algorithm and/or discussion section (see supplementary Table S1 and Results, available at Annals of Oncology online). Occasionally, in situations where the therapy had been recently added and the discussion had not yet been updated, the data were referenced in the Transparency Document.

Figure 1.

Consort diagram of off-label recommendations in the National Comprehensive Cancer Network (NCCN) guidelines. Of the 44 off-label recommendations, our analysis found that 14 of the indications were later approved by the Food and Drug Administration (FDA) and/or supported by randomized controlled trial (RCT) data. In addition, 13 of these recommendations were identified to be either very minor extrapolations from the FDA label (n = 8) or were actually on-label as described in the NCCN Guidelines (n = 5). Of the 17 remaining extrapolations, 8 recommendations were for rare cancers or subsets with few available treatment options, 7 recommendations were based on non-RCT data showing high (>50% response rates), and 2 recommendations were later removed from the NCCN Guidelines based on data showing that new therapies had better efficacy and/or safety.

To break down these 44 off-label recommendations listed in the NCCN Guidelines further, our analysis found that 14 of the indications were either later FDA-approved or were supported by RCT data [8-12] (see Figure 1 and supplementary Table S1, available at Annals of Oncology online). Additionally, 13 of the off-label recommendations were either very minor extrapolations from the FDA-approved label (n = 8, described in more detail in supplementary Table S1, available at Annals of Oncology online) or the indications recommended within the NCCN Guidelines actually represented on-label use (n = 5) and were incorrectly scored by Wagner et al. [7] (supplementary Table S1, available at Annals of Oncology online).

After subtracting the off-label recommendations that were later FDA-approved, based on trial data, minor extrapolations from the label, and/or uses that were actually on-label, a total of 17 off-label NCCN Guidelines recommendations that were based on lower-level data remained. These 17 indications could be further grouped into 3 categories: (i) significant agent activity in rare cancers or subsets with few treatment options (n = 8) (median response rate = 43%) [13-21]; (ii) non-RCT showing high (over 50%) response rates (n = 7) [22-27]; or (iii) recommendations that were later removed from the NCCN Guidelines based on data showing that new therapies had better efficacy and/or safety (n = 2) [28-32].

A detailed discussion of the off-label recommendations listed in the NCCN Guidelines that were not supported by RCT, and comments from disease experts on why these therapies are appropriate in the given settings, is included in the supplementary Results, available at Annals of Oncology online.

Discussion

As described by Wagner et al. [7], 47 drugs that were approved by the FDA (1 January 2011 through 31 December 2015) were recommended by NCCN Guidelines for 113 indications, 69 of which were aligned with FDA-approved indications, and 44 of which were extrapolations (see Figure 1 and supplementary Table S1, available at Annals of Oncology online). Wagner et al. [7] claimed that 36% of these recommendations were made with no evidence given. Yet, our review found that 100% of the off-label recommendations were supported by data, which were most often referenced in the NCCN Guidelines algorithm and/or discussion section or, more rarely, in the Transparency Document. In addition, 14 of the 44 extrapolations (32%) in the NCCN Guidelines were later FDA-approved or were supported by RCT data (Figure 1) and an additional five (11%) were actually consistent with FDA on-label use (total = 19/44 [43%]). Therefore, in many cases, evidence- and consensus-based treatment recommendations such as those within the NCCN Guidelines make promising therapies available to patients months to years before FDA approval.

The NCCN Guideline algorithms aim to graphically display treatment pathways, which can often be complex, in as clear and concise a manner as possible. To accomplish this, important details about certain treatment options are sometimes listed within footnotes (e.g. previous therapy requirements for trifluridine/tipiracil for colorectal cancer; panobinostat or pomalidomide for multiple myeloma) and the flow of the algorithm provides important context regarding the treatment options offered at each setting (e.g. previous therapy requirements for nivolumab in kidney cancer; specific disease setting for crizotinib in NSCLC). In several cases, Wagner et al. listed a supposed off-label extrapolation that actually represented an on-label use once the context of the algorithm flow and relevant footnotes were taken into account.

Physicians have the responsibility to ensure that their patients receive the best possible treatment, and, at the same time, share the societal responsibility for judicious use of limited resources—two tasks that can at times appear to be in conflict. Responsible physicians not only strive to make evidence-based treatment decisions, when possible, but also consult with the patient, taking into account their wishes. The value of a therapy for a specific patient cannot be purely interpreted based on P values, hazard ratios, and survival outcomes. In addition, cancer treatment is often applicable to small numbers of patients that, especially for lethal diseases, cannot afford the time and expense that a large RCT would entail. Hence, physicians must often make decisions based on other types of compelling evidence. Cancer patients and oncologists struggling to wade through the complex, and often confusing, data emerging from clinical trials need recommendations that outline evidence-based treatment options across a variety of clinical situations, including those for which there is no FDA-approved therapy available.

The purpose of the NCCN Guidelines is to provide insight into treatment strategies for lethal diseases that are often complicated. Disease experts who serve on the NCCN Panels are charged with evaluating the disease landscape to define settings where there is robust evidence to guide treatment and also use their clinical experience to help fill gaps in evidence, giving patients treatment options across the disease continuum. As a result, payers look to NCCN as a responsible party to help guide this complex morass of disease settings and the complicated array of drugs and drug combinations.

Importantly, over 20% of the cancer burden in the United States comprised rare malignancies [33] (and an additional substantial portion represents rare subsets of common cancers), where it would take years to accrue an RCT; furthermore, such trials in these uncommon cancers may be prohibitively expensive. Hence, data from RCTs are not and may never be available for a substantial subgroup of patients with cancer. Indeed, few FDA-approved therapies exist for many patients with rare cancers. Clinical trial design for rare cancers or subsets of cancer is changing to attenuate this challenge. FDA approvals are increasingly based on phase II trials for diseases where phase III trials would not be feasible. Furthermore, basket studies which test the efficacy of a drug that targets a specific mutation across tumor types are becoming increasingly common. However, while changes to the FDA approval process for rare cancer treatments are ongoing, many rare cancers have few or no treatment options that are FDA-approved or supported by RCT data. Physicians have an obligation to support therapeutic recommendations for patients with rare tumors, particularly when the tumor biology and best available evidence supports the benefit of therapy. Finally, for some therapies with remarkable efficacy, an RCT may not meet the equipoise criteria (meaning that the better arm is uncertain) that provide the ethical foundation on which these trials are built.

In contrast to the study by Wagner et al., a recent study reviewed the indications listed in the NCCN Compendium for 43 cancer drugs that were approved between 1999 and 2011 and compared these with the FDA-approved indications [34]. Of the 253 off-label uses across the 43 drugs reviewed in the study, 91% were deemed a ‘well-accepted off-label use’ by the authors, meaning the drug received either a Category 1 or Category 2A NCCN Guidelines recommendation in that setting [1, 34]. Additionally, 65% of the off-label uses were for cancer types not represented in the FDA labeling at the time of analysis. Off-patent drugs were found to have more ‘well-accepted off-label uses’ than on-patent drugs. The authors of this study came to the conclusion that steps should be taken to better align FDA-labeling with real-world clinical cancer care in cases where high-quality data exist. Internationally, the same argument could be made, encouraging regulatory and reimbursement authorities to continually update indications for use and safety based on the evolution of high quality data generated after the initial approved labelling—with the NCCN Guidelines as an example.

There are limitations to our study. Twenty-five of the 113 analyzed NCCN recommendations have not been FDA approved to date. Studies have shown that for drugs approved based on single-arm trials or expedited FDA pathways, unrecognized side-effects are sometimes later reported [35]. Although NCCN does not accept any form of industry support for guideline development and has strict policies in place restricting support received by panelists, it remains conceivable that even limited financial or other undefined incentives might yield conflicts of interest.

In conclusion, our findings are in agreement with previous research [34] that the NCCN recommendations for systemic agents are based upon scientific evidence in settings frequently beyond the FDA-approved indications. In most cases, the strength of the evidence cited by the NCCN supporting such recommendations is robust. Recommendations without supporting data from RCT are often for mechanism-based drugs in rare cancers or subsets with no effective approved therapies. In these cases, the NCCN recommendations allow for treatment, often with high response rates, giving these patients a chance for therapeutic benefit that they would not otherwise have. The NCCN asserts that when effective drugs are available—often with strong evidence supporting their use—patients should have access to these therapies, even if they do not have an FDA-approved indication for that specific clinical context.

Funding

None declared.

Disclosure

R. K. receives research funding from Incyte, Genentech, Merck, Serono, Pfizer, Sequenom, Foundation Medicine, Konica Minolta, Grifols, and Guardant, as well as consultant fees from X Biotech, Loxo, NeoMed, and Actuate Therapeutics, speaker fees from Roche, and is a shareholder in IDbyDNA and in Curematch Inc. D. S. E. is an advisor for BeyondSpring Pharmaceuticals, Boehringer Ingelheim, Bristol-Myers Squibb, Eli Lilly & Co, Genentech, Golden Biotech Corp, and Guardant Health Inc. S. M. H. receives fees for scientific advisory boards, consulting, or serving as an expert witness for ADC Therapeutics, Celgene Corporation, Corvus, Innate, Kiowa Hakka Kirin, Millennium Pharmaceuticals Inc, Miragen, Portola, Seattle Genetics, Takeda Pharmaceuticals North America Inc, and Verastem. S. K. K. receives research funding for clinical trials to the institution from Celgene, Takeda, Janssen, BMS, Sanofi, KITE, Merck, Abbvie, Medimmune, Novartis, Roche-Genentech, and Amgen as well as participating in consulting/advisory boards (with no personal payments) through Celgene, Takeda, Janssen, KITE, Merck, Abbvie, Medimmune, Genentech, Amgen and (with personal payment) Oncopeptides, and Adaptive; in addition, he receives honoraria from Dr. Reddys Lab and Ono Pharmaceuticals. M. v. M. receives research funding from Deciphera Pharmaceuticals, Blueprint Medicines, Lilly, Novartis, and Immune Design; she also serves as a consultant for Deciphera Pharmaceuticals. A. B. B. receives research funding from Novartis, Acerta, Celgene, Advanced Accelerator Applications, Infinity Pharmaceuticals (data monitoring committee), Merck Sharp and Dohme, Taiho Pharmaceutical, Bristol-Myers Squibb, Medimmune/AstraZeneca, Xencor, and Bristol-Myers Squibb (data monitoring committee), as well as functioning in a consulting or advisory role for Bristol-Myers Squibb, Guardant Health, Eli Lilly & Company, Exelixis, Purdue Pharma, Harborside (CME presentations), Xcenda, NCCN (meeting presentations), Emron (editorial board, publications), Inventive Health Inc, Axio, Genentech, Bayer, Merck, Rafael Pharmaceuticals, Astellas (data monitoring committee member), and Terumo. L. M. has disclosed that she has no relevant financial relationships. R. W. C. and L. A. G. are employed by NCCN. R. K., D. S. E., S. M. H., S. K. K., L. M., M. v. M., and A. B. B. serve on NCCN Panels, Committees, and/or Boards.

Click here for additional data file.