Enhancing Next-Generation Sequencing-Guided Cancer Care Through Cognitive Computing.

BACKGROUND: Using next-generation sequencing (NGS) to guide cancer therapy has created challenges in analyzing and reporting large volumes of genomic data to patients and caregivers. Specifically, providing current, accurate information on newly approved therapies and open clinical trials requires considerable manual curation performed mainly by human "molecular tumor boards" (MTBs). The purpose of this study was to determine the utility of cognitive computing as performed by Watson for Genomics (WfG) compared with a human MTB.
MATERIALS AND METHODS: One thousand eighteen patient cases that previously underwent targeted exon sequencing at the University of North Carolina (UNC) and subsequent analysis by the UNCseq informatics pipeline and the UNC MTB between November 7, 2011, and May 12, 2015, were analyzed with WfG, a cognitive computing technology for genomic analysis.
RESULTS: Using a WfG-curated actionable gene list, we identified additional genomic events of potential significance (not discovered by traditional MTB curation) in 323 (32%) patients. The majority of these additional genomic events were considered actionable based upon their ability to qualify patients for biomarker-selected clinical trials. Indeed, the opening of a relevant clinical trial within 1 month prior to WfG analysis provided the rationale for identification of a new actionable event in nearly a quarter of the 323 patients. This automated analysis took <3 minutes per case.
CONCLUSION: These results demonstrate that the interpretation and actionability of somatic NGS results are evolving too rapidly to rely solely on human curation. Molecular tumor boards empowered by cognitive computing could potentially improve patient care by providing a rapid, comprehensive approach for data analysis and consideration of up-to-date availability of clinical trials. IMPLICATIONS FOR PRACTICE: The results of this study demonstrate that the interpretation and actionability of somatic next-generation sequencing results are evolving too rapidly to rely solely on human curation. Molecular tumor boards empowered by cognitive computing can significantly improve patient care by providing a fast, cost-effective, and comprehensive approach for data analysis in the delivery of precision medicine. Patients and physicians who are considering enrollment in clinical trials may benefit from the support of such tools applied to genomic data. © AlphaMed Press 2017.

Introduction

Next‐generation sequencing (NGS) has emerged as an affordable and reproducible means to query patients’ tumors for somatic genetic anomalies [1], [2]. The optimal utilization of NGS is fundamental to the promise of “precision medicine,” yet the results of even targeted‐capture NGS are highly complex, returning a variety of somatic events in hundreds of analyzed genes. The majority of such events have no known relevance to the treatment of patients with cancer, and even for well‐understood driver events (e.g., BRAF V600E), the combinatorial significance of other mutations is poorly understood. Tumors may also possess a large degree of intratumor heterogeneity [3], adding further complexity to the analysis.

In order to handle somatic NGS data, many institutions have created a “molecular tumor board” (MTB) made up of physicians and biomedical scientists to analyze the results of NGS and make recommendations. MTBs in turn rely on an “actionable gene list” of specific genetic events that should be considered of clinical significance. Mutations on this list can guide therapy (e.g., BRAF mutations in melanoma), provide prognostic information (e.g., FLT3 mutations in acute myelogenous leukemia), or be of diagnostic utility (e.g., EWS‐FLI translocations in a poorly differentiated pediatric sarcoma). At present, the state of our clinical knowledge regarding the actionability of genetic events evolves and grows sporadically, largely through the reporting of new clinical results in a variety of scientific venues. As thousands of new cancer‐relevant publications are produced daily, staying abreast of new clinical information in oncology is extremely challenging and differs from institution to institution.

In 2015, there were more than 2.5 million scholarly articles published, with more than 150,000 directly related to cancer, a volume of literature that is growing at an estimated rate of 6.7% per year over the past decade [3]. At the time of submission of this article, a total number of 61 targeted inhibitors are approved for the treatment of a few conditions. There are more than 650 targeted therapies in development, with thousands of clinical trials in progress. In order to be used by current computer systems, relevant data from these articles and clinical trials, which are written in a free‐text format (unstructured data), need to be extracted, cleaned, translated to canonical forms, validated, and formatted in structured tables and databases. In many cases, this process is done manually by subject matter experts who must read the free text and transpose the information into a structured format. However, with such rapid growth of new data, this is both expensive and not a practical, scalable approach.

Materials and Methods

Targeted Exon Sequencing

Targeted exon sequencing was conducted after institutional review board (IRB) approval through the UNCseq (University of North Carolina, Chapel Hill, NC, http://www2.cscc.unc.edu/unchreg/UNCseq) pipeline, as previously described [4], [5]. In brief, sequencing data are routed through an automated pipeline. This workflow uses paired tumor and normal libraries to detect somatic mutations, large and small indels, structural variants, and copy number aberrations. Raw sequences are aligned using the Burrows‐Wheeler Aligner (BWA)‐mem algorithm and refined using our Assembly Based ReAlignment (ABRA [6]) process. Our UNCeqR [7] algorithm, combined with Strelka [8], then provides sensitive detection of somatic variants.

UNCseq Actionability

Genes were defined as “actionable” by the University of North Carolina (UNC) MTB if they met criteria that placed them in Tier 1 (variant targeted by commercially available drug that is approved to treat this specific genetic variation), Tier 2A (variant potentially treatable by commercially available targeted drug, but the drug is not indicated for this use), or Tier 2B (variant is potentially treatable by targeted drug that is in clinical trials) [1].

All variants of unknown significance (VUS) are evaluated by the UNC MTB and, based upon characteristics of the VUS and a search of the published literature, are considered for return into the medical record. Not all VUSs are returned to the medical record.

Watson for Genomics

Upon sequence completion by UNCseq, the following information was uploaded to Watson for Genomics (WfG): (a) tumor type, (b) a list of variants as a variant calling file (.vcf), and (c) gene level copy number alterations as a log 2 ratio of tumor to normal. After the above information was uploaded, the following steps were executed by WfG for each gene with a variant or copy number alteration (supplemental online Fig. 1):

Evidence from functional studies and protein structure, combined with programming logic, was used to classify variants into five categories: pathogenic, likely pathogenic, benign, likely benign, and VUS. Benign and likely benign mutations were removed from the report.

Once the alterations were categorized by the MPA, WfG assigned a gene as actionable if (a) the variant was pathogenic or likely pathogenic, (b) the variant was directly targetable or part of a pathway that was targetable based on evidence from the literature, and (c) a U.S. Food and Drug Administration‐approved or investigational target therapy was available.

In this step, potential therapeutic options were associated with actionable mutations. Note that only clinical trials actively recruiting were considered during this step. Analysis of resistance was also performed during this step. WfG uses levels of evidence (Table 2) for the gene, variant, cancer type, and drug association.

Table 2.

Levels of evidence used by Watson for Genomics

Abbreviation: FDA, U.S. Food and Drug Administration.

A report was generated by WfG showing the variants (pathogenic, likely pathogenic, and VUS) alongside potential targeted drugs. An example of the user interface is shown in supplemental online Figure 1.

The supplemental online Methods provides further details on WfG natural language processing (NLP) techniques and literature extraction. Supplemental online Figure 1 outlines the workflow for Watson for Genomics Analysis, representing the four steps described above.

Results

Several cancer centers have started using cognitive computing systems [9], yet few data exist on the validity of these approaches or their promise to deliver personalized medicine to patients. Cognitive computing is a form of artificial intelligence (AI) that relies on machine learning, NLP, and other data analysis technologies to understand and draw patterns from massive volumes of disparate data. Cognitive computing approaches such as WfG rely on computerized models to simulate aspects of human thought. In particular, using state‐of‐the‐art machine learning, NLP, and cognitive insights, WfG is well suited to ingest, organize, aggregate, and extract relevant insights from large volumes of rapidly emerging clinical data. Although cognitive systems can ingest new information in real time, WfG uses an offline process that consists of extraction, validation, and testing before the insights are incorporated into real‐time analysis (supplemental online Fig. 1). Importantly, WfG can learn new information and analyze data at a rate that far exceeds manual curation and analysis [10].

In order to compare the effectiveness of the WfG cognitive computing engine with human‐only MTBs at identifying treatment options for patients, we retrospectively examined 1,018 consecutive cases from UNC that had undergone targeted DNA sequencing (supplemental online Table 1) of matched tumor and normal tissue and subsequent analysis by the UNCseq informatics pipeline and the UNC MTB between November 7, 2011, and May 12, 2015 (Fig. 1). Baseline genomic characteristics of the 1,018 patients demonstrated that the relative mutational burden of different tumor types appeared to be in keeping with the relative mutational burden previously reported across TCGA tumors (e.g., melanoma, lung, and bladder cancers had the highest mutational burden; Fig. 2A and supplemental online Table 2) [11]. A mean of 37% of protein‐altering mutations was considered actionable as defined by the UNC MTB (Fig. 2B).

Figure 1.

Study outline. Sequencing data from 1,018 cases were run through the UNCseq informatics pipeline to generate lists of variants and copy number alterations. Genomic profiles for each patient were reviewed at the UNCseq MTB and, based on the genomic alteration and its presence on the list of actionable genes previously determined by the UNCseq Clinical Committee for Genomic Research, were deemed actionable. For each patient, WfG was provided information on the type of cancer and the full list of variants and copy number alterations that had been detected by the UNCseq informatics pipeline to derive its list of actionable genomic events.

Abbreviations: MTB, molecular tumor board; UNC, University of North Carolina; WfG, Watson for Genomics.

Figure 2.

Mutational and actionable mutation burden by tumor type. (A): Median (dark line), interquartile range (box), and range (whiskers) of protein‐altering mutations by cancer type. (B): Number of actionable mutations (blue bar) as determined by the University of North Carolina Molecular Tumor Board by cancer type.

Abbreviation: GI, gastrointestinal.

We next asked WfG to perform an independent analysis of these 1,018 patients (Fig. 1). All 1,018 cases were analyzed by WfG in November 2015. For each patient, WfG was provided information on the type of cancer and the full list of variants and copy number alterations that had been detected by the UNCseq informatics pipeline (for details, see the Materials and Methods section). The WfG analysis took approximately 3 minutes per case in this study. Of these 1,018 cases, the human‐only MTB had previously requested validation of actionable variants in 703 of the patients (Fig. 3). These results were reported to the treating physician and included in the patient record in accordance with the IRB‐approved UNCseq protocol (LCCC1108/IRB #11–1115). In its independent analysis of these 1,018 patients, WfG also categorized as actionable the entirety of the human‐only MTB‐defined actionable variants in all 1,018 cases. This result shows that WfG is able to identify all reportable genetic events found by an MTB using standard practices, including a human‐curated actionable gene list.

Figure 3.

Sankey diagram of the flow of the UNCseq molecular tumor board (MTB) and WfG comparison. Of the 1,018 patients previously analyzed by the University of North Carolina (UNC) MTB, 703 were determined to have alterations in genes that met the UNC MTB definition of actionability (A) and 315 did not (B). The WfG analysis suggested that an additional eight genes not previously defined as actionable should be added to the actionable gene list. (C): Mutations in these eight genes were found in 231 and 96 patients out of the 703 and 315 patients with actionable mutations and no actionable mutations, respectively. (D): Of the eight newly identified WfG genes, seven passed the criteria for actionability as determined by the UNC CCGR. Mutations in at least one of these seven genes were found in 323 patients. (E): Re‐examination of these 323 patients revealed that while 47 had potential to change therapy, the majority of patients did not have the potential to change therapy for several reasons (no evidence of disease, n = 145; lost to follow‐up, n = 29; withdrew from study, n = 4; and deceased, n = 98).

Abbreviations: CCGR, Clinical Committee for Genomic Research; pts, patients; WfG, Watson for Genomics.

In addition to identifying reportable events already conveyed by the UNC MTB, WfG detected actionable variants in 323 (32%) of cases across a spectrum of tumor types (Fig. 4) that were not reported by the UNC MTB (Fig. 4 and supplemental online Table 3). These events reflected eight genes that were deemed actionable by WfG but not the human MTB at the time of the reanalysis (Table 1). Importantly, even at the time of the WfG analysis, these eight genes were still not on the UNC MTB list of actionable genes, suggesting that our disparate results were not merely reflecting the retrospective nature of this study. Seven of the eight genes met the UNCseq program's own definition of actionability (e.g., being part of entry criteria for a biomarker selected clinical trial; see the Materials and Methods section), but were not identified by the human‐curated actionable gene list. These seven genes were subsequently approved by the UNC Clinical Committee for Genomic Research (CCGR). This left 323 patients with WfG‐identified actionable mutations that met the UNCseq definition of actionability. The majority of newly actionable events discovered by WfG were based on recent publications or newly opened clinical trials for patients harboring inactivating events of ARID1A, FBXW7, and ATR (with or without concomitant mutations of ATM; Table 1). Of the 323 patients with newly identified events, 283 (88%) patients were made potentially eligible for enrollment in a biomarker‐selected clinical trial that had not been identified by the MTB. In particular, several of the relevant clinical trials had opened and begun enrollment within weeks of the WfG analysis (e.g., NCT02576444 for AIRD1A mutant cancers). These observations suggest that staying abreast of enrolling clinical trials is a particular challenge for human‐curated MTBs, and this problem is likely to worsen as larger numbers of genotype‐driven studies are opened nationally.

Figure 4.

Additional actionable mutations identified by WfG and not by the University of North Carolina Molecular Tumor Board (MTB), categorized by tumor type. Tumor types are plotted on the y‐axis and number of mutations are plotted on the x‐axis. The solid circles represent the mean number of mutations identified by WfG and not by the MTB for each subtype, and the whiskers show the minimum and maximum.

Abbreviations: GI, gastrointestinal; WfG, Watson for Genomics.

Table 1.

Description of actionable findings in 283 patients identified by WfG missing in MTB recommendations

Abbreviations: CCGR, UNC Clinical Committee for Genomic Research; MTB, molecular tumor board; N, no; N/A, not applicable; Ph., clinical trial phase; PMID, PubMed identifier; Rec. agent, recommended agent; Ref., reference; WfG, Watson for Genomics; Y, yes.

As the WfG analysis was performed retrospectively on selected patients, new findings were not relevant to the majority of patients analyzed. In most cases, patients were not candidates for further therapy for a variety of reasons (e.g., deceased, n = 98; no evidence of disease, n = 145). Despite the historical nature of the analysis, the MTB determined that the results of the WfG analysis were of value in 47 patients with active disease potentially needing further therapy. In these cases, the NGS results were confirmed by a standard‐of‐care, Clinical Laboratory Improvement Amendments‐approved assay in accord with the UNCseq guidelines, and then included in the patient's medical records and reported to the treating physicians. To our knowledge, no patient with a WfG‐identified potential therapy went on to be enrolled in a clinical trial.

Discussion

In this work, we employed Watson for Genomics, a cognitive computing approach with sophisticated NLP, to extract relevant information from unstructured data and to identify actionable insights in patients enrolled in a clinical sequencing program. Through ingestion of thousands of cancer‐relevant publications and analysis of ongoing clinical trials, WfG was able to identify the same actionable events reported by the human‐only MTB. In addition, WfG identified new actionable events in more than a quarter of the patients in this study. The majority of these genetic events met criteria for enrollment in clinical trials, but a minority also potentially predicted response to off‐label therapy and/or provided clinically meaningful prognostic information.

One reasonable response to this analysis is that a human MTB can be infrequently informed by a cognitive computing approach. For example, one might infer that a WfG analysis could be run once on an institution's actionable gene list and then further human‐only use would be adequate. This view, however, misunderstands the fact that knowledge in this field is rapidly evolving, with new publications and clinical trials changing the pathologic significance of a given genetic event on a near daily basis. Therefore, the actionable gene list is never “finished,” but instead requires real‐time updating, which is a task to which humans are poorly suited. Additionally, the evolving nature of a given gene's clinical utility provides an additional problem: a given patient may not have actionable events at the time of initial diagnosis, but certain mutations may later become actionable as research in the biomedical space progresses and additional clinical trials become available. Therefore, reanalysis of a tumor's somatic mutations would seem prudent using recent publication and trials information, but frequent reanalysis of multiple patients would further increase the burden on overstretched MTBs. Therefore, the ability of a cognitive tool like WfG increases the scalability of somatic tumor sequencing and subsequent analysis of mutations for precision medicine.

Cognitive computational systems have shown promise in other areas of medicine. For example, they have been successfully used to augment physicians’ ability to detect spinal lesions in patient magnetic resonance images [12]. Medical cognitive applications have also been found to be efficient for surgical phase recognition and image processing for tumor progression mappings [13]. In the arena of drug repurposing and drug target identification, pilot studies have suggested that cognitive computing can accelerate identification of novel drug targets [14]. Despite the power of cognitive computing approaches in these other areas, however, to date their penetration into genomic analysis has been limited. One reason for this has been fragmentation of medical data (i.e., the storage of data that cannot be shared in multiple different formats across disparate platforms); therefore, the large datasets required to train cognitive computing approaches have not been available. This problem is being directly addressed with several laudable data aggregation efforts in both public and private settings (e.g., the American Society of Clinical Oncology's CancerLinQ and the American Association for Cancer Research's Project Genomics Evidence Neoplasia Information Exchange). In this work, we were able to identify a large amount of data (from http://www.ncbi.nlm.nih.gov/pubmed and https://clinicaltrials.gov/, etc.) for an analysis well suited to cognitive computing providing a highly useful function for care of patients with cancer.

Conclusion

Although it is clear that cognitive computing will not substitute for the ability of trained physicians and biomedical scientists to interact with patients and to interpret clinical data, we have reached a juncture at which many of the tasks related to cancer care require a real‐time analysis of large, continuously maintained datasets. Humans are poorly suited to tasks such as a comprehensive analysis of the enormous amount of new clinical information generated on a daily basis, and the present study shows that a cognitive computing approach such as WfG can enhance care for a significant fraction of patients.

Our study suggests that cognitive computing can expand the treatment options for patients with cancer. However, the majority of the 323 patients that WfG identified as having actionable alterations were reclassified because an alteration in the gene allowed them to be considered for enrollment in a biomarker‐selected clinical trial. We realize that these genes have yet to demonstrate their capability as predictive biomarkers of response to the proposed therapy and that while patients may value the comprehensive nature of WfG, one downside of the exhaustive nature of WfG may be the presentation of too many options. Moreover, we recognize that the mere identification of actionable and targetable genomic events does not necessarily translate into patient benefit [15]. Results of the SHIVA trial question whether off‐label use of molecularly targeted agents does benefit patients [16], and this topic remains debated in the medical literature [15], [17]. Nonetheless, we remain in the early days of NGS‐guided cancer care and believe that the road to improved patient outcomes likely lies in an iterative process that needs to be instituted between MTB‐recommended therapies and patient outcomes as well as the availability of potent and effective therapies [1].

See http://www.TheOncologist.com for supplemental material available online.