NCI Workshop Report: Clinical and Computational Requirements for Correlating Imaging Phenotypes with Genomics Signatures.

The National Cancer Institute (NCI) Cancer Imaging Program organized two related workshops on June 26-27, 2013, entitled "Correlating Imaging Phenotypes with Genomics Signatures Research" and "Scalable Computational Resources as Required for Imaging-Genomics Decision Support Systems." The first workshop focused on clinical and scientific requirements, exploring our knowledge of phenotypic characteristics of cancer biological properties to determine whether the field is sufficiently advanced to correlate with imaging phenotypes that underpin genomics and clinical outcomes, and exploring new scientific methods to extract phenotypic features from medical images and relate them to genomics analyses. The second workshop focused on computational methods that explore informatics and computational requirements to extract phenotypic features from medical images and relate them to genomics analyses and improve the accessibility and speed of dissemination of existing NIH resources. These workshops linked clinical and scientific requirements of currently known phenotypic and genotypic cancer biology characteristics with imaging phenotypes that underpin genomics and clinical outcomes. The group generated a set of recommendations to NCI leadership and the research community that encourage and support development of the emerging radiogenomics research field to address short-and longer-term goals in cancer research.

Introduction

To examine opportunities and challenges in the field of radiogenomics and the allied discipline of computational bioinformatics, the NCI Cancer Imaging Program (CIP) convened two related workshops on June 26 to 27, 2013, entitled “Correlating Imaging Phenotypes with Genomics Signatures Research” and “Scalable Computational Resources as Required for Imaging-Genomics Decision Support Systems.” The first workshop focused on clinical and scientific requirements, exploring our knowledge of phenotypic characteristics of cancer biological properties to determine whether the field is sufficiently advanced to correlate with imaging phenotypes that underpin genomics and clinical outcomes, and exploring new scientific methods to extract phenotypic features from medical images and relate them to genomics analyses. The second workshop focused on computational methods that explore informatics and computational requirements to extract phenotypic features from medical images and relate them to genomics analyses and improve the accessibility and speed of dissemination of existing NIH resources such as The Cancer Genome Atlas (TCGA) and The Cancer Imaging Archive (TCIA) to enable cross-disciplinary research. A secondary goal of the workshops was to explore the importance of correlating in vivo imaging with digital pathology and the importance of including preclinical research. In this article, we outline the background and rationale for organizing this workshop, provide clinical examples that demonstrate early progress made to date, outline clinical and technical research progress to date and related research barriers, review research resources, and finally provide succinct short- and long-term recommendations to the NCI leadership and the research community to encourage and support further research in this important emerging field.

Background

Correlating specific imaging phenotypes with large-scale genomic analyses is an emerging research topic in recent literature. The research area, commonly referred to as radiogenomics or imaging-genomics, addresses novel high-throughput methods of associating radiographic imaging phenotypes with gene expression patterns as illustrated in Figure 1. Radiogenomics should not be confused with the term “radiomics,” which addresses high-throughput extraction of large amounts of image features from radiographic images. Radiogenomics has potential to impact therapy strategies by creating more deterministic and patient-specific prognostics as well as measurements of response to drug or radiation therapy. Methods for extracting imaging phenotypes to date, however, are mostly empirical in nature, and primarily based on human, albeit expert, observations. These methods have embedded human variability, and are clearly not scalable to underpin high-throughput analysis. Until recently, prognosis and therapeutic decisions that distinguish between the varieties of cancers were generally based on distinctions inferred by consolidating clinical records of patient groups who share a common cancerous organ of origin (e.g., lung, breast, renal, prostate, etc.). The likely aggressiveness of the cancer (and prognosis) was usually only assessed by laboratory microscopy, as well as staged at the time of discovery. Recent subcellular genomic and molecular biophysical discoveries now offer numerous plausible alternatives to this dominant organ-specific cancer model. Similarly, conventional in vivo anatomic imaging has long been used to access efficacy of response to chemotherapy or radiation therapy for various cancers, based primarily on gross quantitative measurements of changes in tumor size or extracted texture features. These approaches to date have limitations for predicting recurrence and effective treatment response. With emerging functional and molecular imaging methods, such as combining positron emission tomography (PET) with computed tomography (CT), or use of dynamic contrast-enhanced (DCE) or diffusion-weighted magnetic resonance imaging, a potentially more accurate assessment of response to therapy at the cellular level is to assess the in situ tumor’s metabolic and proliferative activity. While these functional and molecular imaging approaches are already an improvement over conventional imaging methods [1], their integration with -omics information can be a powerful strategy, potentially enabling clinical decision tools for improving diagnostic accuracy and patient care. Radiogenomics represents a synergy derived from data integration by these complementary biomedical assessment tools. Effectively addressing the field of radiogenomics requires implementation of advanced functional and molecular imaging methods as well as new approaches to robust feature extraction, data integration, and scalable computational strategies to implement clinical decision support.

Figure 1

A logic diagram of an example of the field of radiogenomics for breast cancer using digital mammography and DCE MRI.

The cancer research community faces a plethora of conundrums, such as tumor cellular heterogeneity, both within the primary tumor and among its metastases; disease signatures that are more complex than a single pathway; stem cell-driven tumor evolution; and immune system tumor interactions. Impacts of these biological factors are not fully understood, and are more likely entwined with cancer progression, metastasis, resistance to therapy, and recurrence. To address these emerging complexities, new cross-disciplinary research approaches and teams are required, encompassing a wide range of research domains that should include genomics, epigenomics, biostatistics, and informatics as applied to pathology and clinical and preclinical imaging.

Extraction of spatial and temporal features from images, including the use of modeling methods, is required for correlation of imaging phenotypes with genomic signatures. In correlating imaging and omics data, the large dimensionality of omics datasets potentially poses significant challenge in integration with imaging data that are typically of much smaller dimensionality. Mathematical approaches for dimension reduction and the validation of these approaches using clinical data are urgently needed in order to integrate these disparate datasets [2], [3], [4].

Methods for feature extraction should ideally be independent of the different data collection platform(s), data collection sites, and method of analysis, which may include image acquisition and analysis protocols, unrestricted collection of and access to image data, harmonization of data collection, and analysis across clinical sites and different commercial imaging platforms, including the formalization of structured reporting and uniform semantics. However, these requirements may not always be needed as several research sites are making progress with standard of care images. These themes were recently addressed by NCI, CIP [5], and later by the professional imaging societies (Radiological Society of North America, American College of Radiology Imaging Network, Society of Nuclear Medicine, and the American Association of Physicists in Medicine) [6]. Common approaches to defining strategies for broad adoption of imaging standards in therapy treatment trials are currently in progress. Integrating image phenotypes and genomic signatures into clinical decision-making, however, will require a significant extension of these quantitative imaging strategies. Similarly, there is a critical need to scale up the computational methods required for clinical decision-making using high-throughput analysis that may require scalable cloud computing strategies.

Developing and implementing clinical decision support systems requires access to data collected from all the above research domains in order to optimize their performance and test their reliability in the clinical or preclinical setting. NCI has initiated this effort through TCIA [5], and designed it to be compatible and interoperable with NIH TCGA [7]. The primary goal of creating this research resource was to improve its accessibility and enable cross-disciplinary research in both of these research domains, supporting initial efforts to correlate imaging phenotypes with genomics signatures. The TCIA-TCGA interface currently meets personal health information de-identification and data inter-operability requirements, while preserving the means to support diverse research projects. However, this research resource is unable to meet future requirements for radiogenomics research, such as supporting very large, statistically tractable, diverse datasets that are much broader and more inclusive than have been conceived to date in the cancer imaging community. Finally, there is a similar need to develop the open-access software tools required to evaluate clinical decision support systems. NCI is currently exploring NCIP HUB (HUBzero) as a tool-sharing resource for the above research domains. These additional requirements, however, will need significantly more investment by NCI or NIH, and success will greatly depend on the research community’s willingness to share data and related software tools and success in reaching a consensus on standardized methodology for the rapidly emerging field of radiogenomics.

Clinical Examples

Brain: Glioblastoma (GBM)

Genomic differences discovered between patients with GBMs, which are known to have uniformly poor survival times, might be better understood by simply knowing the tumor extent and the hemisphere of involvement, for example, by MRI at the time of first diagnosis. Genomically equivalent GBMs might differ in their overall survival (OS) time if their location and extent at presentation occurs differently in neurologically silent brain areas, or in the extent of peritumoral edema. TCGA researchers have cataloged recurrent genomic abnormalities in GBMs and in lower grade gliomas. As a parallel effort, NCI, CIP is retrospectively obtaining imaging data for TCGA patients and making it available via TCIA [5]. These programs provide easy access to genomic and imaging data collected from multiple institutions, and have resulted in supporting initial research work on GBMs. Three case studies are briefly reviewed below as examples.

For the first case, methodologies and tools were developed to investigate conventional and advanced neuroimaging-based biomarkers for predicting OS and molecular signatures using TCGA GBM data [8]. Presurgical MRIs of 75 GBM patients were downloaded from TCIA and independently reviewed by three neuroradiologists for 27 features that assessed size, location, and tumor morphology as illustrated in Figure 2. The results demonstrated the presence of contrast enhancement (CE) on post-gadolinium MRIs (> 33%), a significant and independent predictor of poor survival. Associations between genetic alterations revealed that epidermal growth factor receptor (EGFR) mutant GBMs were significantly larger on T2-weighted and fluid attenuated inversion recovery (FLAIR) images than wild-type EGFR GBMs; TP53 mutant GBMs were smaller than wild-type GBMs; and EGFR mutant GBMs were significantly larger than TP53 mutants.

Figure 2

Radiological and histological feature extraction and correlation to genomic data. Following identification of the contrast enhancing region (top-left) or cancer nuclei (right) various imaging features can be extracted. Using corresponding genomic data such as DNA expression, mutations, and methylation, these features can be correlated with molecular data and underlying biological characteristics of tumorigenesis can be identified.

The second case involved the use of perfusion parameters. For example, Jain [9] reported provocative results demonstrating a genomic basis for the commonly employed quantifiable perfusion parameters and gave impetus to implement this added knowledge into clinical practice. Integrating these quantitative perfusion parameters with the genomic markers in GBMs generated better prognostic models than either imaging or genomics could provide alone [10]. More recently, his group demonstrated that combining clinical, imaging, and genomic markers could also provide important and unique prognostic information about the poorly understood non-enhancing tumor regions in GBMs [11]. The results, illustrated in Figure 3, demonstrated tumor infiltration beyond the contrast enhancing component and increased regional cerebral blood volume (rCBV) within the non-enhancing component. Graphs of survival estimates demonstrated that rCBVNEL (CBV of the non-enhancing component) is a significant predictor of OS (log-rank test, P= .0103) and progression-free survival (log-rank test, P= .0223), which also showed an association with wild-type EGFR mutation.

Figure 3

MR images fused with CBV map showing a contrast-enhancing GBM (red ROI) with central necrosis in the medial left parieto-occipital lobe in a 56-years old male. Surrounding non-enhancing component of the tumor is outlined by multiple ROIs showing different grades of increased CBV, suggesting tumor infiltration beyond the contrast enhancing component.

The third case involved building gene expression-based models to predict quantitative microscopic disease phenotypes. The potential advantage of using microscopic disease phenotypes (rather than patient survival) to supervise identification of biologically meaningful expression signatures is the presence of multiple phenotypic targets per patient. For example, Brat et al. have used TCGA molecular data together with MR images within TCIA and whole slide pathology images to investigate molecular correlates of morphology in GBMs [12]. To streamline glioma morphology-omics investigations using whole slide pathology images, they developed an end-to-end image analysis and data integration pipeline [13], [14], [15] and developed morphologic “signatures” from hundreds of millions of cells in digitized whole slide images. Using digitized images from TCGA GBM collection, three prognostically significant patient clusters were found based on biological functions of associated genes: cell cycle, chromatin modification, and protein biosynthesis clusters, as illustrated in Figure 4. Several cancer-related pathways were differentially enriched among the morphology clusters, including the ATM and TP53 DNA damage checkpoints, the NF-κB pathway, and the Wnt signaling and PTEN-AKT pathways. This analysis demonstrated the potential of high-throughput morphometrics to develop sub-classifications of the disease.

Figure 4

Integrated morphologic analysis for the identification and characterization of disease subtypes. Approximately 500 whole slide images representing 162 glioblastoma cases were analyzed and cellular features were extracted and analyzed. Clustering results identified three morphologically driven subtypes which differed from each other based on prognosis, pathology, genetics, methylation, and TCGA subtype associations (for example, neural subtypes were particularly enriched in the chromatin modification subtype).

Breast Cancer

Discovery of therapeutic effectiveness among tumor-type subpopulations for breast cancer has been masked by the presumption of cancer-type uniformity. A few adaptive clinical trial designs are now in progress that link quantitative imaging with the -omic profiling of patients (e.g., Investigation of Serial Studies to Predict Your Therapeutic Response With Imaging and Molecular Analysis, I-SPY 2 TRIAL [16] and ALCHEMIST [17]). Data from the I-SPY 2 trial has permitted computer analyses of imaged lesions that can potentially be related to molecular classifications in breast cancer (e.g., estrogen receptor [ER] status, HER2 status, and progestin receptor status). For example, computer‐extracted features of the tumor potentially can be used to assess tumor aggressiveness.

In the pilot study shown in Figure 5, lesion features were automatically extracted from DCE breast MRI images (obtained with 1.5 T and 3 T scanners) and analyzed on their own as well as merged into lesion signatures to assess molecular classification. Results shown in Figure 5, Figure 6 demonstrated that individual lesion features were only weak classifiers, as evidenced by the modest areas under the receiver operating characteristic curve (AUC value), but when artificial intelligence was used to merge the features into lesion signatures, performance substantially improved (last four data points in plot below). Giger et al. have been developing and investigating computerized quantitative methods for extracting data from multi‐modality breast images and mining the data to yield image‐based phenotypes relating to breast cancer risk, diagnosis, prognosis, and response to therapy [18], [19], [20].

Figure 5

Relationship of MRI-based phenotypes in distinguishing breast cancer subtypes (big data analyses). Performance is given in terms of AUC (y-axis) for various MRI-based Phenotypes (x-axis) as well as a Tumor Signature formulated from merging selected MRI-based phenotypes using linear discriminant analysis. Features 1 to 11 are kinetic features, 12 to 25 are texture features, 26 to 30 are morphological features, 31 to 34 are size features, and the far right data point refers to the tumor signature from the merged features.

Figure 6

Image-based phenotype array of computer-extracted characteristics of breast cancer tumors on MRI for ER − and ER + tumors from the TCGA/TCIA breast cancer dataset. Bottom row gives values and the corresponding AUC (area under the receiver operating characteristic curve) of the image-based signature in the task of distinguishing between ER − and ER + tumors.

Renal Cancer

Currently, the primary role of imaging in the management of renal cell carcinoma (RCC) consists of tumor detection, staging, and gauging response to treatment. Although numerous modalities can be employed to image RCC, multi-detector CT (MDCT) is most commonly used [21], [22] because of its speed, high spatial resolution, sensitivity to contrast enhancement, and ability to provide a global multi-planar view of the abdomen. However, while MDCT has achieved success for detection of RCC and accurate anatomic staging, continued reliance on this technique alone will likely prove inadequate in the future. Over the past decade, several studies have attempted to further characterize RCC, focusing mainly on enhancement characteristics of the tumor [23], [24], as illustrated in Figure 7. A few interesting studies correlated imaging features of RCCs with chromosomal changes. Karlo et al. [25], [26] found significant associations between gene mutations and phenotypic characteristics of clear cell RCC by contrast-enhanced MDCT. RCC radiogenomics, however, can only contribute new insights if clear associations between imaging characteristics and molecular aberrations of the tumors are determined.

Figure 7

Enhancement analysis of renal cell carcinoma. Axial multidetector CT images obtained during the A) noncontrast, B) corticomedullary, C) nephrographic and D) excretory phases show regions of interest (red circles) drawn within a renal cell carcinoma of the right kidney. Attenuation values measured by the regions of interest are used to investigate the enhancement characteristics of the tumor during dynamic imaging.

All of the above clinical examples posed one or more imaging protocol limitations. For example, clinical protocols should be optimized and harmonized across imaging platforms, clinical data collection sites, and organ-site specific methods of analysis. The development of consensus taxonomy will be required to coordinate meaningful future research results. Furthermore, specific features to be addressed include establishing definitions to quantify necrosis, criteria for tumor margin assessment, and quantifying the degree of enhancement and neovascularity. Once the key imaging features are clearly defined, the inter-observer variability for future radiogenomics research will need to be reduced and structured reporting will be required to achieve reporting stability and consistency necessary for large-scale clinical studies. Theses biological and technical limitations are discussed further below.

Research Barriers

Tumor Inhomogeneity

Increasing evidence supports the impact of intra-tumor genetic heterogeneity on the metastatic ability of tumors and their resistance to therapeutic interventions. Genetic intra-tumoral heterogeneity may contribute to treatment failure by initiating phenotypic diversity that introduces tumor sampling bias and enables drug resistance to emerge [27], [28], [29]. Recent massively parallel sequencing studies and epigenetic analysis of different tumor types have revealed that cancers are composed of mosaics of non-modal clones [30], [31] which harbor distinct constellations of genomic alterations in addition to the founder genetic events, and that clonal selection occurs during metastatic progression [32], [33]. Intra-tumor genetic heterogeneity, for example, may be present in high-grade serous ovarian cancer (HGSOC) [27], [28], [34], [35], [36], resulting in incomplete response to chemotherapy [34]. Using phylogenic tree analysis to evaluate relationships between tumor deposits in patients with ovarian cancer, Cowin et al. [34] found substantial copy number differences between metastatic deposits within individual patients and identified signaling pathways plausibly linked to peritoneal dissemination and establishment of metastatic foci. Significantly greater genomic change was observed in patients who experienced relapse after responding to chemotherapy than in patients who were resistant from the outset, possibly reflecting the requirement for selection of a subpopulation of resistant cells in cases initially sensitive to treatment [34]. Incorporating multiregional tumor analysis of both primary and metastatic disease into the development of new targeted therapies and validation of biomarkers of therapeutic response is therefore crucial; image-informed multiregional tumor analysis may be required to fully characterize tumor heterogeneity.

Intra-tumor functional heterogeneity is often manifested by intermingled vascular compartments with distinct pharmacokinetic properties. DCE imaging provides a noninvasive method to evaluate tumor vasculature or metabolism rate based on contrast accumulation and washout. However, intra-tumor functional heterogeneity cannot be resolved directly by most in vivo dynamic imaging methods due to intermingled cellular subpopulations and limited imaging resolution. This inability to distinguish different cell/tissue types with tracer signals can confound compartment modeling and deep phenotyping for association studies [37], [38], [39].

An important step in developing such a characterization is to determine the tumor “cytotype”, defined as the identity, quantity, and location of the different cells that make up a tumor and its microenvironment, by careful microscopic identification [40], [41], [42]. Specific probes defining subtypes of tumor cells or stroma need to be established and verified. Molecular imaging using radionuclide probes have been employed that promises to detect specific tumor or stromal cell targets. It is crucial to carefully consider what types of tumors will be best suited for such studies and what tumor sampling strategy should be used. Imaging methods that identify different types of tumor architectures promise to improve all types of cancer diagnoses and treatment [43], [44]. Therefore, development of more sophisticated imaging methods to characterize this multi-cellular structure and how the microenvironment influences tumor behavior is urgently needed. An example of this is shown in Figure 8, which shows diagnostic CT scans from two patients with non-small cell lung cancer (NSCLC). The bottom panels show the same images plotted as the gradient of attenuation in Hounsfield units per cm. The patient on the left with the more heterogeneous tumor died seven months after surgery, and the patient on the right is still alive more than 30 months post-surgery.

Figure 8

Diagnostic CT scans from two patients with non-small cell lung cancer, and gradient maps of image intensity.

Cancer cells can evolve to adapt to therapy, leading to therapeutic failure. Such adaptations not only cause heterogeneity, but also create consequences ranging in scale from single-cell genetic mutations to large feature variations. Even within a single tumor, marked variations in imaging features such as necrosis or contrast enhancement are common. Radiologic heterogeneity is usually governed by blood flow, though genetic heterogeneity is typically ascribed to random mutations. This tumor evolution is marked by environmental selection forces and cell phenotype (not genotype) [45]. An alternative means to describe intra-tumoral heterogeneity is through creation of “habitat maps”, wherein images containing orthogonal information are combined to identify regional differences. An example is the combination of CE MRI, a measure of blood flow and perfusion, with diffusion MRI, a measure of cell density. These individual images can be separated into low- and high-enhancing regions using fuzzy clustering or Otsu thresholding. Combining the images can yield four different “habitats,” as illustrated in Figure 9. In addition to imaging approaches, tracking mutations in cell free DNA [46] provides complementary information in understanding the cancer cell evolution process.

Figure 9

Example of a contrast enhanced T1 map and a diffusion ADC map clustered into high (red) and low (green) values using Otsu thresholding. These can be combined to yield regions with low ADC (high cell density) and low T1 (low contrast enhancement) in violet; and regions with high ADC (edema) and low T1 in green.

Opportunities for researching intra-tumoral heterogeneity would benefit from more image-informed regional tumor tissue genetic/expression mapping. For instance, expanding imaging genomics into the analysis of gliomas could focus on the intra-tumoral heterogeneity in high- and low-grade lesions. Correlation of quantitative imaging parameters with locus-specific gene expression will help identify not just a genomic basis for specific imaging phenotypes, but pave the way to monitor any phenotypic changes occurring during the treatment/observation phase with serial imaging, using imaging as surrogate markers, as surveillance tools.

Tumor heterogeneity is multidimensional. For example, within a tumor, there can be genetic and epigenetic heterogeneity; differences in microenvironments; phenotype differences; heterogeneity arising over time; and heterogeneity between primary tumor and metastases. Imaging phenotype can be characterized by one or more spatially registered imaging modalities (e.g., CT, PET, molecular imaging, MR, and ultrasound). Imaging is the only technique that can characterize the whole tumor as well as any pertinent surrounding tissues; it is non-invasive and can be repeated over time (assuming issues of radiation dose, where applicable, are addressed). Specific attention should be paid to “serial imaging,” to understand molecular mechanisms behind treatment success/failure and changes in spatial/temporal/habitats that accompany treatment, and to observe tumor evolution over time (e.g., resistance development). Image analysis methods to predict and detect the emergence of resistance, correlate with genomic heterogeneity, and identify homogeneous subtypes within a heterogeneous tumor would be invaluable.

Within the context of tumor heterogeneity, microscopic images represent an extremely valuable resource of disease phenotype data. Visual analysis of microscopic images is considered the gold standard diagnostic modality for virtually all cancer types [47], [48]. Importantly, a large amount of cell type-specific and tissue region-specific biomedical knowledge encoded in morphological data is not directly recoverable from -omics data, which requires destroying tissue structure prior to extraction of molecular analytes and molecular profiling. This suggests that there may be value in integrating molecular and morphological phenotype data to take advantage of the unique strengths of each data type (depicted in Figure 10).

Figure 10

Glioblastoma morphology pipeline: (A) Image analysis algorithms segment cell nuclei and calculate a feature set for each nucleus describing its shape and texture. Nuclear features are aggregated over the hundreds of millions of cells in whole-slide images to produce a morphometric profile for each patient. (B) Analysis of morphometric profiles reveals clusters of patients with cohesive morphologic characteristics. (C) The correlates of morphologic patient clusters are identified through deeper analysis of clinical and genomic data.

Similarly, within the context of tumor heterogeneity, image-guided (IG) semi-automated needle core biopsy methods will prove to be very important. These IG methods, capable of extracting 30 + mg tumor tissue samples suitable for micro-fluidic -omic analysis, are now available, but have not yet been widely deployed. Such targeted tumor sampling, coupled with increased fresh frozen biospecimens pioneered by TCGA, could extend the reliability of -omic sampling and analysis procedures. Many individual comprehensive cancer centers are currently engaged in this type of biospecimen harvesting but further standardization is required. For example, in RCC, the use of IG biopsy, has allowed determination of imaging phenotypes with clinical relevance, since it has been shown that the clear cell variant is often subject to intra-tumoral genomic heterogeneity [30]. Use of IG biopsy coupled with deformable image registration should permit improved longitudinal sampling [12].

All of the above work could have significant clinical implications, not just for identifying a more effective therapeutic drug target, but also for monitoring treatment response. Identifying molecular targets with specific imaging markers should lead to development of better chemotherapeutic agents with less toxicity. Early detection of a favorable response or failure of a treatment regimen using combined imaging and genomic markers could potentially help expedite drug approval, generating substantial cost savings for clinical trials.

Preclinical Imaging

Mouse and human-in-mouse models of malignancies (e.g., patient-derived xenografts, transgenic) are routinely used for drug efficacy and toxicity testing [49], [50]. The mouse model research strategies prove to be promising for understanding biological factors in prediction and response to therapy, as direct access to tissues during longitudinal studies is possible. In addition, a growing body of evidence shows that reliable preclinical data can be merged with patient data and used to determine what therapy may be used to treat specific malignancies [51]. This newer approach to integrated cross-species testing, termed co-clinical trials, involves concurrent assessment of novel drug combinations in mouse and human-in-mouse models of tumors, and in patients with recurrent or metastatic disease with whom the mice are genotypically matched [52], [53]. Recent published literature demonstrates that well-documented, integrated cross-species approaches are of value for clinical decision making [54]. Radiogenomics will clearly play an important role in co-clinical trial studies where imaging phenotypes will be correlated with genomics signatures. A powerful component of both pre- and co-clinical testing is the use of various in vivo imaging modalities that either mirror medical imaging practices or provide additional biological information [52], [53], [55]. Imaging is a key to success in co-clinical investigations, providing real-time monitoring of the animal subjects for response, disease progression, recurrence, or metastasis, and ready access to longitudinal tissue samples for genomic analysis using image guidance. The evolving pre- and co-clinical approaches require development and incorporation of data and semantic standards to ensure reliability of interpretation and use of research resources such as data archiving and the implementation of quality improvement methods as reviewed later.

Computational Methods

It is becoming increasingly clear that molecular changes in gene expression elicit structural and vascular changes in cancer imaging phenotypes that are in turn observable indirectly by various imaging modalities across different spatial resolution scales. For example, for the above clinical examples, these observations were evident in anatomical, molecular, and/or functional imaging methods in vivo. In addition, tumor morphology in standard H&E stained tissue specimens may reflect the sum of all molecular pathways in tumor cells. It is therefore possible to postulate that by extracting quantitative disease-specific information across different scales of image data, different imaging phenotypes can be identified via association for different organ sites.

To exploit this potential, efforts have already been directed to using data presented in TCGA and TCIA. The information-rich content of both multiplex -omics platform assay datasets and modern digital images along with the accompanying complexity of metadata and annotations, however, poses new challenges for computational methods. Thus, increasingly sophisticated computational methods and archival storage capabilities to make the data accessible and interpretable for the desired clinical context is necessary. A wide range of new computational methods are available for image analysis methods and data integration strategies in the published computer science and image processing literature, which will not be reviewed here in the interest of space [56]. They include texture analysis methods, multi-resolution feature extraction methods such as wavelets, feature reduction methods, a range of statistical classifiers including semi-supervised and unsupervised clustering methods with the ability to differentiate tissues within the tumor bed, and modeling methods that address tumor heterogeneity. Finally, a number of statistical methods for performance assessment of these methods have been reported.

Perhaps the more important barrier to implementation of advanced computational image analysis methods is the critical need for annotated data across different resolution scales, as required to optimize and validate the performance of these different software tools and final clinical decision support systems. While image or molecular datasets are widely available (e.g., TCGA, TCIA, and other database resources [57], [58], [59], [60], [61]), only a few of these datasets exist in a structured, deeply annotated form. For example, while the shape of breast lesions in image scan help distinguish between benign and malignant lesions, to quantitatively assess lesion shape and type (e.g. via angularity or spicularity), segmentation of the lesion boundary is required. Progressing to using a wider range of features, including features extracted across different modalities, will clearly require a much higher level of deep annotation across different resolution scales invariably absent in most publicly available datasets. A further complication is that annotation is intrinsically specific to the scale of data being interrogated. For instance, digitized histopathology images (typically several gigabytes in size each) can be annotated at multiple scales depending on the specific problem to be addressed (nuclear segmentation or tumor detection). Thus, software tools for annotation, often referred to as metrology tools [62], are required as opposed to observer annotation measurements that are not scalable and impractical. To maximally extract value from these large diverse datasets (often referred to as BIG DATA), unstructured representations need to be annotated across different levels of detail, as illustrated in Figure 11.

Figure 11

Multi-scale deep annotation and population-based atlas: by employing advanced segmentation on MRI we can segment (A) prostate (yellow) on MRI. However, to “see” more on routine MRI we need to rely on coregistering MRI with (d, f) pathology to appreciate the (B) central gland (purple) and peripheral zones (cyan), anatomic landmarks such as the ejaculatory ducts (green), urethra (blue), and neurovascular bundles (yellow) and (C) cancer extent (red). Immuno-histochemically stained pathology (D, F) can inform on cancer aggressiveness; population-based prostate atlas (H) shows the 3D distribution of cancer relative to the prostate anatomic regions.

Multi-scale data enrichment refers to the process of identifying at a particular scale features that become obvious or discoverable only when the data is viewed in conjunction with corresponding representations at finer, more granular size scales. A large body of empirical and theoretical studies has confirmed that the intelligent combination of multiple, independent sources of data can provide more predictive power than any single source. For example, Madabhushi et al. have shown that an upstream classifier combining imaging and molecular features allows for improved prediction of high risk prostate cancer patients, as shown in Figure 12[63]. Additionally, the Madabhushi group showed that the combination of histologic images and proteomic features could allow for improved prediction of five-year biochemical recurrence in prostate cancer patients following radical prostatectomy (see survival curves in Figure 13). Finally, multi-scale deep annotation tools will allow for generation of highly curated, “ground truth” datasets, facilitating training and evaluation of different classes of analytic methods (image, signal analysis and bioinformatics), and for building and evaluating fused classifiers for disease characterization. The same annotation strategies will also allow for creation of multi-scale disease ontologies that incorporate quantitative disease attributes ranging from the imaging to the electrophysiological and cellular level, down to molecular-length scales.

Figure 12

Computer extracted MRI markers of aggressive prostate cancer. (A) DCE-MRI, corresponding (B) CD31 (vascular) stained slice with PCa annotations (red), (C) histology-MRI registration, (D) DCE-MRI feature map, (E) microvessel architecture used for histologic feature extraction, (F) correlation heat map of histology and DCE MRI features, (G) imaging biomarkers identified in (F) allowed for separation of Gleason grade 3 from 4 tumors, with an AUC = 0.92.

Figure 13

Upstream fusion of Big Data streams may improve prediction of 5 year PSA failure in prostate cancer patients following surgery. Panels (a)–(c) show survival curves for distinguishing men with (red) without 5 year PSA recurrence (blue) via (A) histologic image features from excised specimens, (B) proteomics from mass spectrometry from dominant nodule on the excised specimen, and (C) combination of histologic image and proteomic features.

Imaging Standards

The correlation of imaging phenotypes with genomics signatures may require the implementation of imaging standards as outlined in the background section. The degree to which imaging standards are required will depend greatly on the data collection strategy. For example, if the intent is to collect large data sets using standard of care studies to validate and implement clinical decision support systems, the requirements for data collection harmonization would need to be relaxed. However, the use of standardized methods for data analysis, feature extraction, and data integration will be important in order to reduce the measurement uncertainty for data analysis across different clinical or research sites. Similar imaging standards for data collection and analysis hold for preclinical imaging where the problem is compounded by a more diverse range of preclinical imaging platforms, and the use of imaging methods such as optical imaging where Digital Imaging and Communications in Medicine (DICOM) standards are not yet fully developed [52], [53], [54]. The requirements for quantitative imaging, particularly as applied to predicting and/or measuring response to therapy, are extensively covered in a special issue of this journal and will not be addressed in this report due to space limitations [6], [64], [65].

Research Resources—Database Archives

Databases linking imaging with molecular data are just beginning to emerge at a slow pace due to the high cost of large-scale imaging studies and lack of standards for interpretation. To conduct meaningful imaging genomic correlation studies, big scale (Big-N) imaging studies will be needed, which will require data acquisition, aggregation, management, and analysis methodologies, as well as technologies quite different from those used in research today. Achieving such large-scale aggregation will require new incentive structures, computing infrastructure, security policies, and analysis methods. In addition to the NIH supported TCGA-TCIA data archive, there are three other examples of note for platforms being built for the purpose of integrating disparate data. They include (a) the Information Sciences in Imaging at Stanford (ISIS) group, (b) the I-SPY TRIAL, and (c) the Georgetown Database of Cancer (G-DOC). ISIS is developing several tools to collect and integrate annotated imaging, clinical, and molecular data through novel computational models that help identify relationships within the data [66]. The I-SPY TRIAL breast cancer data collection was a collaboration of ACRIN, Cancer and Leukemia Group B (CALGB), and NCI’s Specialized Programs of Research Excellence (SPORE). The study aimed to identify molecular markers of response to conventional neoadjuvant chemotherapy and imaging markers associated with response to therapy [67], posing new challenges for data archiving. G-DOC, developed at Georgetown University, deals with five types of -omics data integrated with clinical metadata and patient outcome data. It offers a model for how to store, integrate, and visualize multiple disparate data types. A major challenge in analyzing the potentially enormous datasets, however, is to design them to be useful for the end user—the translational researcher who is either developing clinical decision support systems or implementing these methods into clinical trials. The generation and computer visualization of reports from such data-integrating platforms are critically needed to reduce the multi-dimensional data into graphical representations that can be more readily interpreted. Thus, it is clear that more consensus approaches are potentially needed to develop interoperable web-based data archives using common standards that are initially being promoted by the NCI-funded TCIA-TCGA database.

Research Resources—Cloud Computing

Cloud-based computing and resources present new opportunities for supporting imaging and genomics correlation research. Scalability of cloud-based resources for storage, sharing, and analysis of research data enhances computing power and tools for individual researchers whose proprietary resources may be much more limited. Enhancement of community resources on a large scale should provide a major incentive and increased ability to accomplish the full integration of genomics capabilities into research programs. Cloud computing can provide novel opportunities for a collaborative environment that fosters re-use of data and community-driven creation of tools and analytics.

Industry Role

Technology companies could play multiple roles in supporting an imaging-genomics correlation initiative, from implementation vendors to marketplace contributors and facilitators, and ultimately as community stakeholders. They can contribute by providing input and feedback that help to shape technical standards in their development and implementation. Technology companies need recognition as key stakeholders in this new model, since they are the source of continued innovation, ongoing technical expertise, and professional networks for furthering the ecosystem.

Collaboration with industry under public-private partnerships could help to ensure industry participation. The NIH’s Biomarkers Consortium is a public-private partnerships that has successfully benefited the federal government as well as industry, helping to accelerate new biomarkers for discovery, and ultimately for marketed therapies and drugs. Creation of an interactive community that enables collaboration through the cloud computing environment and utilization of other crowd sourcing technologies will help to develop innovative solutions. Community-driven tool development can be enabled with the provision of a software development kit. User-provided analytics can be vetted by the community. Crowd sourcing challenges can be issued to solve especially intractable problems for analytics, display, or data integration.

Opportunities and Overall Recommendations

The recommendations for this new field of radiogenomics was developed by workshop attendees, who have very diverse experiences in fields of imaging sciences, genomics, molecular biology, bioinformatics, computer science, and industry. The recommendations address both short-and long-term requirements where appropriate to advance the field of radiogenomics specifically in predicting and/or measuring response to therapy. Four breakout groups were formed: (a) clinical opportunities, (b) scientific opportunities, (c) computational methodology opportunities, and finally (d) research resource opportunities. Breakout group reports are listed below.

Clinical Opportunities

Short-term Clinical Recommendations:

Define what we mean by imaging-genomics (and all the other similar terms):

Identify the gene mutations in a tumor

Search terms to mine unseen data in the images

Need to clinically establish:

Is imaging heterogeneity reflective of genetic instability?

Is genetic analysis of tumors relevant in the absence of heterogeneity analysis?

Improve existing biomarker signature panels by adding imaging features:

For instance, tumor image size and texture added to Oncotype DX

Heterogeneity is easily seen in images generated in clinical trials; however, we need:

Imaging-directed multiple biopsies for heterogeneity and correlation—co-register spatially

Genetics of the entire tumor, not just the small biopsy sample as currently practiced.

Address highly targeted studies by evaluating image features across tumor types:

Specifically with same mutations—e.g., RAS or EGFR to find commonalities

Use the databases we have now for retrospective studies, despite obvious inadequacies, as lessons learned to help with future prospective studies

Long-term Clinical Recommendations:

Replace repeated biopsies with validated imaging approaches using feature extraction methods

Optimize imaging features/image analyses and clinical decision support systems independently to do:

What humans observers do best

What computers do best

Back up imaging predictive elements from trial data with targeted therapy

Natural language processing to scribe physician interpretations

The performance of clinical decision support systems should be:

Independent of data collection and analysis platforms and clinical sites, and

Ideally operator independent as required for their clinical adoption

Scientific Opportunities

Short-term

Tumor heterogeneity: Need to establish multiple definitions:

Genetic, epigenetic heterogeneity within tumor

Differences in microenvironments within tumor

Phenome differences within tumor

Heterogeneity involving primary tumor and metastases

Tumor heterogeneity: Characterization across research domains:

Imaging phenotype (radiology, pathology, optical) and molecular phenotype

Spatially characterized molecular phenotype (laser-captured micro-dissection, imaging mass spectroscopy, molecular imaging)

Long-term

Develop imaging and analysis methods to characterize heterogeneity:

Within a tumor at one time point

Evolution over time

Among different tumor types

Develop imaging metrics that can:

Predict and detect emergence of resistance

Correlate with genomic heterogeneity

Correlate with habitat heterogeneity

Identify more homogeneous sub-types

Serial imaging (longitudinal studies):

To understand molecular mechanisms behind treatment success/failure

To understand changes (spatial/temporal/habitats) behind treatment success/failure

To observe tumor evolution over time, e.g., during of therapy

Multi-scale characterization, multimodality registration

In vivo (radiological, optical, microscopic):

Ex vivo (traditional pathology specimens, microscopy used to generate 3-D reconstructions, spatially mapped molecular studies)

Computational methods to support multi-scale tissue characterization

Computational methods to support multi-modality registration

Analytical methods to characterize time dependent changes in large datasets

Management, query of large, complex spatiotemporal datasets

Computational Methodology Opportunities

Short-term

Large annotated datasets with clinical outcome using existing retrospective data:

Matching of reports from radiology and pathology to drive the annotations

Creating semi or fully automated algorithms for annotation. Semi-supervised learning approaches for curating training sets which might be inherently noisy

Prospective data generation wherein the different clinical disciplines are engaged to create the annotated datasets

Long-term

New image analytics and mathematics of predictive modeling:

Creating a catalog of image based descriptors, similar to the MPEG 7 effort

Supporting fundamental developmental work in new mathematics for defining new image features using an array of imaging modalities

Quantifying “error” of the analytics, defining ground truth for evaluation:

Leverage experience from the field of computer assisted detection/classification

Distinguishing the evaluation of the different analytics—segmentation, feature extraction, classification, etc.

Better annotation tools for facilitating generation of surrogate ground truth

Annotation mechanisms to achieve consensus amongst experts, including active learning based annotation

Hierarchy of data annotations in order to leverage differently sophisticated experts (mixture of experts)

Educated/trained crowd sourcing for annotations

Compare algorithms and prepare them for high performance computing:

Need for organized support to migrate algorithms into an optimized format for supporting high-performance computing; i.e., see US Dept. of Energy SciDAC software centers for examples

Infrastructure for comparing algorithms

Data collections, algorithm collections, mechanisms for adding algorithms and testing against others

Meteorology tool sharing to permit objective performance comparisons

Mechanism for being able to compute on the data—send the analytics to the datacenter to compute remotely such as cloud computing:

Examples: NCI TCIA and NCIP HUB (HUBzero)

Data-sharing Research Resource Opportunities

Short-term

Support and expand data collection using NCI TCIA-TCGA

Explore how to develop a means for metrology tool sharing such as the NCIP HUB (HUBzero)

Explore the interest of cancer centers to share data and metrology tools

Explore the interest in data sharing with NCI National Clinical Trial Network or other funded clinical trials (retrospective data with clinical outcome)

Leverage the technical resources being developed by NCI’s Quantitative Imaging Network

Cooperative groups and VA hospitals—EMR, genomics being acquired from each VA patient (ongoing)—could be a model.

Suggested priority list for data collections: Core set + additional pre-therapy a must (post-therapy and follow-up needed:

Diagnostic/therapeutic populations; target is 1000 patients to develop decision support system core set +

Lung—CT; PET/CT (dual reconstruction at thin and radiology slice)

Breast MRI—DCE/T2/STIR (DWI)

Prostate—endorectal (> 1.5 T) or body (3.0 T) T1, DCE, T2, DWI (ADC)

GBM MRI—DCE/T2/FLAIR (DWI, MRS), immediate post-operative images valuable

Ovarian—CE CT

HNCC—PET/CT

RCC—DCE/T2/STIR (DWI)

Screening Populations?

Mammography

Lung CT screens

Long-term

Long-term requirements for database research resource:

Need to collect a million imaging phenomes, radiologists have to become the point of entry into populating the databases

Adding images to current TCGA in the long term may not useful, without major resources to scale up this archive and maintain inter-operability with TCGA.

Incentivization may come from Centers for Medicaid and Medicare Services, if effective DSS can be developed

Leverage TCGA to build a database for development of SS (with low bar AUC 0.85).

This strategy will not be successful unless processes are fully automated, and not require significant imaging physician’s time

What is needed to meet these long term database requirements?

Collection and curation of images including annotation

Annotated and qualified segmentations

Common metrology tools for annotation

Provenance and Health Insurance Portability and Accountability Act compliant

EMR links to clinical, genomic, pathology, and treatment data

Strategy for either federated or centralized data bases

Need to capitalize on the NCI cloud for data and metrology tool access

Industry support to sustain imaging and data archives

Logistical and Funding Recommendations

Research Resources Support

A means for sharing images and related metadata should be included in current and future TCIA-TCGA agreements with support for their annotation and additional archiving/hosting requirements

A means for metrology tool sharing and evaluation of feature extraction and clinical decision tools using cloud computing methods

A means for sustaining the above resources for their useful life times and support for scaling up these resources to meet the demands by the clinical and research communities, including industry participation.

Clinical and Basic Research Support

Targeted support for multidisciplinary research teams to populate the above resources for clinical and preclinical investigations

Targeted support to develop and evaluate advanced computational and informatics required using the above resources

Targeted support for investigations to optimize and implement decision support systems for targeted clinical and preclinical research projects

Uploading clinical trial data to a federally managed data repository with multiple centers in the US