Quantitative Imaging Enters the Clinical Arena: A Personal Viewpoint.

The National Cancer Institute's Quantitative Imaging Network (QIN) has thrived over the past 12 years with an emphasis on the development of image-based decision support software tools for improving measurements of imaging metrics. An overarching goal has been to develop advanced tools that could be translated into clinical trials to provide for improved prediction of response to therapeutic interventions. This article provides an overview of the successes in development and translation of new algorithms into the clinical workflow by the many research teams of the Quantitative Imaging Network.

Introduction

As Director of the National Cancer Institute's Quantitative Imaging Network (QIN) for the past 12 years, I have witnessed the growth of QIN from early development of decision support software tools that use imaging for measuring or predicting response to cancer therapies to the early entrance of tested and refined tools into clinical trials. This journey has shed light on the fact that translation into clinical workflow is every bit as challenging as the original steps of development and testing. Knowledge that the challenges associated with translating research from the laboratory to the clinical arena exist is not new, but it appears that this fact must be rediscovered every few years. Dr. Paul Ehrlich, 1908 Nobel Prize winner in medicine, stated, “The step from laboratory to patient's bedside…is extraordinarily arduous and fraught with danger.” Although no details concerning the “danger” were provided by Dr. Ehrlich, his comment communicates the spirit of the difficulty experienced by QIN teams today.

Overview

QIN was initiated in 2008 with a program announcement and its first successfully reviewed application. Since that time, 43 teams have been supported to participate as members of the network, and 26 teams from around the world participated in the process of creating and testing clinical decision support tools for quantitative assessment of therapy response as associate members without National Cancer Institute (NCI) support. This growth in quantitative imaging is not isolated to QIN alone. Surveying PubMed, it is seen that there has been an increase in the number of articles containing the phrase “quantitative imaging” in the title by over a factor of seven in the past decade (Figure 1). It should not be inferred that the network was a large motivating factor in this increased interest in quantitative imaging. Rather, the best we can say is that the network was created at an opportune time when interest in the challenges of quantitative imaging was maturing.

Figure 1.

Growth of articles from PubMed with the words “Quantitative Imaging” in the title.

Over the past 12 years, applications to the network are traditionally received 3 times each year in response to an active program announcement published by the NCI. In each round, the applications are reviewed in a study section organized by NCI's Division of Extramural Activities. This process results in a network with staggered starts of its member teams with the expected effect of staggered degrees of progress toward eventual clinical testing of quantitative tools at any given time. With the large number of teams currently collaborating in the network, it is difficult to determine the performance readiness of each specific tool to enter clinical trials. Therefore, a method of benchmarking QIN tools was developed over a year ago (1). This process has enabled the network to prioritize the tools and to determine tools that are most prepared for entering clinical trials. To date, 67 tools have been benchmarked from a prebenchmark level (level 1) to clinical trial–ready (level 5).

Benchmarking of Tools

This benchmarking process was highlighted in an earlier issue of Tomography (Volume 5, Number 1; 2019). However, this was not the first time that progress from QIN teams was highlighted in special issues of prominent journals. The first was in 2012 in Magnetic Resonance Imaging (Volume 30, Number 8). At this stage in the network, teams were focusing on the development of algorithms and methods for measuring or predicting response to therapies. The papers in that issue reflect this stage in quantitative imaging development.

In 2014, Translational Oncology (Volume 7, Number 1) published a collection of QIN research articles showing the status of network progress. At this point in the progression of the network, diversity in the advancement through the translation pathway can be seen. Papers emphasizing development still dominate, but a number of articles are seen that discuss advanced topics of reproducibility, multisite data collection, and clinical trial challenges. This diversity in translation maturity is the result of the staggered start dates for the teams in the network.

The 2016 issue of Tomography (Volume 2, Number 1) covered all aspects of quantitative imaging research from QIN at that point in the program. Topics from test–retest for radiomic stability to simulating the effects of spectroscopic magnetic resonance imaging for radiation planning in glioblastoma showed that the network was moving away from initial development of algorithms and methods to real-world testing and analysis of quantitative methods.

In the SPIE Journal of Medical Imaging (Volume 5, Number 1), QIN was invited to submit original research articles on quantitative imaging. By this time, the field had grown so large that a number of institutions outside those supported by the QIN program contributed to the special SPIE issue. Despite this, the articles from QIN show a sustained progress from development discussions to clinical evaluation of quantitative tools. In addition, a glance at the author lists shows extensive collaboration among QIN teams.

As mentioned earlier, Tomography (Volume 5, Number 1; 2019) highlighted the effort made by QIN to benchmark developed tools in order to prioritize tools and methods ready for clinical testing. This issue stands as a watershed between QIN as a developer of quantitative imaging tools and QIN as a contributor of tools used in clinical evaluation. A complete list of the papers published in that issue of Tomography are listed in the References section of this paper [1-27].

Benchmark prioritization, however, is only the first step toward bringing qualified imaging tools into clinical utility. Announcing the availability and function of the clinically ready tools to appropriate users is another necessary task in the translation from development to clinical utility. To do this, QIN initiated efforts to present information on several clinically ready tools to the cooperative groups of the National Clinical Trial Network (NCTN). This process of presenting tools and their performance characteristics cannot be a one-time event, and the introduction of tools to these groups will continue. This issue of Tomography is an excellent example of another way to present QIN tools to the user community. Focusing specifically on tools and progress from the QIN, this issue shows the breadth of the challenges being met by the network.

QIN Network of Experts and Advances

A strength of the network is the extent of collaboration that exists, as noted by the enclosed 24 articles coauthored by members of different teams within QIN (28–51). Another notable feature to highlight is the continued interest and use of deep learning and radiomics to extract quantitative information from medical images. QIN was an early subscriber and contributor to these methods, and several articles here show that the use of these information extraction methods in imaging is increasing.

QIN often tackles research challenges that individual research investigators ignore. I direct your attention to several important examples in this issue. The article, “Standardization in Quantitative Imaging: A Multi-Center Comparison of Radiomic Features from Different Software Packages on Digital Reference Objects and Patient Data Sets” by McNitt-Gray et al. (35) shows not only the degree of collaboration that has been established in the network but also the type of challenge the investigators are choosing to attack. Standardization of software packages is a task that can be undertaken only when a large number of collaborating research groups come together and apply the necessary resources to accomplish the task.

Other studies that are necessary for acceptance of quantitative methods in the clinical environment include multisite uses of tools to determine performance characteristics in different clinical environments and imaging platforms. Smith et al. (29) look at this problem in “Multi-Site Technical and Clinical Performance Evaluation of Quantitative Imaging Biomarkers from 3D FDG PET Segmentations of Head and Neck Cancer Images.”

Jones (28) provides insight into the workings of one of the several working groups of the network in her article, “Clinical trial design and development work group within the Quantitative Imaging Network.” Other working groups in the network include bioinformatics/IT and data sharing, image analysis, and performance metrics, magnetic resonance imaging data collection, and positron emission tomography (PET)/computed tomography (CT) data collection.

QIN has had the privilege of receiving 2 research teams from Canada into the network. Both teams were supported by the Canadian Government and have been eager contributors to network progress even after their support ended. The group from Princess Margaret Hospital takes a serious look at 4DCT attenuation correction in gated PET for hypoxia (49). Researchers from the other Canadian member, the University of British Columbia, are part of the standardization article mentioned earlier (35).

Finally, I urge the readers to pay attention to the various articles on radiomics, deep learning, and data mining methods for extracting information from the images. These show that quantitative imaging is firmly rooted in the aforementioned statistical and analytical methods.

Direct support for QIN by NCI is now terminated, but that does not mean that research in quantitative imaging will not be supported in the future. Teams already part of the network will continue to be supported throughout their grant cycle, and new opportunities for quantitative imaging research can be provided through existing grant mechanisms such as the NCI parent announcement. Special funding to support the transition of tools into clinical utility is being planned and may be in place soon.

Summary

Quantitative imaging has matured from small development and demonstration projects in isolated universities and clinics to a substantial evolutionary effort encompassing scientific and statistical skillsets. It is being used to show increased efficiency in clinical trials and to shedd light on important cancer characteristics. QIN has a strong history in moving quantitative imaging forward and will continue to do so for years to come.