Clinical Trial Design and Development Work Group Within the Quantitative Imaging Network.

The Clinical Trial Design and Development Working Group within the Quantitative Imaging Network focuses on providing support for the development, validation, and harmonization of quantitative imaging (QI) methods and tools for use in cancer clinical trials. In the past 10 years, the Group has been working in several areas to identify challenges and opportunities in clinical trials involving QI and radiation oncology. The Group has been working with Quantitative Imaging Network members and the Quantitative Imaging Biomarkers Alliance leadership to develop guidelines for standardizing the reporting of quantitative imaging. As a validation platform, the Group led a multireader study to test a semi-automated positron emission tomography quantification software. Clinical translation of QI tools cannot be possible without a continuing dialogue with clinical users. This article also highlights the outreach activities extended to cooperative groups and other organizations that promote the use of QI tools to support clinical decisions.

Introduction

The Quantitative Imaging Network (QIN) program under the National Cancer Institute (NCI) is chartered to improve clinical decision-making by developing and validating quantitative imaging (QI) methods including data acquisition and analysis methods to support the measure and prediction of tumor response to therapy (1). The clinical focus of the QIN requires both the promotion and the broad dissemination of the aforementioned tools within the imaging and clinical oncology communities (2). To accomplish these goals, the QIN brought together multidisciplinary teams under the governance of the Executive Committee comprising QIN principal investigators and the Coordinating Committee with leaders from three working groups (WGs) (Figure 1). These WGs are tasked with addressing problems encountered in Clinical Trial Design and Development (CTDD), Bioinformatics and Data Sharing (BIDS), and Image Analysis and Performance Metrics (IAPM), with subgroups focusing on positron emission tomography computed tomography (PET-CT) and magnetic resonance imaging (MRI). In this article, we summarize the activities and challenges of the CTDD working group (CTDDWG) in the past 10 years.

Figure 1.

Quantitative Imaging Network (QIN) governance structure. The Executive Committee oversees the entire technical network, while the Coordinating Committee deals with working group (WG) issues. Each QIN technical team provides membership to all of the WGs, namely, Clinical Trial Design and Development and Data Acquisition, Bioinformatics and Data Sharing, and Image Analysis and Performance Metrics, with subgroups focusing on positron emission tomography-computed tomography (PET-CT) and magnetic resonance imaging (MRI).

The mission of the CTDDWG is to develop, validate, and harmonize methods and tools of quantitative imaging for use in cancer clinical trials to predict outcome and tumor response to therapy. This group specifically aims to:

Identify challenges and opportunities in CTDD relevant to trials using QI.

Identify best practices for QI-based clinical trial design, analysis, and reporting.

Facilitate and introduce QIN-developed methods into cancer imaging trials through collaboration with other QIN WGs.

Disseminate the best clinical trial design and development method through publications and guidelines.

Facilitate outreach to cooperative groups and organizations to apply QIN methods in multicenter trials through cross-membership, education, and presentations.

Translate relevant and mature QIN methods into clinical practice settings as appropriate.

Initial activities of the CTDDWG began in 2008 that involved a fledgling group of 5 institutions. Initial efforts sought to identify constraints of using QI in oncology trials that included a review published by Kurland et al. (3) to examine the potential constraints on clinical trials involving QI and strategies to overcome them. The group additionally explored barriers to effectively accrue patients to single- and multi-institutional trials.

The CTDDWG was active in presenting at national meetings such as American Society for Therapeutic Radiology and Oncology (ASTRO), American Association of Cancer Research (AACR), and the Radiological Society of North America (RSNA) to publicize the work of the QIN and the potential for these tools to impact clinical trials. A white paper was published in Clinical Cancer Research (4), and a letter to Cancer Center Directors was published in the Journal of Clinical Oncology (5).

Identify Challenges and Opportunities in Clinical Trial Design and Development

Patient accrual is essential for the success of clinical trials. With additional procedures and administration of imaging agents, trials involving the development of new QI methods/tools may face more challenges in patient recruitment. The CTDDWG surveyed 25 prospective clinical studies from 12 sites to evaluate the accrual performance, barriers, and solutions that are specific to QI-based trials (6). Our findings showed that the median percent annual accrual attained was 94.5% (range, 3%–350%). The most common barrier to recruitment was that “patients decline participation” (44%), followed by “too few eligible patients” (40%). Response from our survey indicated that interactions between institutions, physicians, and patients are essential factors for accrual success, with attention paid to ensuring and communicating potential trial benefits to enrolled and future patients.

Recent advances in radiation oncology have shown an increasing reliance on both high-resolution anatomic-based imaging (CT and MRI) and functional imaging (PET, diffusion-weighted imaging (DWI), magnetic resonance spectroscopic (MRS) imaging, etc.) for accurate radiation therapy delivery. The CTDDWG reviewed the utility of QI by CT, MRI, and PET/single-photon emission computed tomography (SPECT) with different tracers for radiation treatment planning and patient management (7). It was concluded that QI provides quantitative measures that complement the standard qualitative assessment methods. Further integration of QI into the radiation oncology workflow for treatment planning, risk stratification, guidance of dose escalation, and characterization of post-treatment effects is expected.

Identify Best Practices for Clinical Trial Design, Analysis, and Reporting

As QI methods are being increasingly used in clinical trials as primary, secondary, or correlative endpoints, standard criteria for QI reporting should be defined to ensure that quantitative data extracted from images are reported in a meaningful, consistent, and repeatable manner. The group led the effort of assembling a guideline of what is viewed as “best practices” for reporting (and potentially assessing) studies in which QI has been performed and reported. This guideline of standardizing the reporting of quantitative imaging (STIRQI) adapts the STAndards for Reporting of Diagnostic Accuracy (STARD) criteria first reported in 2003 (8, 9) and subsequently updated in 2015 (10–12). Information regarding the purpose/background of the study, patient characteristics, image acquisition methodology, quantification methods, and statistical analysis should be described in detail. The outcome of the STIRQI is to ensure rigorous approaches are applied in QI studies so that results are reproducible and suitable for generalization to larger populations. Recommendation gathered from members of the CTDDWG and the leadership of the Quantitative Imaging Biomarkers Alliance (QIBA) group was drafted in the form of a white paper that will soon be published.

Translate Relevant and Mature QIN Methods into Clinical Practice Settings

PET Response Criteria in Solid Tumors (PERCIST) was initially proposed by Wahl et al. (13) as an approach to standardize interpretation of FDG-PET results using a consistent PET protocol. This group examined the variability for interpretation of FDG-PET results in 22 readers across 15 institutions on 30 test cases with scans before and after therapy using each institution's preferred software for this analysis. This study (14) showed a high correlation across readers and institutions but also identified some level of variability. Based on the PERCIST criteria, Wahl's team developed a semiautomated QI quantification software (AutoPERCIST™) to identify/measure reference tissue (liver), to set disease threshold values, and to calculate SUVs (standardized uptake values) (peak, max, mean, volume, and total lesion glycolysis). Using the CTDDWG as a testbed for tool validation, a multireader study was initiated. The main goal was to determine whether variance among readers and institutions was further reduced when AutoPERCIST™ was used. The latest version of this software was installed by 11 participating institutions (from United States, Asia, and Europe) through a material transfer agreement (MTA), and the 30 paired cases of anonymized FDG PET-CT images were downloaded for evaluation. Instructions for this study were given to each participating institution, with up to 5 tumor lesions from each PET image to be evaluated. All selections were recorded and sent to the central database of the Image Response Assessment Team at Johns Hopkins for quality control. Initial results show that consistent SUVpeak values were obtained, and the AutoPERCIST™ analysis had less variability than was seen when different institutional software was used. The initial results were presented at the Society of Nuclear Medicine and Molecular Imaging annual meeting (15), and a full manuscript was drafted and will soon be submitted for publication.

Outreach to Cooperative Groups and Organizations

The CTDDWG plays a key role in QIN outreach activities with the goal to raise awareness of the potential utility of advanced quantitative imaging for both research and general clinical practice. Several barriers (shown below) have limited the wider adoption of advanced quantitative imaging techniques in the clinic. The CTDDWG designed various panel discussions with oncology groups to educate trialists about the value of quantitative imaging and further promote the development and validation of new tools and methods needed to bring quantitative imaging into the clinical setting.

Learning Objectives

Members from the CTDDWG took the lead to promote the collaborations with the National Clinical Trials Network (NCTN) cooperative groups, testing QI tools in the clinical trial settings in the Alliance, Children's Oncology Group (COG), Eastern Cooperative Oncology Group, and the American College of Radiology Imaging Network (ECOG-ACRIN), NRG Oncology, and Southwest Oncology Group (SWOG). In addition, special panels highlighting QIN tools and their potential use in clinical trials were organized and presented at the American Association for Physicists in Medicine (AAPM) and the American Society for Therapeutic Radiology and Oncology (ASTRO) annual meetings.

Differentiate conventional qualitative from advanced quantitative imaging techniques and describe the added value of the quantitative methodologies to clinical management.

Demonstrate knowledge of specific tools being developed by featured QIN research groups that will allow various advanced quantitative imaging techniques to be utilized on oncology clinical trials.

Demonstrate an understanding of computing architecture/pipeline and informatic needs to fully exploit development of quantitative imaging methods/tools for clinical use.

Pathways for Quantitative Imaging Tools to Clinical Translation

Bringing imaging tools and methods into clinical utility is not a simple task. Tools need be tested–retested, optimized, and validated to be clinically ready. In addition, open dialogues between tool developers and clinicians are needed to crystalize tool capabilities that should not only reliably serve specific clinical needs but also be easily incorporated into the clinical workflow and be compatible with existing data systems.

To date, there are 67 QIN imaging tools at various stages of development. Using a benchmark process for clinical-readiness: 1) prebenchmark; 2) basic benchmark; 3) technical test; 4) clinical trial and 5) clinical use, Farahani et al. identified approximately 12 tools to be qualified at levels 4 and 5 (16). These tools include software packages for tumor segmentation, volumetric measurement, metric calculation, and bioinformatics interface that are all important for disease assessment, management, and intervention. A representative of QIN tools being used in clinical trials is listed in Table 1.

Table 1.

Representation of QIN Tools Used in Clinical Trials

Trial	Description	Modality	Measurement	QIN Tool	Application
NCT01015833	Phase III randomized study of sorafenib + doxorubicin vs sorafenib alone in patients with advanced hepatocellular carcinoma (HCC)	MRI/CT	Tumor volume/contour	Tumor segmentation	Treatment monitoring
NCT03137888	Spectroscopic MRI-guided radiation therapy planning in glioblastoma	MRI	Spectroscopic MR signal	Web-based clinical interface for spectral MRI analysis, visualization, and integration into patient management	Treatment planning
NCT01042379	I-SPY 2 TRIAL: Neoadjuvant and personalized adaptive novel agents to treat breast cancer	MRI	Functional tumor volume	Tool kit for image processing, tissue segmentation, and tumor volume calculation	Treatment monitoring
NCT02398733	[18F]Fluoroestradiol (FES) PET as a predictive measure for endocrine therapy in patients with newly diagnosed metastatic breast cancer	PET	Tracer uptake value	Phantom and companion software for PET scanner calibration	Imaging quality assurance and control (QA/QC)

To translate these sophisticated tools to the clinic, the QIN formed a team of “ambassadors,” with members from the CTDDWG reaching out to the NCTN cooperative groups to highlight the use of QI tools in treatment trials. In particular, a new membership was extended to ECOG-ACRIN with the intent of developing additional resources to support retrospective studies for tool testing and prospective studies for tool validations. Through the collaboration between the QIN members and the National Institute of Standards and Technology (NIST), imaging phantoms have been developed and commercialized to support image quality assurance and control (QA/QC) in clinical trials. All these collaborations and outreach activities have proven to be valuable as part of our effort in translating QI to clinical utility.

Future Directions

Looking forward, the CTDDWG members are working together to develop a clear roadmap and imaging workshops to provide education on how to include QI as part of the clinical trial. Other future directions also include a new collaboration with the Bioinformatics and Data Sharing Working Group (BIDSWG) to package QIN tools into open-source Docker containers (https://www.docker.com/get-started) in an automated pipeline infrastructure (17) to facilitate the connection, execution, and deployment of QIN tools into the clinical workflow. The BIDSWG is planning to work with the Imaging and Radiation Oncology Core (IROC) to test and validate the resulting pipeline. Through these activities, the CTDDWG will continue to facilitate the clinical translation of QI tools.