The Impact of Arterial Input Function Determination Variations on Prostate Dynamic Contrast-Enhanced Magnetic Resonance Imaging Pharmacokinetic Modeling: A Multicenter Data Analysis Challenge, Part II.

This multicenter study evaluated the effect of variations in arterial input function (AIF) determination on pharmacokinetic (PK) analysis of dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) data using the shutter-speed model (SSM). Data acquired from eleven prostate cancer patients were shared among nine centers. Each center used a site-specific method to measure the individual AIF from each data set and submitted the results to the managing center. These AIFs, their reference tissue-adjusted variants, and a literature population-averaged AIF, were used by the managing center to perform SSM PK analysis to estimate Ktrans (volume transfer rate constant), ve (extravascular, extracellular volume fraction), kep (efflux rate constant), and τi (mean intracellular water lifetime). All other variables, including the definition of the tumor region of interest and precontrast T1 values, were kept the same to evaluate parameter variations caused by variations in only the AIF. Considerable PK parameter variations were observed with within-subject coefficient of variation (wCV) values of 0.58, 0.27, 0.42, and 0.24 for Ktrans, ve, kep, and τi, respectively, using the unadjusted AIFs. Use of the reference tissue-adjusted AIFs reduced variations in Ktrans and ve (wCV = 0.50 and 0.10, respectively), but had smaller effects on kep and τi (wCV = 0.39 and 0.22, respectively). kep is less sensitive to AIF variation than Ktrans, suggesting it may be a more robust imaging biomarker of prostate microvasculature. With low sensitivity to AIF uncertainty, the SSM-unique τi parameter may have advantages over the conventional PK parameters in a longitudinal study.

Introduction

As a noninvasive method to measure tissue microvascular perfusion and permeability, dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is increasingly used in oncologic imaging for cancer diagnosis and therapeutic monitoring (1, 2). DCE-MRI generally involves the serial acquisition of heavily T1-weighted images before, during, and after the injection of a paramagnetic contrast agent (CA). Quantitative pharmacokinetic (PK) modeling of DCE-MRI time-course data allows estimation of imaging biomarkers, such as Ktrans (volume transfer rate constant) and ve (extravascular, extracellular volume fraction), that are direct measures of tissue biology and in principle independent of data acquisition details and MRI scanner platform (3). However, the accuracy and precision of the derived PK parameters can be largely affected by the selection of the PK model for data fitting (3–5), errors in quantification of the native tissue T1 value (3, 4, 6), and variance in determination of arterial input function (AIF; the time-course of CA plasma concentration) (3, 4, 7–9). These challenges lead to substantial variations in the reported PK parameter values for the same disease and are fundamental obstacles in translating quantitative DCE-MRI into multicenter clinical trials and general clinical practice. Therefore, it is important for the DCE-MRI community to investigate the impact of variations/errors in different steps of PK data analysis on the estimated parameter values, establish ways to reduce parameter variance, and identify those parameters that are less sensitive to certain variations in data analysis and, therefore, the more robust imaging biomarkers for multicenter studies.

Quantification of the AIF is generally required in most PK models to fit the DCE-MRI time-course data from the tissue of interest. There are many approaches to determine the AIF, including blinded estimation (10), reference tissue (11–13), empirically derived population-averaged AIF (14), and direct measurement of AIF from a feeding artery if the artery is clearly visible within the image field of view (9). In a previous multicenter data analysis challenge (9) within the Quantitative Imaging Network (QIN) of the National Cancer Institute, we have shown, with shared DCE-MRI data sets from patients with prostate cancer, different extents of PK parameter variations owing to differences in individually measured AIFs using site-specific methods. We have shown that parameter variations could be reduced by using a reference-tissue method (15, 16) to adjust the amplitude of the measured AIF. The commonly used standard Tofts model (17, 18) with two independent fitting parameters (Ktrans and ve) was used for PK analysis in that study. A recent single-center prostate DCE-MRI study (19) also shows parameter variations when individually measured AIFs and literature population-average AIFs were used for PK analysis with the Tofts model, resulting in substantial variations in diagnostic accuracy of prostate cancer.

In this study, part II of the QIN multicenter data analysis challenge, the shutter-speed model (SSM) (20, 21) was used to perform PK analysis of the shared data sets with AIFs measured by multiple QIN centers. The main difference between the SSM and the Tofts model is that the former takes into account intertissue-compartment water-exchange kinetics. An additional parameter, the mean intracellular water lifetime (τi), is used in the SSM to account for the transcytolemmal (cross cell membrane) water-exchange kinetics. Recent studies show that the SSM-derived Ktrans parameter is a more accurate diagnostic marker for both breast (22, 23) and prostate cancer (24), and pretreatment τi is predictive of breast cancer response to neoadjuvant chemotherapy (25) and overall survival in patients with head and neck cancer (26). Furthermore, recent results suggest that τi is potentially a new imaging biomarker of cellular metabolic activity (27–31), specifically the activity of the Na+-K+-ATPase pump, which is essential for all mammalian cells and is primarily responsible for maintaining the K+ and Na+ gradient in vivo. In addition, a simulation study (16) has shown low sensitivity of τi to AIF amplitude scaling compared with other conventional PK parameters such as Ktrans. Thus, it is important to experimentally investigate the effect of uncertainty in AIF determination on parameters estimated with the SSM, which was the goal of this study.

Materials and Methods

Data Sharing and Multicenter AIF Measurement

Axial prostate DCE-MRI data were collected by one QIN center (32) for pretreatment staging of patients with prostate cancer. Data sets from 11 patients were shared with other QIN centers through TCIA (The Cancer Imaging Archive). These data sets were acquired at 3 T using a 3-dimensional SPoiled Gradient Recalled (SPGR) sequence with repetition time = 3.6 milliseconds, echo time = 1.3 milliseconds, flip angle = 15°, a temporal resolution ranging from 4.4 to 5.3 seconds, and about 60 frames for a 4.5- to 6-minute acquisition time. Nine QIN centers, denoted as QIN1 to QIN9, downloaded the DCE-MRI data and performed AIF measurement from a single image slice for each individual data set using site-specific methods. The smaller circular region of interest (ROI) placed in the left femoral artery (Figure 1A insert) shows the most common location where the AIFs were measured. The derived AIFs in the form of signal intensity time-course data were then submitted to one of the 9 centers, the data managing center, for centralized PK analysis of the 11 DCE-MRI data sets. Additional details on DCE-MRI acquisition parameters and the methods used by each center for AIF measurement from the imaging data are described in Huang et al.'s study (9).

Figure 1.

Individual arterial input functions (AIFs) measured from one subject's prostate dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) data set by 9 Quantitative Imaging Network (QIN) centers. The insert in (A) is a zoomed axial postcontrast DCE-MRI image slice showing the smaller red, circular region of interest (ROI) in the left femoral artery where the blood signals were measured for the AIF time-courses, and the larger red, ellipsoidal reference ROI in the normal-appearing obturator muscle adjacent to the prostate. Substantial variations in both the shape and magnitude can be observed in the AIF curves determined by the 9 QIN centers (A), which are clearly reduced following magnitude adjustment using the reference tissue method (B).

DCE-MRI Data Analysis

The AIF signal intensity time-course was converted by the managing center to blood R1 ( 1/T1) time-course, R1,b(t), using the steady-state MRI signal intensity equation for a gradient pulse sequence (33) with the known acquisition parameters of flip angle, echo time, and repetition time, and a fixed precontrast blood R1 of 0.61 s−1 (34), and then to plasma CA concentration time-course, Cp(t), using the following equation: where r1 is the CA relaxivity at 3 T, set at 3.8 mM−1s−1; h is the hematocrit, set at 0.45.

For comparison with the individually measured AIFs, a frequently cited and used population-averaged AIF published by Geoff Parker (GP) et al. (14) was also included in this study. The analytical expression of the GP AIF was implemented at the managing center and resampled to match the temporal features of the prostate DCE-MRI data sets.

For each data set, the prostate tumor ROI was defined on a single image slice through the central portion of the tumor by one investigator from the center where the data were generated. The signal intensity time-course for each voxel within the tumor ROI was converted by the managing center to R1 time-course, R1(t), in the same way as for R1,b(t), but with a fixed precontrast R1 for the tumor tissue, R10, assumed to be 0.63 s−1 (7). Following calculation of Cp(t) [equation (1)] for each of the AIFs measured by the 9 QIN centers and the literature GP AIF, and R1(t) for each tumor voxel, the managing center performed PK analysis of the shared 11 prostate DCE-MRI data sets on a voxel-by-voxel basis using an in-house Python-based SSM software package. All AIF arrival times were manually aligned with the uptake phase of the average tissue response curves from the tumor ROIs. The 2-compartment–3-parameter version of the SSM (20, 21) was used for R1(t) data fitting in this study: where R1i is the intrinsic intracellular longitudinal relaxation rate constant and is assumed to be equal to the tissue R10. The PK model fitting returned Ktrans, ve, and τi parameter values for each voxel within the tumor ROI, and the CA efflux rate constant, kep, was calculated as kep = Ktrans/ve. The mean parameter values of the single-slice tumor ROI were obtained by averaging the voxel parameter values within the ROI.

Owing to large differences in the site-specific methods for the AIF measurement (9), such as the placement of the ROI in the artery and the ROI size, substantial variations in AIF amplitude were observed in the AIFs measured from the same data set. A reference tissue method (15, 24) was used to adjust the amplitude of the measured AIFs, as well as the literature GP AIF, in an attempt to reduce the variations (9). In this approach, an ellipsoidal ROI (Figure 1A insert) was drawn in the adjacent, normal-appearing obturator muscle area on the same image slice as the one for the AIF measurement and used as the reference tissue ROI. The AIF amplitude was varied until the Tofts model fitting of the DCE-MRI data from the muscle reference tissue ROI returned a ve value of 0.1 (35). In total, 20 AIFs, including unadjusted and reference tissue-adjusted AIFs measured by the 9 QIN centers and of the literature GP AIF, were used for PK modeling of each prostate DCE-MRI data set using the SSM, resulting in 20 sets of mean tumor Ktrans, ve, kep, and τi values that were then separated into two groups of results based on the unadjusted and reference tissue-adjusted AIF approaches.

Because a physically meaningful ve is in the range of 0.0 to 1.0, these two values were used as the lower and upper boundaries, respectively, for SSM fitting of all voxel data. All returned voxel ve values were within the two boundaries (none at boundary values) when the reference tissue-adjusted AIFs were used, while, on average, <3% voxels (range: 0%–6.6% for all the AIF and data set combinations) had returned ve values reaching the upper boundary of 1.0 when the unadjusted AIFs were used. In the latter case, the parameter values from these limited number of voxels with ve value of 1.0 were not excluded from the calculation of tumor mean parameter values.

Statistical Analysis

The mean parameter values for the tumor ROI obtained from all fittings were used for statistical analysis. Descriptive statistical analysis was conducted to summarize the PK parameter values returned using different AIFs, with the distribution graphically assessed by boxplots. Intraclass correlation coefficients (ICC), within-subject coefficient of variation (wCV), and concordance correlation coefficients (CCC) were calculated, and these were reported with the corresponding 95% confidence intervals (CIs). Although all three coefficients were computed to assess the reproducibility of the PK parameter values obtained with different AIFs, each had specific focus. The ICC measures the proportion of total variation contributed by between-subject differences, with a high ICC value indicating good agreement (36). The wCV is the ratio of within-subject standard deviation to the mean of a parameter, with smaller wCV value suggesting better reproducibility. Closely related to ICC, CCC represents the level of pairwise linear agreement to a 45° line of which the intercept is forced to be zero. A larger CCC indicates better agreement between results from a pair of measurements and thus better reproducibility. Bland–Altman plots were used to graphically demonstrate pairwise agreements in results from different AIF measurements. SAS 9.4 (Cary, NY) was used for all statistical analysis. SAS macro “%ICC9” and “%mccc” were used for the estimations of ICC, wCV, and CCC.

Results

Variations in AIF Determination

For each data set, substantial variations in both the amplitude and shape of the Cp(t) time-course can be observed as a result of direct AIF measurement from the DCE-MRI data by the 9 QIN centers using site-specific methods. A clear example of Cp(t) variation is shown in Figure 1A. Following amplitude adjustment of Cp(t) using the reference tissue (Figure 1A insert), the agreement among the individually measured AIF curves was clearly improved (Figure 1B). Table 1 lists the standard deviation (SD) of the Cp(t) peak amplitude for unadjusted and reference tissue-adjusted AIFs from measurements by the 9 centers for each patient. Two-tailed paired t test shows that the AIF peak value SD of the reference tissue-adjusted AIFs is significantly (P = .018) smaller than that of the unadjusted AIFs.

Table 1.

Standard Deviation of AIF Peak from Multicenter Measurements

Patient	SD of AIF Peak Value (mM)
	Unadj. AIF	Adj. AIF[a]
1	0.88	0.54
2	2.36	0.72
3	4.74	1.98
4	0.75	0.65
5	0.55	0.32
6	0.68	0.32
7	0.55	0.76
8	1.63	0.64
9	0.41	0.42
10	1.28	0.56
11	4.45	2.27

a Standard deviation (SD) of AIF peak value is significantly smaller for reference tissue-adjusted (Adj.) AIFs in comparison with unadjusted (Unadj.) AIFs: 2-tailed paired t test, P = .018.

PK Parameter Variations Due to AIF Differences

Figure 2 shows the boxplots of Ktrans, ve, kep, and τi parameters estimated from SSM modeling of the 11 DCE-MRI data sets with adjusted and unadjusted AIFs (including those from the GP AIF). For most measurements, the mean is greater than the median, which is commonly seen when distributions are skewed toward larger parameter values. The dispersion of the estimated parameter values from the 11 patients varies substantially across the QIN centers (or AIFs), with Ktrans showing clearly the largest variation, while ve and τi exhibiting the least variations. As another marker of microvascular properties, kep shows less variation than Ktrans. Comparing the boxplots between unadjusted and adjusted AIFs, it can be visually observed that the agreement in parameter dispersion among different centers (or AIFs) is improved for Ktrans and ve when the reference tissue-adjusted AIFs were used in data fitting, but this is not clearly the case for kep and τi. Similar observations can be obtained from Table 2, which shows the mean SSM parameter values and 95% CIs for each patient under the unadjusted and reference tissue-adjusted AIF approaches. The mean values were calculated by averaging the tumor parameter values derived with the individual AIFs determined by the 9 QIN centers.

Figure 2.

Boxplots of the tumor mean Ktrans, ve, kep, and τi parameters for the 11 subjects obtained with shutter-speed model (SSM) analysis using unadjusted (left column) and reference tissue-adjusted (right column) AIFs measured by the 9 QIN centers and the population-averaged Geoff Parker (GP) AIF from the literature (14). The diamond and bar symbols represent the mean and median values, respectively. The body of the box is bounded by the upper 75% and lower 25% quartiles, representing the interquartile range of the middle 50% of the measurements. The upper and lower whiskers define the range of non-outliers. The outliers are plotted as dots beyond the whiskers.

Table 2.

Mean and 95% Confidence Interval of the SSM PK Parameters Obtained with Unadjusted and Reference-Tissue-Adjusted AIFs

Patient	Unadj. AIF				Adj. AIF
	Ktrans (min−1)	ve	kep (min−1)	τi (s)	Ktrans (min−1)	ve	kep (min−1)	τi (s)
1	0.52 (0.26, 0.77)	0.65 (0.55, 0.76)	0.80 (0.60, 1.00)	0.38 (0.32, 0.44)	0.35 (0.26, 0.43)	0.48 (0.46, 0.51)	0.75 (0.60, 0.90)	0.31 (0.28, 0.34)
2	0.99 (0.45, 1.52)	0.41 (0.25, 0.57)	2.26 (1.92, 2.61)	0.20 (0.14, 0.27)	0.94 (0.79, 1.08)	0.41 (0.39, 0.43)	2.27 (1.92, 2.60)	0.19 (0.16, 0.22)
3	1.89 (1.32, 2.46)	0.35 (0.26, 0.43)	5.58 (4.55, 6.61)	0.34 (0.24, 0.44)	2.29 (1.88, 2.65)	0.44 (0.41, 0.47)	5.44 (4.59, 6.49)	0.36 (0.25, 0.46)
4	2.67 (2.45, 2.89)	0.74 (0.65, 0.83)	3.73 (3.26, 4.20)	0.40 (0.29, 0.50)	2.15 (1.88, 2.42)	0.49 (0.46, 0.53)	4.47 (3.87, 5.07)	0.34 (0.24, 0.44)
5	0.60 (0.43, 0.77)	0.44 (0.39, 0.49)	1.45 (1.20, 1.71)	0.60 (0.42, 0.83)	0.48 (0.38, 0.58)	0.36 (0.34, 0.40)	1.44 (1.21, 1.68)	0.58 (0.41, 0.82)
6	1.21 (0.77, 1.65)	0.92 (0.89, 0.96)	1.26 (0.84, 1.68)	0.42 (0.37, 0.48)	1.06 (0.83, 1.28)	0.93 (0.92, 0.94)	1.11 (0.87, 1.34)	0.46 (0.44, 0.48)
7	0.44 (0.33, 0.54)	0.84 (0.60, 0.99)	0.48 (0.38, 0.58)	0.41 (0.36, 0.46)	0.22 (0.16, 0.28)	0.81 (0.76, 0.85)	0.29 (0.23, 0.36)	0.56 (0.54, 0.58)
8	0.68 (0.32, 1.04)	0.78 (0.62, 0.94)	0.82 (0.47, 1.18)	0.42 (0.37, 0.47)	0.63 (0.35, 0.91)	0.79 (0.74, 0.84)	0.79 (0.46, 1.12)	0.41 (0.39, 0.44)
9	1.10 (0.75, 1.45)	0.63 (0.58, 0.67)	2.01 (1.50, 2.52)	1.25 (1.18, 1.32)	0.80 (0.59, 1.02)	0.51 (0.49, 0.53)	1.97 (1.42, 2.51)	1.21 (1.13, 1.29)
10	1.25 (0.59, 1.91)	0.62 (0.49, 0.76)	1.87 (1.29, 2.44)	0.35 (0.24, 0.46)	1.28 (0.75, 1.82)	0.71 (0.66, 0.76)	1.81 (1.14, 2.40)	0.39 (0.32, 0.46)
11	1.13 (0.55, 1.71)	0.52 (0.42, 0.61)	2.13 (1.45, 2.81)	0.23 (0.20, 0.26)	0.90 (0.31, 1.40)	0.37 (0.30, 0.45)	2.36 (1.56, 3.25)	0.22 (0.19, 0.24)

The values in the parenthesis represent the lower and upper bounds of the 95% confidence interval.

Figure 3 shows a column graph of wCV for Ktrans, ve, kep, and τi obtained with the unadjusted (gray) and adjusted (white) AIFs. The error bars represent the 95% CIs. The larger the wCV value, the higher the variation in a measurement performed on the same subject by different methods. The wCV values for Ktrans, ve, kep, and τi are 0.58, 0.27, 0.42, and 0.24 for unadjusted AIFs, and 0.50, 0.10, 0.39, and 0.22 for adjusted AIFs, respectively. The wCV of Ktrans is the largest among all 4 parameters with either unadjusted or adjusted AIFs, while those of ve and τi are the smallest. From unadjusted to adjusted AIFs, the decrease in parameter variation is more prominent for Ktrans and ve (wCV value decreases from 0.58 to 0.50 and from 0.27 to 0.10, respectively), compared with kep and τi (0.42 to 0.39 and 0.24 to 0.22, respectively). Figure 4 shows a similar graph of ICC values for Ktrans, ve, kep, and τi obtained with the two AIF approaches. The ICC values for Ktrans, ve, kep, and τi are 0.44, 0.51, 0.72, and 0.92 for unadjusted AIFs, respectively, and 0.59, 0.91, 0.79, and 0.93 for adjusted AIFs, respectively. Consistent with the results shown in Figure 3, Ktrans has the smallest ICC value with either AIF approach, while τi has the largest ICC value. From unadjusted to adjusted AIFs, the increase in ICC is the most obvious for Ktrans and ve (ICC value increases from 0.44 to 0.59 and from 0.51 to 0.91, respectively) compared with kep and τi (0.72 to 0.79 and 0.92 to 0.93, respectively).

Figure 3.

Column graphs of within-subject coefficient of variation (wCV) for the SSM Ktrans, ve, kep, and τi parameters obtained with the unadjusted (gray) and adjusted (white) AIFs. The respective 95% confidence intervals (CI) are shown as error bars.

Figure 4.

Column graphs of intraclass correlation coefficient (ICC) for the SSM Ktrans, ve, kep, and τi parameters obtained with the unadjusted (gray) and adjusted (white) AIFs. The respective 95% CIs are shown as error bars.

As an example of differences in AIF-caused variations in estimated PK parameters when unadjusted and reference tissue-adjusted AIFs were used for SSM analysis, Figure 5 shows voxel-based parametric maps of Ktrans and τi of a prostate tumor generated from the SSM analysis. The tumor ROI was in the peripheral zone, as indicated by the arrow in the postcontrast DCE-MRI image. Ktrans and τi maps obtained with unadjusted AIFs from the 9 QIN centers are shown on the left panels and those with reference tissue-adjusted AIFs are shown on the right. These maps are displayed under the same Ktrans and τi color scales, respectively. With either AIF approach, substantially higher variations among the 9 Ktrans maps can be visually observed compared with the 9 τi maps. While the variations among the Ktrans maps can be seen reduced when the reference tissue-adjusted AIFs were used, there is no noticeable improvement in agreement among the τi maps going from unadjusted to adjusted AIFs. It is interesting to note, however, that despite considerable variations in Ktrans maps owing to AIF differences, the spatial pattern of voxel Ktrans distribution largely remains the same in all the maps. This was also observed in the τi maps, and in the maps of ve and kep (data not shown for the latter two parameters).

Figure 5.

Voxel-based Ktrans (top two panels) and τi (bottom two panels) parametric maps in a prostate tumor ROI, with each panel consisting of 9 maps corresponding to those obtained with AIFs measured by 9 QIN centers. The left and right two panels show the maps obtained with unadjusted and adjusted AIFs, respectively. The grayscale image at the center shows an axial postcontrast DCE-MRI image slice, with the arrow pointing to the cyan-colored prostate tumor ROI. The color scales of Ktrans and τi are kept the same, respectively, for the unadjusted and adjusted AIF approaches.

Concordance Analysis

Concordance correlation analysis was conducted to assess parameter agreement between any two AIFs under the same condition (adjusted or unadjusted). Tables 3 and 4 tabulate the CCC values for Ktrans and τi, respectively, with those for the unadjusted AIFs listed in the top right half and those for the adjusted AIFs in the lower left half. The CCC ranges for Ktrans and τi are 0.005–0.937 and 0.558–0.993, respectively, for unadjusted AIFs, and 0.102–0.991 and 0.640–0.997, respectively, for adjusted AIFs. Reflective of the results shown in Figures 3 and 4, there is generally a considerable increase (comparing values that are symmetric to the diagonal line in Table 3) in the CCC value for pair-wise comparisons of the Ktrans parameter going from unadjusted to adjusted AIFs, while little CCC changes are observed (comparing values that are symmetric to the diagonal line in Table 4) for the τi parameter. The CCC ranges for ve and kep (tables not shown) are 0.334–0.986 and 0.145–0.957, respectively, for unadjusted AIFs, and 0.554–0.993 and 0.129–0.965, respectively, for adjusted AIFs. From unadjusted to adjusted AIFs, the changes in CCC for ve and kep are similar to those for Ktrans and τi, respectively. In addition, it is important to note that with either AIF approach, the CCC values for pair-wise comparisons that included the GP AIF are among the smallest values in the aforementioned CCC ranges.

Table 3.

CCC Values for Ktrans

	QIN1	QIN2	QIN3	QIN4	QIN5	QIN6	QIN7	QIN8	QIN9	GP
QIN1		0.239	0.702	0.683	0.914	0.846	0.921	0.838	0.790	0.005
QIN2	0.406		0.464	0.188	0.197	0.358	0.317	0.325	0.337	0.084
QIN3	0.836	0.642		0.440	0.666	0.937	0.825	0.857	0.639	0.159
QIN4	0.462	0.277	0.498		0.669	0.565	0.541	0.600	0.581	0.182
QIN5	0.960	0.409	0.840	0.643		0.820	0.864	0.747	0.718	0.089
QIN6	0.881	0.586	0.991	0.548	0.880		0.937	0.886	0.685	0.144
QIN7	0.990	0.447	0.862	0.562	0.969	0.906		0.780	0.800	0.045
QIN8	0.975	0.372	0.864	0.682	0.942	0.911	0.961		0.595	0.148
QIN9	0.977	0.488	0.866	0.620	0.938	0.895	0.981	0.931		0.057
GP	0.191	0.102	0.162	0.348	0.224	0.173	0.209	0.159	0.194

CCC values for unadjusted (unadj.) AIFs are presented in the top right triangle and those for reference-tissue-adjusted (adj.) AIFs are presented in the bottom left triangle.

Table 4.

CCC Values for τi

	QIN1	QIN2	QIN3	QIN4	QIN5	QIN6	QIN7	QIN8	QIN9	GP
QIN1		0.858	0.937	0.821	0.947	0.977	0.972	0.933	0.953	0.583
QIN2	0.920		0.935	0.835	0.895	0.855	0.869	0.881	0.882	0.577
QIN3	0.945	0.976		0.849	0.974	0.899	0.908	0.949	0.920	0.594
QIN4	0.803	0.845	0.859		0.860	0.842	0.864	0.840	0.872	0.773
QIN5	0.938	0.955	0.995	0.849		0.922	0.925	0.964	0.941	0.600
QIN6	0.997	0.906	0.938	0.806	0.937		0.993	0.949	0.973	0.619
QIN7	0.989	0.920	0.945	0.844	0.941	0.990		0.943	0.965	0.662
QIN8	0.974	0.916	0.960	0.815	0.957	0.971	0.965		0.954	0.617
QIN9	0.978	0.929	0.965	0.830	0.962	0.979	0.971	0.992		0.558
GP	0.702	0.640	0.658	0.840	0.653	0.714	0.764	0.675	0.677

CCC values for unadjusted (unadj.) AIFs are presented in the top right triangle and those for reference-tissue-adjusted (adj.) AIFs are presented in the bottom left triangle.

Bland–Altman plots are shown in Figure 6 to show examples of pair-wise agreements in Ktrans (Figure 6A) and τi (Figure 6B). The plots are displayed only for the AIF pairs with the largest (top rows in Figure 6, A and B) and smallest (bottom rows in Figure 6, A and B) CCC values for the unadjusted (left columns) and reference tissue-adjusted (right columns) AIFs. Although the differences between the measurements are mostly within the 95% CIs for all the plots, it is clear, with the vertical axis scales kept the same for the Ktrans and τi plots, respectively, that the width of the CI band differs substantially between AIF pairs with greater CCC values and those with smaller CCC values: narrower for the former and wider for the latter. For Ktrans and τi with the largest CCC values (ie, the best pair-wise agreements in the estimated Ktrans and τi values), the means of parameter difference represented by the dotted lines are 0.22 min−1 and 0.012 seconds, respectively, for unadjusted AIFs, and 0.078 min−1 and 0.005 seconds, respectively, for adjusted AIFs. For Ktrans and τi with the smallest CCC values (ie, the worst pair-wise agreements in the estimated Ktrans and τi values), the means of parameter difference represented by the dotted lines are −0.56 min−1 and −0.18 seconds, respectively, for unadjusted AIFs, and −1.29 min−1 and −0.18 seconds, respectively, for adjusted AIFs. From unadjusted to adjusted AIFs, the decrease in the width of the 95% CI band is substantially greater for the Ktrans parameter than that for the τi parameter; the average percent decrease (from the cases with the largest and smallest CCCs) is 37% for Ktrans and 15% for τi. This indicates that the use of reference tissue-adjusted AIF has a stronger effect in improving parameter agreement in Ktrans compared with τi. The same observation was made when comparing Ktrans and kep (data not shown). In addition, in cases of poor Ktrans agreement (bottom row of Figure 6A), there appears to be a correlation (linear bias) between the difference in Ktrans and the mean of Ktrans; the larger the parameter value, the larger the difference in the parameter value between the two measurements. No clear correlation is observed for τi, even in cases of poor agreement (bottom row of Figure 6B).

Figure 6.

Bland–Altman plots showing agreements in Ktrans (A) and τi (B) for AIF pairs with the largest (top row in A and B) and smallest (bottom row in A and B) CCC values under the conditions of unadjusted (left column in A and B) and adjusted (right column in A and B) AIFs. The two solid horizontal lines represent the upper and lower limits of the 95% CI, while the dotted horizontal line represents the mean value of Ktrans (A) and τi (B) difference between the paired measurements.

Discussion

In this part II of a multicenter data analysis challenge to evaluate the effect of variations in AIF determination on estimated PK parameters from prostate DCE-MRI data, the SSM was used for PK modeling of the DCE-MRI data. All other aspects in the data analysis were kept the same as those in part I (9) of the challenge where the standard two-parameter (Ktrans and ve) Tofts model was used. For example, quality control measures such as fixed tumor ROI definition, fixed tumor T10, and central data analysis with a single SSM software package were adopted to ensure that PK parameter variations are mainly due to variations in only AIF. Compared with challenge part I (9), where the effect of AIF uncertainty was evaluated on parameters of Ktrans, ve, and kep, one additional parameter, τi, was included in this part II study.

Consistent with results from challenge part I (9), substantial variations in the estimated PK parameters were observed in this study owing to variations in AIF quantification by 9 QIN centers using site-specific methods (9), especially in Ktrans and kep. Among the four parameters derived with the SSM using unadjusted AIFs, Ktrans shows the largest AIF-caused variation with a wCV value of 0.58, while ve and τi show the smallest variations with nearly equal wCV values of 0.27 and 0.24, respectively. Although higher than ve and τi, kep has a lower AIF-caused variation than Ktrans, with a wCV value of 0.42. Our findings are in agreement with a recent study comparing fully automated and semiautomated AIF determination approaches for prostate DCE-MRI data analysis (7), showing that Ktrans variation owing to AIF uncertainty is the most prominent compared with other PK parameters. A similar conclusion was drawn in a brain DCE-MRI study (8) that investigated PK parameter variations caused by the use of AIFs measured from different vessels.

As shown by this study using the SSM, as well as part I of the challenge (9) using the standard Tofts model, adjusting the amplitudes of individually measured AIFs with a reference-tissue method (15, 24) by placing the reference ROI in the adjacent normal muscle region can decrease AIF variance (Table 1) and, as a result, reduce parameter variations. For example, the wCV values were decreased from 0.58 to 0.50 and from 0.27 to 0.10 for Ktrans and ve, respectively, when the reference tissue-adjusted AIFs replaced the unadjusted AIFs in the SSM analysis. The effect of AIF amplitude adjustment was smaller, however, on kep (wCV: 0.42 to 0.39) and τi (wCV: 0.24 to 0.22) parameters. These observations are consistent with the results from a simulation study using the SSM (16), which found significantly lower sensitivity of kep and τi to a 30% change in AIF amplitude compared with Ktrans and ve. Interestingly, the aforementioned brain DCE-MRI study (8) using the extended Tofts model (18) also showed lower variation of kep in response to different AIF sources compared with Ktrans. Because kep, like Ktrans, is also a measure of perfusion and permeability, the low sensitivity of kep to AIF amplitude uncertainty suggests that kep could be a more robust and reproducible imaging biomarker than Ktrans for DCE-MRI characterization of tissue microvasculature (37) when consistent and accurate AIF quantification is difficult.

In pair-wise assessment of agreement in parameter values obtained with two different AIFs, the worst agreements (or the smallest CCC values) generally occurred when a measured AIF (from acquired DCE-MRI data) was paired with the literature population-averaged GP AIF, for any parameter and under the condition of either unadjusted or adjusted AIFs. It is important to note that, in addition to amplitude, the AIF curve shape also influences the estimation of the PK parameters (3, 9). Although the methods used by the 9 QIN centers to measure the AIFs were quite different (9), the individually measured AIFs captured the actual AIF curve shapes from the DCE-MRI data. The curve shape is specific to data acquisition details and data sampled for AIF quantification. This may not be well represented by the GP AIF, which is modeled on the basis of data from the aorta or iliac arteries acquired with different pulse sequence parameters at a different field strength. Such differences between the measured AIFs and GP AIF are probably a central reason why any pair-wise comparison of the GP AIF with a measured AIF resulted in large differences in estimated PK parameter values. Therefore, whenever possible, an individually measured AIF should be used for PK analysis of DCE-MRI data instead of a generic population-averaged AIF, which may be unrelated to a specific study. This conclusion is based on the results from this study, as well as on those from part I of this data analysis challenge (9), which were obtained from a single time-point pretreatment prostate DCE-MRI data sets. For longitudinal DCE-MRI studies of cancer response to treatment, percent changes in parameter values (rather than absolute values) are generally used to assess therapy response, and high parameter repeatability is crucial. The use of a fixed population-averaged AIF may have advantages over individually measured AIFs because of the likely randomness of AIF measurement errors in the latter approach across multiple studies over a period of time. A recent DCE-MRI study of 13 patients with abdominal metastases by Rata et al. (38) shows that the highest parameter repeatability in a baseline test–retest study was achieved with a population-averaged AIF in comparison with three approaches of direct AIF measurement from acquired imaging data, and that, as a result, parameters derived with the population-averaged AIF have the highest sensitivity to treatment-induced changes. Further investigations with larger patient cohorts and data collected from different organs are needed before clear recommendations can be made in terms of direct AIF measurement versus fixed population-averaged AIF (39) for longitudinal DCE-MRI evaluation of cancer therapy response.

The representative parametric maps of Ktrans and τi (Figure 5) indicate that DCE-MRI parameter variations caused by AIF variations are mostly systematic. Despite differences in absolute voxel parameter values owing to different AIFs used in SSM analysis, it can be seen that the pattern of voxel parameter distribution largely remains the same for all the Ktrans or τi maps, that is, there are no visible changes in the spatial locations of the parameter “hot” and “cold” spots when different AIFs were used. This is also observed in the voxel-based kep and ve parametric maps (not shown). Therefore, the assessment of PK parameter spatial heterogeneity using texture analysis of parametric maps may not be affected by variations in AIF determination. However, quantitative texture feature analysis needs to be conducted to test this hypothesis.

The τi parameter is unique to the SSM, and its reciprocal, 1/τi, is a measure of the rate of water cycling across cell membrane. Previous studies (27–31) indicate that τi is an imaging biomarker of cellular metabolic activity, specifically the activity of Na+-K+-ATPase, which consumes ATP and drives active water cycling. The relationship between τi and Na+-K+-ATPase was recently validated by a study of breast cancer cell lines using magnetic resonance and immunofluorescence measurements (40). The present multicenter data analysis challenge shows that τi (along with ve) not only has the smallest AIF-caused variance among the PK parameter but is also (along with kep) the least sensitive to changes in AIF amplitude. Therefore, the inclusion of the τi parameter (or the use of the SSM) in DCE-MRI studies could be advantageous, especially for studies of therapeutic monitoring when random errors of AIF measurement in multiple exams over time could lead to low accuracy and precision in parameters such as Ktrans and consequently either over- or underestimation of true response to treatment.

Similar to challenge part I (9), this multicenter study has several limitations. The study cohort size is small (11 patients) and the results should be validated with a larger cohort size. Due to the lack of data for R10 measurement in the shared data sets, a fixed R10 value was used for PK analysis of all voxel data across all 11 patients. Although this approach eliminated the contamination of R10 variation in the evaluation of the effect of AIF variation on SSM parameters, the use of a uniformed presumed R10 value most likely reduced the accuracy in the estimated parameter values, as well as in the assessments of intra- and intertumoral heterogeneity. The AIF determination methods used by the 9 QIN centers are constrained to direct measurement from the imaging data. Other AIF quantification methods were not evaluated in this study. It would be interesting to investigate if AIF variations from a method like blinded estimation (10) will have similar effects on PK parameter variance. Because the shared prostate DCE-MRI data sets were all acquired before treatment, it was not possible to assess the effects of AIF variation on DCE-MRI assessment of prostate cancer response to treatment, particularly the comparison of the individually measured AIFs with the population-averaged GP AIF.

Conclusion

The results from this part II of a multicenter DCE-MRI data analysis challenge using the SSM are generally consistent with those obtained using the standard Tofts model (9). Variations in AIF quantification result in considerable variance in the estimated PK parameters. Among the three conventional PK parameters (ie, Ktrans, ve, and kep), the AIF-caused parameter variation is the highest in Ktrans and the lowest in ve. The SSM-specific τi parameter has low AIF-caused variation, similar to ve. Use of the reference tissue method to adjust the amplitude of measured AIF can improve agreement in AIF and reduce variations in Ktrans and ve, but it has little effect on kep and τi. kep may be a more robust and reproducible marker of prostate microvasculature than Ktrans because of its lower sensitivity to AIF uncertainty. Because τi is the least sensitive among the four parameters to AIF variation and has the potential of being an imaging biomarker of metabolic activity, the SSM could be the better choice for PK analysis of DCE-MRI data acquired with sufficient sensitivity to the water-exchange kinetics (41), especially those acquired in longitudinal studies to assess cancer response to treatment. In multicenter quantitative DCE-MRI studies, central data analysis with a fixed AIF determination method should be adopted to minimize parameter variations due to inconsistency in AIF determination by each local site. If local PK data analysis is required, the AIFs used by the local sites need to be consistent: either individually measured from acquired data or a population-averaged AIF, but not both. Furthermore, the reference tissue-adjusted AIF should be used in data modeling to reduce AIF-caused parameter variations.