Do the variations in ROI placement technique have influence for prostate ADC measurements?

Background: Prostate apparent diffusion coefficient (ADC) values calculated from diffusion-weighted imaging have been used for evaluating prostate cancer (PCa) aggressiveness. However, the way of measuring ADC values has varied depending on the study. Purpose: To investigate inter- and intra-reader variability and diagnostic performance of three kinds of shaped 2D regions of interests (ROIs) for tumor ADC measurements in PCa. Material and
Methods: Seventy-four patients with PCa undergoing 3-T MRI before surgery were included. Histologic findings from radical prostatectomy specimens were reviewed to define each patient's dominant tumor. Three readers independently measured the tumor ADCs using three different ROI methods: freehand, large-circle, and small-circles ROIs. Readers repeated measurements after 3 weeks. Bland-Altman analysis was performed to evaluate the inter- and intra-reader variability. Receiver Operating Characteristic analysis was used for assessment of tumor aggressiveness for PCa.
Results: For intra-reader and inter-reader variability, the mean coefficient of repeatability for freehand ROIs, large-circle ROIs, and small-circles ROIs were as follows: 13.7%, 12.4%, and 11.5%; 9.4%, 9.7%, and 9.5%. For differentiating Gleason score (GS) = 3 + 3 from GS ≥ 3 + 4 tumors, the area under the curves were 0.90 for freehand ROIs, 0.89 for large-circle ROIs, and 0.94 small-circles ROIs (p = 0.31).
Conclusion: The variations in ROI method did not have a major influence on intra-reader or inter-reader reproducibility or diagnostic performance for prostate ADC measurements.

Introduction

Prostate cancer (PCa) is the second most common cancer and the fourth leading cause of cancer-related mortality in men worldwide. Low-risk groups have been reported to have an indolent nature, while high-risk groups such as combination of high-risk factors or the International Society of Urological Pathology (ISUP) prognostic score 5 (Gleason grade 9–10) have high mortality rates. Accurate risk stratification, identification of high-risk PCa, and prompt treatment are essential for improving prognosis and reducing mortality of PCa.

Prostate multiparametric magnetic resonance imaging (mpMRI) is now being widely used for PCa detection. Apparent diffusion coefficient (ADC) calculated from diffusion-weighted imaging (DWI) in the mpMRI has played a role as an important imaging biomarker of PCa. ADC values have been used for evaluating PCa aggressiveness in numerous studies.[3-12] However, the way of measuring ADC values has varied depending on the study; many researchers have used round shapes of the region of interest (ROI), while others have used free delineated ROI.[3,5-13] The intratumoral heterogeneity, determined by the presence of multiple cancer cell phenotypes of different grade within a single tumor, is well recognized in PCa.[14,15] Furthermore, PCa can be composed of high density of malignant glands or consist of low density of malignant glands scattered within normal tissue. Because of these characteristics, ADC values within PCa could be heterogeneous. This lack of uniformity in the method of ADC measurements and heterogeneity of PCa may not allow an accurate and reproducible ADC value assessment. Optimization of measurements of ADC values could be critical for accurate risk stratification of PCa. Tamada et al. has reported that use of a 3D ROI did not improve intra-reader or inter-reader reproducibility or diagnostic performance compared with use of a 2D ROI for prostate ADC measurements. However, to the best of our knowledge, none of the studies have evaluated the influence of different-shaped 2D ROIs in prostate imaging.

In this study, we aimed to investigate inter- and intra-reader variability and diagnostic performance of three kinds of shaped 2D ROIs for tumor ADC measurements in PCa.

Materials and methods

This retrospective study was approved by our institute’s ethics committee, and written informed consent was waived.

Patient population

A total of 110 consecutive patients with biopsy-proven PCa underwent 3-T MRI examinations including T2-weighted images and DW images of the prostate followed by radical prostatectomy between May 2013 and January 2015. Total of thirty six patients were excluded as follows: (a) ten patients had a max tumor diameter of <5mm or were not visualized on MRI; (b) nine MR studies were degraded with severe motion artifacts; (c) five patients had inadequate histopathological reports; (d) twelve patients had received hormone (i.e., luteinizing hormone-releasing hormone agonist, antiandrogen, or 5-alpha reductase inhibitors) and radiation therapy before or instead of surgery. These exclusions resulted in a final study cohort of 74 patients (mean age, 63.5 ± 7 [standard deviation] years; mean prostate specific antigen level, 9.12 ± 4.62 ng/mL).

MR imaging

All patients were scanned with a 3-T MR unit (Achieva; Philips Medical Systems, Best, The Netherlands) using a multichannel phased-array (SENSE Cardiac 32ch-coil; Philips Medical Systems) for signal reception. No endorectal coil was used. T2-weighted turbo spin-echo images, covering the entire prostate gland and seminal vesicles, were acquired in two orthogonal planes, axial and coronal. The parameters for axial T2-weighted images were: repetition time (TR)/echo time (TE), 4000/130msec; number of excitations (NEX), 3; echo train length, 16; slice thickness, 3 mm; interslice gap, 0mm; field of view (FOV), 20×20 cm; acquisition voxel size, 0.78×0.78×3.00mm; number of slices, 25. These images were acquired within 3 min 40 s. DW images were obtained in the axial plane using the spin-echo echo-planar imaging sequence with the following parameters: TR/TE, 4900/65msec; flip angle, 90°; NEX, 3; b-values, 0 and 2000 s/mm2; slice thickness, 3mm; interslice gap, 0 mm; FOV, 45×36 cm; acquisition voxel size, 3.52×2.81×3.00 mm; number of slices, 25. The ADC map was generated by the MRI unit console by means of the mono-exponential model. Although T1-weighted images and dynamic contrast-enhanced images were also obtained for clinical examinations, they were not evaluated in this study. Peristalsis was suppressed with intramuscular administration of 20 mg of scopolamine butylbromide (Buscopan; Boehringer Ingelheim, Yamagata, Japan) or 1 mg of glucagon (Glucagon-G Novo; Eisai Co. Ltd, Tokyo, Japan).

Histopathological analysis

Pathological analysis of the radical prostatectomy specimen served as the reference standard. Specimens were cut into 3–4 mm thick axial step-section slices and were handled and processed according to the International Society of Urological Pathology Consensus. The institutional urologic pathologists created pathologic maps of the cancer areas and documented the Gleason score (GS) in all of the cancer foci.

Image analysis and data collection

A study coordinator (A genitourinary radiologist with 22 years of experience in prostate MRI: approximately 300 prostate MRIs interpretation per year) reviewed all MR images and pathological reports of the radical prostatectomy specimen and decided the targeted lesion for each patient. The tumor with the highest GS or the largest one, if multiple tumor foci shared the highest grade, was defined as the targeted lesion. An area showing high signal intensity on DW images and low signal intensity on an ADC map compared with the signal from adjacent tissue corresponding to each patient’s targeted lesion as indicated by the histopathologic result was identified. T2-weighted images were used to assist anatomic cross-referencing between the ADC map and the pathologic map. The study coordinator prepared screen shots of a single slice of the largest section of each targeted lesion for reference.

Three board-certificated radiologists (6 [Y. U.], 10 [K. S.], and 20 years of experience [T. T.] in prostate MRI) who had no knowledge of either the histopathologic findings or the clinical data retrospectively analyzed the images. Two of three (Y.U. and T.T.) interprets approximately 300 prostate MRIs per year, and the other (K.S.) interprets approximately 120 prostate MRIs per year. The radiologists were provided with a PowerPoint file with the screen shots prepared by the study coordinator depicting the location of the targeted lesions. Three readers independently measured each lesion’s ADC using three different ROI methods: freehand ROI, large-circle ROI, and small-circles ROIs on the largest cross-sectional areas. The freehand ROI was drawn along the border of the low signal comprising the tumor to cover the entire tumor area. Large-circle ROI was defined to be as large as possible for the target lesion. The two or three small, circle ROIs (>3 mm2) were positioned within the same slice but not overlapping each other, and the mean value was subsequently calculated (referred to as Small-circles ROI). The representative figure for each ROI is shown in Fig. 1. Readers re-measured each ROI 3 weeks later. Delineation of all ROIs was done using Osirix DICOM (Digital Imaging and Communications in Medicine) Viewer.

Fig. 1.

ADC measurements of prostate cancer (PCa) on an ADC map using three different ROI protocols. (a) Freehand ROI: ROI was drawn along the border of the low signal comprising the tumor to cover the entire tumor area. (b) Large-circle ROI: ROI was defined to be as large as possible for the target lesion. (c) Small-circles ROI: Small ROIs were positioned within the same slice but not overlapping each other.

Statistical analyses

Bland-Altman analysis was used to derive the coefficient of repeatability (CoR) for intra-reader and inter-reader reproducibility. The range defined by ± CoR describes the 95% limits of agreement between two measurements and represents a 95% CI for the percentage difference between replicate measures provided for any one randomly selected patient such that lower CoR indicates higher reproducibility. For intra-reader variation, the analysis was performed for the three readers individually and combining all reader data. For inter-reader variation the analysis included the individual assessments from both sessions in one overall analysis with session number included as a fixed classification factor.

Receiver operating characteristic (ROC) analysis was used for the evaluation of differences between each of the ROI methods for separating tumor with GS = 3 + 3 from GS ≥ 3 + 4 tumors. The areas under the ROC curves (AUC) were estimated non-parametrically for ordinal score assessments. The sensitivity, specificity, and accuracy were calculated using Youden index. The pooled AUC, sensitivity, specificity, and accuracy were also estimated. All statistical tests except for pooled AUC were performed at the two-sided 5% significance level with SAS software (version 9.2; SAS Institute, Cary, NC, USA). The pooled AUC of the first and second session for each reader was calculated using MedCalc for Windows, version 19.4 (MedCalc Software, Ostend, Belgium).

Results

The GSs assigned at radical prostatectomy were as follows: 3 + 3 (n = 6), 3 + 4 (n = 27), 4 + 3 (n = 26), 3 + 5 (n = 1), 4 + 4 (n = 10), and 4 + 5 (n = 4). The pathological stages were as follows: pT2a (n = 18), pT2b (n = 7), pT2c (n = 24), pT3a (n = 20), pT3b (n = 5). The areas and ADCs of each ROI are shown in Tables 1 and 2.

Table 1.

The area of each ROI.

	Freehand ROI (mm2)	Large-circle ROI (mm2)	Small-circles ROI (mm2)
The first analysis
Reader 1	70.5 ± 60.7	48.9 ± 44.0	11.2 ± 8.61
Reader 2	77.0 ± 58.8	37.8 ± 33.9	15.1 ± 9.41
Reader 3	71.0 ± 59.1	41.9 ± 37.6	11.1 ± 7.21
The second analysis
Reader 1	68.1 ± 69.2	57.1 ± 50.1	10.3 ± 7.1
Reader 2	83.5 ± 58.1	38.1 ± 31.7	14.7 ± 8.39
Reader 3	73.5 ± 60.2	45.5 ± 43.0	10.0 ± 7.00

Note. Data are mean ± standard deviation.

ROI: region of interest.

Table 2.

The ADC of each ROI.

	Freehand ROI	Large-circle ROI	Small-circles ROI
The first analysis
Reader 1	753.5 ± 152.1	722.6 ± 152.2	687.7 ± 161.7
Reader 2	753.3 ± 151.6	702.2 ± 158.8	693.5 ± 183.9
Reader 3	745.1 ± 156.2	711.6 ± 157.5	698.0 ± 161.9
The second analysis
Reader 1	748.7 ± 149.8	721.6 ± 157.7	688.7 ± 166.0
Reader 2	772.3 ± 142.8	718.5 ± 147.4	719.0 ± 159.3
Reader 3	750.0 ± 147.5	715.7 ± 150.6	697.2 ± 151.5

Note. Data are mean ± standard deviation.

ADC: apparent diffusion coefficient; ROI: region of interest.

Table 3 shows the intra-reader and inter-reader reproducibility, as indicated by CoR, for each ROI method. For intra-reader variability, the absolute difference in CoR among three different shaped ROIs was less than 3% for the three individual readers. For inter-reader variability, the mean CoRs among readers were 9.4% for freehand ROIs, 9.7% for large-circle ROIs and 9.5% for small-circles ROIs.

Table 3.

Inter- and Intra-observer agreement.

Reproducibility	Freehand ROI, %	Large-circle ROI, %	Small-circles ROI, %
Inter-reader	Coefficient of Repeatability
Reader 1 vs 2	8.7%	10.8	10.3
Reader 2 vs 3	9.6%	7.8	9.8
Reader 1 vs 3	10.0%	10.6	8.4
Intra-reader	Coefficient of Repeatability
Reader 1	9.5%	12.0	9.1
Reader 2	8.9%	9.0	10.8
Reader 3	13.7%	12.4	11.5

ROI: region of interest.

The pooled diagnostic performance for separating GS = 3 + 3 tumor from GS ≥ 3 +4 tumor are shown in Table 4. The pooled AUCs for freehand ROI, large-circle ROI and small-circles ROI were achieved more than 0.90 and there was no significant difference for all readers (Table 5). The pooled sensitivity, specificity, and accuracy for each reader were also shown in Table 4; 0.76–0.87, 1.00, and 0.78–0.89 for freehand ROI, 0.77–0.86, 1.00, and 0.78–0.88 for large-circle ROI, and 0.78–0.86, 1.00, and 0.80–0.89 for small-circles ROI, respectively.

Table 4.

The pooled diagnostic performances for differentiating Gleason score ≧ 7 from 6

		Freehand ROI	Large-circle ROI	Small-circles ROI
Reader 1	AUC	0.91 [0.84–0.98]	0.91 [0.83–0.98]	0.95 [0.89–1.00]
	Sensitivity	0.78 [0.70–0.84]	0.79 [0.71–0.85]	0.82 [0.74–0.88]
	Specificity	1.00 [0.65–1.00]	1.00 [0.65–1.00]	1.00 [0.65–1.00]
	Accuracy	0.79 [0.71–0.85]	0.80 [0.73–0.86]	0.83 [0.76–0.89]
Reader 2	AUC	0.91 [0.83–0.98]	0.91 [0.83–0.99]	0.92 [0.85–0.99]
	Sensitivity	0.76 [0.68–0.83]	0.77 [0.69–0.83]	0.78 [0.70–0.84]
	Specificity	1.00 [0.65–1.00]	1.00 [0.65–1.00]	1.00 [0.65–1.00]
	Accuracy	0.78 [0.70–0.84]	0.78 [0.70–0.84]	0.80 [0.73–0.86]
Reader 3	AUC	0.91 [0.83–0.99]	0.94 [0.89–0.99]	0.95 [0.91–0.99]
	Sensitivity	0.87 [0.80–0.92]	0.86 [0.78–0.91]	0.86 [0.79–0.91]
	Specificity	1.00 [0.65–1.00]	1.00 [0.65–1.00]	1.00 [0.65–1.00]
	Accuracy	0.89 [0.83–0.93]	0.88 [0.81–0.92]	0.89 [0.82–0.93]

Note. Data in the brackets indicate the 95% confidence interval.

AUC: area under the curve; ROI: region of interest.

Table 5.

The diagnostic performances for differentiating Gleason score ≧ 7 from 6 for each reader.

	Freehand ROI	Large-circle ROI	Small-circles ROI
	AUC [95% C.I.] (Threshold)			P value
Reader 1 1st session	0.91 [0.82–1.00] (785.8)	0.91 [0.80–1.00] (780.0)	0.95 [0.88–1.00] (799.3)	0.20
Reader 1 2nd session	0.91 [0.83–0.98] (776.0)	0.89 [0.78–0.99] (781.0)	0.92 [0.83–1.00] (763.0)	0.56
Reader 2 1st session	0.90 [0.80–0.96] (804.0)	0.92 [0.83–0.97] (743.0)	0.93 [0.85–0.98] (784.0)	0.34
Reader 2 2nd session	0.91 [0.83–0.97] (820.0)	0.90 [0.80–0.96] (759.0)	0.91 [0.82–0.97] (794.0)	0.45
Reader 3 1st session	0.94 [0.86–0.98] (867.0)	0.94 [0.85–0.98] (787.0)	0.93 [0.85–0.98] (765.9)	0.72
Reader 3 2nd session	0.93 [0.85–0.98] (862.9)	0.94 [0.86–0.98] (847.7)	0.95 [0.88–0.99] (843.3)	0.10

AUC: area under the curve; ROI: region of interest.

Discussion

Our study showed the effect of 2D-ROI methods on ADC measurement in prostate cancer, using radical prostatectomy as the reference standard. The all three 2D-ROIs had similar intra- and inter-reader reproducibility, and diagnostic performance for PCa with GS ≥ 3 +4 from PCa with GS = 3 + 3. Such consistency is important when applying specific ADC thresholds in clinical examinations or when following serial ADC values in individual patients undergoing surveillance or those who have undergone targeted therapy.

In a previous study evaluating ADC measurements for PCa, Tamada et al. has reported that the use of a 3D-ROI did not improve intra- or inter-reader reproducibility, or diagnostic performance compared with use of a 2D-ROI. However, we are unaware of previous studies in which a detailed evaluation of 2D-ROIs for prostate cancer evaluation was performed. Previous studies have evaluated ROI methods in ADC measurement in other tissues and had mixed results.[20,21] Lambregts et al. reported that the interclass correlation coefficient (ICCs) were moderate with single-slice freehand and round ROIs, although whole-volume ROI offered excellent reliability for locally advanced rectal cancer. For endometrial carcinoma, Inoue et al. reported that four kinds of 2D-ROI methods (freehand ROI; square ROI; round ROI; and five small, round ROIs) had no marked influence on ICCs. They assumed that since the shape of endometrial carcinoma is close to oval or round due to the tumors mainly existing in the intrauterine cavities, the four different ROI methods may not have made a significant difference. Similar results may have been obtained in our study because the shape of the target lesions was mostly round or oval in prostate cancer.

PCa exhibits a phenomenon whereby tumors often comprise of intermixed benign and malignant regions without distinct separation. However, this property may not introduce an element of uncertainty, when readers attempt to place an ROI on the slice on which the lesion is most clearly visualized. Compared with freehand ROI and small-circles ROIs, large-ROI is suggested to be a simpler method.

Our study had a few limitations. First, we did not evaluate the diagnostic performance for differentiating PCa from non-cancerous lesions since we aimed to investigate the influence of different-shaped 2D ROIs on tumor ADC measurements in PCa. Second, complicated analysis methods such as texture analysis were not performed. However, this point should not be critical in the clinical routine. Finally, this was a single-center retrospective study with a relatively small number of patients. With the ADC normalization technique based on the previous reports, the validation study with multi-venders or different b-values might be possible. Further studies with larger populations and multi-centers might be necessary to have robust results.

In conclusion, the variations in the ROI methods had no marked influence on intra- or inter-reader reproducibility, or diagnostic performance for ADC measurements in PCa. Large-circle ROI is suggested to be a simpler and suitable method for ADC measurement in PCa in a clinical setting.