Meta-analysis of the diagnostic performance of [18F]FDG-PET and PET/CT in renal cell carcinoma.

OBJECTIVES: Positron emission tomography (PET) using fluorodeoxyglucose (FDG) is useful for restaging renal cell carcinoma (RCC) and detecting metastatic diseases but is less satisfactory for detecting primary disease. We evaluated whether the integration of computed tomography (CT) scans with the PET system could increase the applicability of FDG-PET for RCC.
METHODS: The MEDLINE databases were searched for relevant studies published since 2001. Two reviewers independently assessed the methodological quality of each study identified. We then performed a meta-analysis of the sensitivity and specificity of FDG-PET findings as reported in all the selected studies.
RESULTS: Fourteen studies were eligible for inclusion. The pooled sensitivity and specificity of FDG-PET were 62% and 88% respectively, for renal lesions. For detecting extra-renal lesions, the pooled sensitivity and specificity of FDG-PET were 79% and 90%, respectively, based on the scans, and 84% and 91% based on the lesions. The use of a hybrid FDG-PET/CT to detect extra-renal lesions increased the pooled sensitivity and specificity to 91% and 88%, respectively, with good consistency.
CONCLUSIONS: For RCC, combining the FDG-PET and CT systems is helpful for detecting extra-renal metastasis rather than renal lesions. The hybrid PET/CT system has comparable sensitivity and specificity with PET in detecting extra-renal lesions of RCC. ADVANCES IN KNOWLEDGE: The FDG-PET and PET/CT systems are both useful for detecting extra-renal metastasis in renal cell carcinoma.

Introduction

Renal cell carcinoma (RCC) is the most common solid kidney cancer. Due to its high metastatic potential, accurate staging is important to determine the appropriate treatment for a patient[]. Fluorodeoxyglucose (FDG)-positron emission tomography (PET) is widely applied in detecting malignancy and predicting the prognosis, as well as in staging/restaging, and in therapeutic decision-making and monitoring of tumors[,]. However, the application of FDG-PET in the urinary tract is relatively limited because this tract is the major excretion route for FDG, which may mean that background activity obscures the presence of lesions.

Martinez de Llano et al.[] evaluated the performance of FDG-PET in detecting primary, recurrent, and metastatic RCC, as reported in articles published before October 2004. Their meta-analytic study suggested that FDG-PET can be useful in restaging and detecting metastatic disease, but not in detecting primary disease. All 7 studies included in that meta-analysis used FDG-PET, without integrated computed tomography (CT) scans, for imaging. With the development and wider availability of the hybrid PET/CT system, more recent studies have investigated the use of PET/CT. The integrated CT system, although a low-dose CT, is believed to be an improvement over PET alone, as helical CT is the best method to assess a renal mass[].

In this study, we evaluated the performance of the hybrid PET/CT system with regard to RCC. We performed a meta-analysis to see if integrated PET/CT provided a better tool to assist patients with RCC.

Materials and methods

Search strategy and study selection

We conducted MEDLINE searches using combinations of the following items: (a) positron emission tomography (PET) and 18F-FDG or fluorodeoxyglucose; (b) renal cell carcinoma (RCC). The search period was limited to between January 2001 and August 2011. The inclusion criteria were: (a) articles (not abstracts or reviews) whose original language was English; (b) studies that evaluated metastases or primary renal tumors; (c) studies that included a minimum of 12 human patients (no animal models); (d) studies that used dedicated PET (or PET/CT) cameras, not coincidence cameras; (e) patients had undergone PET with 18F-FDG without other radiotracers; (f) clear specification of the reference test was provided; (g) data were included on the validity indexes of diagnostic studies, that is, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), or adequate data had been given so that we could calculate these values.

The exclusion criteria were: (a) duplicated studies; (b) studies outdated by later research; (c) articles published before 2001; (d) there was insufficient data to compare cases and controls, or no detailed information was given on the methodology used to obtain the images, the way the images were analyzed (qualitative or quantitative methods), or on the type of quantitative data used; (e) abstracts of presentations or posters displayed at congresses (due to lack of data or inadequate reporting of methodology); (f) studies in which the reference test used was not clearly specified, or was not valid; (g) validation studies of technique; and (h) review studies.

Data extraction and quality assessment

Two reviewers independently extracted the relevant data from each article and recorded them on a standardized form. Disagreements were resolved by consensus. Reviewers were not blinded with regard to information about the journal name, the authors, the authors’ affiliations, or year of publication, as these precautions have been shown to be unnecessary[]. For each study, the following information was noted: (a) year of publication and origin; (b) sample size; (c) age distribution of the study population; (d) reference standard tests; and (e) imaging details, namely imaging system (PET or PET/CT), methods of analysis (qualitative, quantitative, or both), and number of experts interpreting the images. Both reviewers independently assessed the methodological quality of the selected studies.

The criteria list is shown in Table 1. We mainly followed the recommendations of the Cochrane Methods Working Group on Systematic Review of Screening and Diagnostic Tests[], with some modifications for this specific review. Internal validity criteria (IV) were scored as positive (adequate methods), negative (inadequate methods, potential bias), or unclear (insufficient information was provided for a specific item). Standard performance of FDG-PET was scored positively when the type of PET camera, the dose of FDG, the time between injection and scanning, and the method of reconstruction were clearly described. External validity criteria (EV) were assessed to evaluate generalizability. EV was scored positively if sufficient information was provided so that we could judge the generalizability of the findings. After the consensus meeting, we decided to score all unclear scores as negative. Agreement between both reviewers was quantified by Cohen’s κ[]. Quality scores were expressed as a percentage of the maximum score. Subtotals were calculated for internal and external validity, with a maximum possible score of 6 in each case.

Table 1

Criteria used to assess the methodological quality of the studies

Criteria of validity		Positive score
Internal validity
IV1	Valid reference test	Pathology from biopsy or surgery
IV2	Blind measurement of FDG-PET without knowledge of reference test	Mentioned in publication
IV3	Blind measurement of reference test without knowledge of FDG-PET	Mentioned in publication
IV4	Avoidance of verification bias	Assessment by reference test independent of FDG-PET results
IV5	FDG-PET interpreted independently of all clinical information	Mentioned in publication
IV6	Prospective study	Mentioned in publication
External validity
EV1	Spectrum of disease	All stage of disease
EV2	Demographic information	Age and sex information given
EV3	Inclusion criteria	Mentioned in publication
EV4	Exclusion criteria	Mentioned in publication
EV5	Avoidance of selection bias	Consecutive series of patients
EV6	Standard execution of FDG-PET	Type of camera, dose of FDG, time interval, reconstruction

Statistical analysis

Using the original values provided in the articles, we recalculated the data on the sensitivity, specificity, PPV, and NPV of FDG-PET for detecting both locoregional and distant metastases. We did this to avoid rounding-off effects. For articles that did not present their data according to the TNM classification, the reviewers restaged patients according to the TNM classification if the data presented included sufficient detail[].

Numbers of patients with locoregional metastases and distant metastases were placed in a 2 × 2 table independently by each of the 2 reviewers. If data were available for only a subset of patients, those data were included. Meta-analysis was performed using a weighted averages method with inverse of samples for each study, to pool the sensitivity and specificity for locoregional lymph nodes and distant metastases[]. PPV and NPV were not subjected to this analysis because these values depend on prevalence, which is rarely constant across studies included in a systematic review[]. Summary estimates of sensitivity and specificity were calculated, with 95% confidence intervals (CIs), for the detection of renal or extra-renal lesion of RCC on a scan or lesion basis by FDG-PET. We used the normal approximation to binomial with correction for overdispersion. The heterogeneity test was performed by I-square values and Q statistics. The significance level was set at P < 0.05.

We calculated Spearman correlation coefficients for the diagnostic tests to confirm whether or not the diagnostic odds ratio (DOR) changed according to the diagnostic threshold and to determine the fit with a symmetrical or asymmetrical curve (sROC, where DOR = LR+/LR− (where LR+=senstivity/1−specificity and LR−=(1−sensitivity)/specificity). The Spearman correlation coefficients were not statistically significant, thus we used the symmetrical ROC. Statistical analyses were executed with the Meta-Disc free software package version 1.4.

Results

Literature search

The results of our systematic search of the literature in the MEDLINE database are listed in Table 2. The initial search resulted in 249 articles. By limiting the results to articles involving only human subjects and with only English content, 61 articles were excluded. After excluding all review articles (36 articles), we were left with 152 potential articles to include in our analysis. The 152 articles were screened by the 2 investigators by their titles and abstracts to see if the purpose of the studies fitted our aim. Then the full articles remaining were further reviewed by the 2 investigators according to the inclusion and exclusion criteria. That is, the studies that did not fulfill our inclusion criteria were excluded (e.g. dedicated PET camera not used or study did not evaluate primary or metastatic RCC). Similarly, any studies that matched one of the exclusion criteria were also excluded. A final total of 14 articles relevant to the diagnosis of RCC via PET or PET/CT were selected[]. Llano et al.’s study was confined to the clear cell subtype of RCC. The details of the 14 articles are listed in Table 3.

Table 2

The search strategies used in the MEDLINE database

Strategy	Results
(PET or FDG) and RCC	249
Human	210
English	188
Not review article	152

Table 3

Characteristics of the studies selected for evaluating the diagnostic performance of FDG-PET or FDG-PET/CT in RCC

Author	Year	Designa	Male	Female	Age range (years)	Mean age (median) (years)	Referenceb	FDG	Reviewer	Duration
Ramdave et al.[12]	2001	R	14	11	32–79	61	px	10 mCi	2	–
Chang et al.[13]	2003	R	7	8	23–76	55.6	px	10 mCi	–	–
Miyakita et al.[14]	2002	R	15	4	30–72	57.5	Px	260–370 MBq	–	1997.1–1998.3
Safaei et al.[15]	2002	R	28	8	26–69	54	px, fu	15 mCi	report	1996.2–2000.2
Wu et al.[16]	2002	P	12	6	46–73	/	px, fu	10 mCi	2–3	–
Aide et al.[17]	2003	P	32	21	33–86	60	px, fu	2 MBp/kg	2	2000.3–2002.7
Jadvar et al.[18]	2003	R	18	7	42–81	/	px, fu	10–15 mCi	2	–
Majhail et al.[19]	2003	R	19	5	45–82	(63)	px	8–17.3 mCi	2	–
Kang et al.[20]	2004	R	49	17	28–79	58.8	px, fu	–	1	1995.5–2002.1
Park et al.[21]	2009	R	47	16	31–76	(54)	px, fu	10 mCi	–	2004.5–2006.6
Nakatani et al.[22]	2009	R	18	6	45–78	63	px, fu	370 MBq/130 MBq	≥2	2000.8–2008.1
Kumar et al.[23]	2010	R	55	8	22–88	56.85	px fu	370 MBq	2	2006.1–2009.12
Llano et al.[24]	2010	R	42	16	20–79	62.8	px, fu	2.5 MBq/kg	2	1997.3–2005.12
Ozulker et al.[25]	2011	P	8	10	40–81	57.4	px	370–555 MBq	2	2008.12–2010.6

aP, prospective; R, retrospective.

bStudy reference: fu, clinical follow-up; px, pathology.

Methodological quality assessment

Table 4 presents the methodological quality of the selected studies as assessed by the criteria listed in Table 1. The mean total IV score was 2.4, and the mean total EV score was 4.5. The total percentage score for the combined internal and external validity, expressed as a fraction of the maximum score[], ranged from 41.7% to 83.3%, with a mean of 57.7%.

Table 4

Quality assessment of the studies selected for evaluating the diagnostic performance of FDG-PET or FDG-PET/CT in RCC

Study	Year	IV						EV						Total IV score	Total EV score	% of max score
		IV1	IV2	IV3	IV4	IV5	IV6	EV1	EV2	EV3	EV4	EV5	EV6
Ramdave et al.[12]	2001	+	–	–	+	–	–	+	+	+	–	–	+	2	4	50.0
Chang et al.[13]	2002	+	–	–	+	–	–	–	+	+	–	–	+	2	3	41.7
Miyakita et al.[14]	2002	+	–	–	+	–	–	+	+	+	–	+	–	2	4	50.0
Safaei et al.[15]	2002	+	–	–	+	–	–	–	+	+	–	+	–	2	3	41.7
Wu et al.[16]	2002	+	–	–	–	–	+	+	+	+	–	–	+	2	4	50.0
Aide et al.[17]	2003	+	–	–	+	–	+	+	+	+	–	+	+	3	5	66.7
Jadvar et al.[18]	2003	+	–	–	+	–	–	–	+	+	+	–	+	2	4	50.0
Majhail et al.[19]	2003	+	+	–	+	+	–	+	+	+	–	–	+	4	4	66.7
Kang et al.[20]	2004	+	–	–	+	–	–	+	+	+	–	+	–	2	4	50.0
Park et al.[21]	2009	+	–	–	+	–	–	–	+	+	+	+	+	2	5	58.3
Nakatani et al.[22]	2009	+	–	–	+	–	–	+	+	+	+	+	+	2	6	66.7
Kumar et al.[23]	2010	+	+	–	+	+	–	+	+	+	+	+	+	4	6	83.3
Llano et al.[24]	2010	+	–	–	+	–	–	+	+	+	+	+	+	2	6	66.7
Ozulker et al.[25]	2011	+	–	–	+	–	+	+	+	+	–	+	–	3	4	58.3

Total IV/EV score: total number of the positive scores in IV1–IV6/EV1–EV6. % of max score: ((total IV score + total EV score)/12) × 100%.

The low IV score was due to the fact that none of the articles mentioned whether measurement of the reference test was done without knowledge (i.e. with blinding) of FDG-PET (IV3). Only 2 out of 12 articles mentioned the blindness of the FDG-PET interpretation to the knowledge of the reference test and the clinical information (IV2 and IV5). In addition, only 3 articles mentioned that the study design was prospective (IV6). For any criteria not discussed clearly in the article, the relevant dimensions were scored negatively.

The total EV score was higher for more recent articles. For articles published earlier than 2005, only 1 out of the 9 scored up to 5, whereas only 1 of the articles published after 2008 scored less than 5. The lowest EV score was EV4. Only 1 of the 9 articles published before 2005 mentioned the exclusion criteria. The consecutive selection of patients to avoid selection bias (EV5) was not mentioned in 5 articles, all of which were published before 2004.

Accuracy of FDG-PET

Renal lesion

Four articles evaluated the diagnosis of RCC with renal lesions via FDG-PET. The pooled sensitivity and specificity as well as the sROC curve are shown in Fig. 1. However, Miyakita et al.’s study[] contained zero false-positives and true-negatives, thus it is impossible to include it in sROC curve plotting. For renal lesion detection, the pooled sensitivity was 0.62 (95% CI 0.49–0.74) with high heterogeneity; the chi-square value was 11.71, indicating statistical significance (P = 0.0029), and the I-square value was 82.9%. The pooled specificity was 0.88 (95% CI 0.47–1.00), with a non-significant chi-square value of 1.02 (P = 0.5992) and an I-square value of 0.0%.

Figure 1

Summary of the sensitivity and specificity as well as the sROC curve for detecting renal lesions of RCC by FDG-PET.

Extra-renal lesion

Ten articles used FDG-PET to evaluate extra-renal lesions in RCC. Seven of them evaluated on a scan basis, 2 of them on a lesion basis, and one presented both scan-based and lesion-based results. For the scan-based articles, the pooled sensitivity and specificity were 0.79 (95% CI 0.71–0.86) and 0.90 (95% CI 0.82–0.95), respectively (Fig. 2). Neither of the chi-square scores for the pooled sensitivity and specificity were statistically significant, at 9.00 (P = 0.1734) and 7.64 (P = 0.2657), respectively. The I-square scores for the pooled sensitivity and specificity were 33.4% and 21.5%, respectively. The 3 studies that evaluated the results on a lesion basis were high in heterogeneity (Fig. 3). The pooled sensitivity was 0.84 (95% CI 0.75–0.91) with a significant chi-square value of 22.22 (P < 0.001) and I-square value of 91%. The pooled specificity was 0.91 (95% CI 0.72–0.99) with a non-significant chi-square value of 4.59 (P = 0.1006) and I-square of 56.5%.

Figure 2

Summary of the sensitivity and specificity as well as the sROC curve for detecting extra-renal lesions of RCC on a scan basis by FDG-PET.

Figure 3

Summary of the sensitivity and specificity as well as the sROC curve for detecting extra-renal lesions of RCC on a lesion basis by FDG-PET.

Accuracy of FDG-PET/CT

None of the included articles published before 2011 evaluated primary renal lesions in RCC using FDG-PET/CT. By comparing postoperative pathology, Ozulker et al.[] reported a sensitivity of 46.6%, specificity of 66.6% and accuracy of 50% in 18 cases.

Only 2 articles focused on extra-renal lesions in RCC, but the results of these 2 studies demonstrated good consistency (Fig. 4). The pooled sensitivity was 0.91 (95% CI 0.84–0.96), with a non-significant chi-square score of 0.41 (P = 0.5237) and an I-square score of 0.0%. The pooled specificity was 0.88 (95% CI 0.77–0.94), with a non-significant chi-square score of 0.73 (P = 0.3933) and an I-square score of 0.0%.

Figure 4

Summary of the sensitivity and specificity for detecting extra-renal lesions of RCC on a scan basis by FDG-PET/CT.

Comparison of performance among different modalities

We were unable to meta-analyze renal-based data to compare the performance of PET and PET/CT, as only one article focused on the detection of renal lesions by PET/CT. For the detection of extra-renal lesions based on scans, neither sensitivity nor specificity differed significantly between PET and PET/CT.

For articles focusing on the detection of extra-renal lesions by PET, the results were presented either on a scan basis or on a lesion basis. We found no significant differences in the performance of PET scans in detecting extra-renal lesions according to whether the interpretations were scan-based or lesion-based.

Only 2 sets of performance showed significant differences. These were: (a) the sensitivity of PET in scan-based renal lesion detection versus lesion-based extra-renal lesion detection; and (b) the sensitivity of scan-based detection of renal lesions by PET versus the detection of extra-renal lesions by PET/CT.

Discussion

RCC is the most common solid kidney cancer. The primary modality chosen for diagnosing, staging, and detecting a recurrence of RCC is the contrast CT scan, with an overall accuracy ranging from 61% to 91%[]. However, differentiating between malignant renal masses and benign masses using morphological imaging remains challenging, as RCCs variously appear in CT scans as isodense, hypodense, or hyperdense[]. Magnetic resonance imaging (MRI) is generally reserved for patients for whom CT is contraindicated, such as pregnant women or patients who are allergic to the iodinated contrast medium. Also, it is not so accurate.

FDG-PET is a functional imaging technique used to detect various malignancies via observation of increased glucose uptake and increased rate of glycolysis in neoplastic tissue. The application of FDG-PET in cases of renal cancer was first reported by Wahl et al.[] in 1991. However, the value of FDG-PET in RCC is debated, especially for detecting primary renal tumors[,,,,]. The unfavorable performance of FDG-PET in diagnosing renal tumors may partly be due to the fact that the kidneys are the major excretion route for FDG. This results in high and variable background activity that might obscure the actual lesion. Forced diuresis coupled with parenteral hydration could improve the diagnostic accuracy of FDG-PET in various abdominopelvic malignancies, by enhancing urinary flux[]. Aide et al.[] administered 10 mg of furosemide intravenously 30 min before imaging, but those results showed the lowest rates of sensitivity and specificity among the 4 studies that we reviewed that evaluated renal lesions via FDG-PET. Ozulker et al.[] administered diuretics according to the same protocol as that used by Aide et al.[], and obtained no better results, even with a PET/CT system. Kamel et al.[] also found that forced diuresis did not improve the characterization of space-occupying lesions in the kidney. In the study by Kamel et al.[], up to 60% of the post-diuretic kidneys displayed activity higher than the background level, which may have resulted from the physiologic FDG activity at the renal tubular epithelium[,].

In addition to the interference of background activity, the size and the FDG avidity of the RCC may be partly responsible for the performance of FDG-PET. It is generally accepted that well-visualized tumors are larger than non-visualized ones[,]. Most articles stated that FDG avidity was correlated with GLUT-1 expression and the tumor grading, but some did not specify this[,,]. To sum up, based on the current limited data, FDG-PET is not favorable for diagnosing primary RCCs, even with the assistance of diuretics. We cannot make a conclusion about FDG-PET/CT in the diagnosis of RCCs because of the limited data.

With regard to the ability of FDG-PET to detect extra-renal RCCs, our analysis showed that the pooled sensitivity and specificity were 0.79 and 0.90 when classification was based on scans, and 0.84 and 0.91 when based on lesions. The area under the curve was 0.89 and 0.93, respectively, both better than that in detecting primary renal lesions. These results show that FDG-PET is better able to detect extra-renal rather than renal RCCs, as extra-renal lesions are not obscured by urinary FDG activity. However, FDG-PET may not localize a small lesion as accurately as a larger one. According to a previous study, the sensitivity of FDG-PET increased from 76% to 92.9% when the lesion size increased from more than 1 cm to 2 cm[]. In addition to lesion size, FDG-PET may localize high-grade tumors more efficiently than low-grade ones[,]. These factors would affect the sensitivity of FDG-PET in detecting extra-renal lesions of RCC.

The advancement of hybrid PET/CT imaging has improved the definition of a tumor’s anatomical location. For extra-renal lesions, PET/CT greatly improves the pooled sensitivity without compromising the specificity of tumor detection. A study by Kumar et al.[] found that the most common causes of a false-negative result were either microscopic metastasis or high physiologic uptake in the background activity. In contrast, false-positive results were due to infection (tuberculosis and neurocysticercosis). Additional advantages of PET/CT in detecting locoregional recurrence of RCC are noted for patients with conditions that may influence the interpretation of CT scans, such as postoperative scarring, surgical clips, and migration of adjacent normal organs into the renal fossa[]. Finally, PET/CT can provide an entire body image in one scan, without posing any risk to renal function or possibly incurring an allergic reaction to contrast agents[].

The current study had some limitations. The small number of articles included and the variable quality among them may weaken the findings of our meta-analysis. The predominantly retrospective nature of the studies, and the exclusion of all non-English articles, may have introduced selection bias. The generalizability of our findings may be limited by the clinical heterogeneity among the samples and the diversity in study designs. The difference in publication dates may also be a potential limiting factor. All but 2 articles investigating PET were published before 2004, whereas all the articles investigating the PET/CT system were published after 2008. The results of studies undertaken in these different time periods could have been affected not only by the upgrading of instruments, but also by advances in diagnosis, interpretation, and referencing systems. We recommend that further studies in this area should be carried out, using as large samples as possible, and prospective, randomized, and controlled research designs.

Conclusion

The FDG-PET and CT systems are both useful for detecting extra-renal metastasis in RCC. For detecting extra-renal lesions, the hybrid PET/CT system non-significantly enhances the sensitivity of PET without compromising the specificity. However, further research is required to investigate the ability of PET/CT to detect renal lesions.

Conflicts of interest

No potential conflicts of interest were disclosed.

Table 5

Parameters of the diagnostic accuracy of the studies selected for evaluating the diagnostic performance of FDG-PET or FDG-PET/CT in RCC

Method	Site	Study	Year	Scan	TP	FP	FN	TN	Sensitivity (%)	Specificity (%)
PET	Renal	Ramdave et al.[12]	2001	17	15	0	1	1	94	100
		Miyakita et al.[14]	2002	19	6	0	13	0	32	–
		Aide et al.[17]	2003	35	14	1	16	4	47	80
		Kang et al.[20]	2004	17	9	0	6	2	60	100
		Pooled							62	88
				Scan
	Extra-renal	Ramdave et al.[12]	2001	17	2	0	0	15	100	100
		Chang et al.[13]	2002	15	9	1	1	4	90	80
		Aide et al.[17]	2003	53	10	3	0	40	100	93
		Jadvar et al.[18]	2003	25	15	1	6	3	71	75
		Majhail et al.[19]	2003	24	14	0	7	3	67	100
		Kang et al.[20]	2004	52	46	0	6	0	88	–
		Nakatani et al.[22]	2009	28	17	2	4	5	81	71
		Llano et al.[24]	2010	58	29	3	7	19	81	86
		Pooled							79	90
				Lesion
		Safaei et al.[15]	2002	25	15	2	2	6	88	75
		Wu et al.[16]	2002	52	40	0	0	12	100	100
		Majhail et al.[19]	2003	36	21	0	12	3	64	100
		Pooled							84	91
				Scan
PET/CT	Renal	Ozulker et al.[25]	2011	18	7	1	8	2	47	67
	Extra-renal	Park et al.[21]	2009	35	30	5	2	26	94	84
		Kumar et al.[23]	2010	103	63	3	7	30	90	91
		Pooled							91	88

FN, false-negative; FP, false-positive; TN, true-negative; TP, true-positive.