ImmunoCyt test compared to cytology in the diagnosis of bladder cancer: A meta-analysis.

The aim of the present study was to evaluate the diagnostic value of the ImmunoCyt test compared with urine cytology in detecting bladder cancer. A systematic literature search was performed to locate all publications reporting on the diagnostic accuracy of the ImmunoCyt test for bladder cancer. Data were extracted from 2×2 tables or calculated from reported accuracy data. Collected data were meta-analyzed for sensitivity, specificity, positive likelihood ratio (LR), negative LR, diagnostic odds ratio (DOR), and summary receiver operator characteristic (sROC) curve analysis. We applied the Meta-DiSc 1.4 and STATA 13.0 software to the meta-analysis. Seven separate studies consisting of 1,602 patients with bladder cancer were considered in the meta-analysis. We found that the ImmunoCyt test had a higher sensitivity than the urine cytology test [0.725, 95% confidence interval (CI) 0.683-0.765 vs. 0.566, 95% CI, 0.521-0.611], but the specificity, positive LR, negative LR, DOR, area under the curve (AUC) and Q index of the ImmunoCyt test were lower compared with the urine cytology test. In addition, the pooled sensitivity, specificity, positive LR, negative LR, DOR, AUC, and Q index of the combined method (combination of ImmunoCyt and cytology) were 0.833, 0.644, 2.804, 0.228, 13.50, 0.8554 and 0.7863, respectively. The results of the Eggers test showed no publication bias (P>0.05). In conclusion, specificity, positive LR, negative LR, DOR, the AUC, and the Q index of the urine cytology test may be superior to the ImmunoCyt test, but the ImmunoCyt test has greater sensitivity than the urine cytology test. Use of ImmunoCyt and cytology in combination has the potential to improve the sensitivity and promises to be an alternative in the detection of bladder cancer.

Introduction

Bladder cancer, the most common malignancy of the urinary tract, poses a threat to an increasing number of humans (1). Progression of bladder tumors greatly increases the risk of metastasis and subsequent mortality. Consequently, early detection is crucial for improving patient prognosis and long-term survival.

Currently, cystoscopy is considered the gold standard for the detection of primary tumors and also the follow-up of patients after transurethral resection of bladder tumors, despite being an invasive approach (2). Urine cytology, another widely used detection method, has been firmly established as a useful adjunct, not only in the diagnosis, but also the follow-up of patients with bladder cancer (3). However, it is highly grade-dependent, and in low-grade cancers, it seems to not be sensitive enough. In addition, urine cytology also suffers from a high inter- and intra-observer variability, which limits its validity in screening for bladder tumors (4,5).

With the aim of improving the lower sensitivity of cytology and increasing the detection of low-grade tumors, the uCyt+/ImmunoCyt test was developed by Fradet and Lockhard, it uses three monoclonal antibodies to detect urothelial cells in voided urine (6). A number of previous studies demonstrated its high sensitivity, and some investigators suggested that it also plays an important role in predicting urothelial tumor recurrence (7–11). The uCyt+/ImmunoCyt test has been in limited clinical and research use for a decade, but its diagnostic accuracy has never been subjected to a quantitative review. We therefore performed a systematic review and meta-analysis of uCyt+/ImmunoCyt test to fully assess its diagnostic accuracy in detecting bladder cancers and to guide future implementation.

Materials and methods

Search methodology

We searched Medline and PubMed for articles that reported on the diagnostic sensitivity and specificity of the uCyt+/ImmunoCyt test for bladder cancers. The keywords used for the search were: (uCyt+/ImmunoCyt OR ImmunoCyt), (bladder cancer or urothelial tumor), sensitivity, specificity, and cytology. Only data from original published papers were collected, with meeting or conference abstracts being excluded. Primary sources from tracking references were also obtained from manual searches in review papers and original articles. Electronic databases were retrieved by two independent investigators. If the assessments of the investigators were not consistent, a discussion ensued whether to include the data or not. The articles included contained studies of the patients with bladder cancer diagnosed using the uCyt+/ImmunoCyt and cytology tests. The effect sizes of the studies seen as odds ratio (OR), sample size, gender, or range of age did not exclude any articles. Studies that only described the uCyt+/ImmunoCyt test or cytology data were excluded.

Data extraction

Test performance data were extracted as a 2×2 table of true-positive (TP), false-positive (FP), false-negative (FN) and true-negative (TN) values directly from tabulated results (Table I). If these were not directly available, they were calculated from reported sensitivity, specificity, positive predictive value (PPV) and/or negative PV (NPV).

Table I.

Characteristics of studies included in the meta-analysis (n=7).

				uCyt+/ ImmunoCyt				Cytology				Combination of both
Author (refs.)	Year	Sample size	STARD score	TP	FP	FN	TN	TP	FP	FN	TN	TP	FP	FN	TN
Yafi et al (18)	2014	109	18	52	5	31	21	36	4	47	22	54	6	29	20
Soyuer et al (19)	2009	90	15	45	5	9	31	41	12	13	24	48	5	6	31
Horstmann et al (20)	2009	221	16	82	30	31	78	95	41	18	67	105	47	8	61
Têtu et al (7)	2005	870	19	100	281	36	453	39	17	97	717	114	284	22	450
Toma et al (21)	2004	120	15	33	20	9	58	36	16	6	62	37	21	5	57
Hautmann et al (22)	2004	94	14	19	16	11	48	22	13	8	51	25	9	5	55
Sullivan et al (11)	2009	98	18	20	26	6	46	5	2	21	70	20	26	6	46

TP, true positive; FP, false positive; FN, false negative; TN, true negative; STARD, Standards for the Reporting of Diagnostic Accuracy Studies.

Data analysis

The estimates of sensitivity, specificity, likelihood ratios (LRs), diagnostic odds ratio (DOR), and their 95% confidence intervals (95% CIs) were pooled for each study. The within- and between-study variation or heterogeneity were assessed by testing Cochrans Q-statistic. Heterogeneity was calculated using the formula I2 = 100% × (Q - df)/Q (12). A significant Q-statistic (P<0.10) or I2-statistic (I2>50) indicated heterogeneity across studies, and then the random effect model (REM) was used; otherwise, the fixed effect model (FEM) was used. The pooled estimate of ORs was obtained using the Mantel and Haenszel method for FEM, the DerSimonian and Laird and DerSimonian and Kacker method for REM (13–15).

The summary receiver operator characteristic (sROC) curve was used to graphically determine performance following testing for correlation between sensitivity and specificity [as the logit TP rate (TPR) vs. the logit FP rate (FPR)] (16).

The area under the curve (AUC) and an index Q were discussed as potentially useful summaries of the curve. An upper bound was derived for the AUC based on an exact analytic expression for the homogeneous situation, and a lower bound based on the limit case Q, defined by the point where sensitivity equals specificity: Q is invariant to heterogeneity (17).

Analyses were performed using the freeware Meta-Disc version 1.4 (Universidad Complutense, Madrid, Spain) and STATA package version 13.0 (Stata Corporation, College Station, TX, USA). All P-values were two-sided. P<0.05 was considered to indicate a statistically significant difference.

Results

In total, 45 potentially relevant studies were identified (Fig. 1). During the abstract screening, 15 articles were excluded. Thirty studies were considered for full publication review. Of these, 23 were excluded (nine because there were no data available, eight because they were reviews, one because it was a non-English article, five because they only included cytology or ImmunoCyt data and no comparison). Seven studies were eventually analyzed (Table I) (7,11,18–22). These studies were published between 2004 and 2014, and included a total of 1,602 patients with bladder cancer. Their sample size ranged from 90 to 870. In these seven studies, a combination of cystoscopy and biopsy was used as the gold standard of bladder cancer.

Figure 1.

Flow diagram for selection of studies for the meta-analysis.

Assessment of study quality

Studies meeting the criteria were quality assessed using positive scoring in a modified 23-point Standards for the Reporting of Diagnostic Accuracy Studies (STARD) proforma (23,24). Two readers (F.T. and M.M.) independently assessed the included studies according to the prearranged proforma. An open discussion was subsequently held in order to resolve any disagreement between the readers. STARD scores for each study are provided in Table I.

Pooled sensitivity, specificity, LR+, LR-, and DOR

We summarized the overall meta-analysis of bladder cancer patients with the ImmunoCyt test, cytology test, and combined method (Combination of ImmunoCyt and Cytology) (Table II). The REM or FEM was used to combine the data of TP, FP, FN, and TN numbers.

Table II.

The indices of bladder cancer diagnosed by ImmunoCyt test and cytology test.

		Test of association		Test of heterogeneity				Egger's test for publication bias
Diagnostic methods	Parameter	Estimates	95% CI	Q	P-value	I2 (%)	Model	t	P-value
uCyt+/ImmunoCyt	Sensitivity	0.725	0.683–0.765	9.66	0.140	37.9	FEM	2.56	0.051
	Specificity	0.657	0.629–0.685	23.23	0.001	74.2	REM
	Positive LR	2.578	2.003–3.318	17.56	0.007	65.8	REM
	Negative LR	0.385	0.327–0.452	8.43	0.208	28.9	FEM
	DOR	7.114	4.709–10.748	11.24	0.081	46.6	REM
Cytology	Sensitivity	0.566	0.521–0.611	132.48	0.000	95.5	REM	−0.72	0.505
	Specificity	0.906	0.887–0.923	161.32	0.000	96.3	REM
	Positive LR	3.862	2.347–6.353	36.95	0.000	83.8	REM
	Negative LR	0.459	0.320–0.658	66.46	0.000	91.0	REM
	DOR	10.269	7.501–15.795	8.77	0.187	31.6	FEM
Combination of both	Sensitivity	0.833	0.796–0.865	28.17	0.000	78.7	REM	1.80	0.132
	Specificity	0.644	0.615–0.672	34.10	0.000	82.4	REM
	Positive LR	2.804	2.163–3.636	23.59	0.001	74.6	REM
	Negative LR	0.228	0.149–0.350	21.42	0.002	72.0	REM
	DOR	13.50	7.847–23.238	14.56	0.024	58.8	REM

CI, confidence interval; LR, likelihood ratio; DOR, diagnostic odds ratio; FEM, fixed effect model; REM, random effect model.

ImmunoCyt test

The pooled sensitivity of the ImmunoCyt test for all seven studies included in the final meta-analysis was 72.5% (95% CI, 68.3–76.5%), the pooled specificity 65.7% (95% CI, 62.9–68.5%), the pooled positive LR was 2.578 (95% CI, 2.003–3.318), the pooled negative LR was 0.385 (95% CI, 0.327–0.452), and the pooled DOR was 7.114 (95% CI, 4.709–10.748) (Table II).

Cytology test

The pooled sensitivity of cytology test was 56.6% (95% CI, 52.1–61.1%), the pooled specificity was 90.6% (95% CI, 88.7–92.3%), the pooled positive LR was 3.862 (95% CI, 2.347–6.353), the pooled negative LR was 0.459 (95% CI, 0.320–0.658), and the pooled DOR was 10.269 (95% CI, 7.501–15.795) (Table II).

Combined method (ImmunoCyt and cytology)

We also collected the pooled meta-analysis of bladder cancer patients with the combined method. The pooled sensitivity was 83.3% (95% CI, 79.6–86.5%), the pooled specificity was 64.4% (95% CI, 61.5–67.2%), the pooled positive LR was 2.804 (95% CI, 2.163–3.636), the pooled negative LR was 0.228 (95% CI, 0.149–0.350), and the pooled DOR was 13.50 (95% CI, 7.847–23.238) (Table II).

AUC and Q index in the three tests

The AUC and Q index of ImmunoCyt test were 0.7910 and 0.7280, those of the cytology test 0.8239 and 0.7570, and those of the combined method 0.8554 and 0.7863, respectively (Figs. 2–4).

Figure 2.

The summary receiver operating characteristic (sROC) curve of ImmunoCyt test.

Figure 4.

The summary receiver operating characteristic (sROC) curve of combined test.

Publication bias

Eggers test was used to assess the publication bias. For all samples, Eggers test provided no evidence of publication bias for this meta-analysis. Detailed information is provided in Fig. 5.

Figure 5.

Egger's publication bias plots.

Discussion

As a useful adjunct for cystoscopy, cytology has been used for more than 60 years for the diagnosis of urothelial carcinoma (UC). The test suffers from low sensitivity (38–51%), but boasts a high specificity (94–98%), particularly for high-grade disease (25). The reliability of cytology results depends on sample quality and cytopathologist experience to some extent. Atypical results are often found in urine cytology tests (6). Among these atypical cytology results, biopsy reveals that malignancy underlies 23–68% of them. Investigators have also found that its sensitivity is highly associated with tumor stage. For patients with higher-grade bladder cancer, it appears to be more sensitive (26–29).

With the aim of improving the low sensitivity of cytology and the detection of low-risk UC, many voided urine biomarker assays such as UroVysion, NMP22, BTA TRAK, and ImmunoCyt/uCyt+ have been developed but have yet to gain widespread clinical application. The ImmunoCyt test performance has been well studied since its introduction by Fradet and Lockhard in 1997 (6,30). It is a triple immunofluorescent monoclonal antibody assay associated with UC, approved by the US Food and Drug Administration. A previous series reported a sensitivity of 74–87% and specificity of 62–78%, with PPVs and NPVs of 26–67% and 91–96%, respectively (6).

Additionally, the ImmunoCyt test can also provide important information in evaluating hematuria, particularly in patients with negative imaging and cystoscopy but atypical cytology (31).

Many studies have reported on the diagnostic accuracy of the ImmunoCyt and cytology tests in detecting bladder cancer (29–32). The results of those studies, however, have been met with skepticism due to their small sample sizes or low statistical power. In the present meta-analysis, we combined seven separate studies, consisting of 1,602 patients to compare the diagnostic accuracy of the ImmunoCyt test with the urine cytology test in detecting bladder cancer. We found that the ImmunoCyt test had a higher sensitivity than the urine cytology test, but the specificity, positive LR, negative LR, and DOR of the ImmunoCyt test were lower compared with the urine cytology test. In addition, the AUC and Q index of cytology were superior to those of the ImmunoCyt test. Based on these comparisons, we can conclude that the ImmunoCyt test would not replace the urine cytology test in detecting bladder cancer.

We also collected data of a combined method (combination of ImmunoCyt test and cytology). The pooled sensitivity, specificity, positive LR, negative LR, DOR, AUC, and Q index were 0.833, 0.644, 2.804, 0.228, 13.50, 0.8554 and 0.7863, respectively. These data suggest that the ability of cytology to detect malignancies of the bladder can be highly improved in combination with ImmunoCyt. The results of the present study are in agreement with previous findings of Mian et al regarding the clinical usefulness of combining the two tests (32). Using the uCyt1/ImmunoCyt test in combination with cytology to detect bladder cancer appeared to improve the overall sensitivity for cytology alone. Furthermore, the use of the two tests in combination constitute a potential alternative option to limit the amount of cystoscopic evaluations required for the follow-up of patients with low-risk bladder cancer.

One limitation of our study is heterogeneity. The considerable amount of heterogeneity detected between the studies suggests a need for caution when pooling the diagnostic accuracy measures together. Additionally, between-study heterogeneities may distort the meta-analysis. The degree of heterogeneity is one of the major concerns in a meta-analysis, as non-homogeneous data are liable to result in misleading results (33). Different populations also contribute to the heterogeneity. Therefore, the results of this meta-analysis should be interpreted with caution. We minimized the likelihood of bias by developing a detailed protocol prior to initiating the study, by performing a rigorous search of published studies and by using explicit methods for study selection, data extraction, and data analysis.

Another limitation common to diagnostic meta-analyses is the lack of clarity, quality, and standardization in the methodology of diagnostic studies. Studies were assessed for quality using STARD proforma to quantify the methodology of the study design. Consequently, we excluded certain studies due to ambiguity between the raw data and the diagnostic accuracy data in the process of data extraction.

In conclusion, in this meta-analysis, we have evaluated the pooled sensitivity, specificity, positive LR, negative LR, DOR, AUC, and Q index of three tests (ImmunoCyt, cytology, and combined test) from seven studies. We found that ImmunoCyt was superior to cytology only in sensitivity, however, inferior to cytology in specificity, positive LR, negative LR, DOR, AUC, and Q index. In conclusion, cytology remains integral in the detection of bladder cancer. Meta-analysis of the combined tests suggests that the use of ImmunoCyt and cytology in combination significantly improves the sensitivity for detecting bladder cancer and promises to be an alternative option in the clinical setting.