Diagnostic accuracy of the UBC® Rapid Test for bladder cancer: A meta-analysis.

Bladder cancer is one of the most common cancer types globally. The UBC® Rapid Test is a potential novel diagnostic method for bladder cancer, but studies into its accuracy have produced inconsistent results. Thus, the present meta-analysis was conducted in order to determine the overall accuracy of the UBC® Rapid Test in detecting bladder cancer. A comprehensive literature search was conducted using MEDLINE, Embase, Cochrane Library, Web of Science, Chinese WanFang and the China National Knowledge Infrastructure databases for relevant studies. Quality assessment of diagnostic accuracy studies 2 was used to assess the quality of each included study. The diagnostic accuracy of the UBC® Rapid Test was evaluated by pooled sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic odds ratio (DOR) and the area under the curve (AUC). In addition, Deeks' funnel plot was used to evaluate potential publication bias. Eight studies were included in the quantitative meta-analysis. The results were as follows: Sensitivity 0.59 [95% confidence interval (CI), 0.55-0.62], specificity 0.76 (95% CI, 0.72-0.80), PLR 2.55 (95% CI, 1.75-3.70), NLR 0.56 (95% CI, 0.46-0.67), DOR 4.88 (95% CI, 2.82-8.45) and AUC 0.70 (95% CI, 0.67-0.74). According to the present results, the UBC® rapid test is highly accurate in the diagnosis of bladder cancer, however, further studies with better-designed and larger samples are required in order to support the results of the present study.

Introduction

Bladder cancer was the ninth most common cancer type globally and the second most common urogenital malignancy in 2012 (1). More than 60% of bladder cancer cases occur in less well-developed countries, including China, and 75% of these cases occur in men (2). Furthermore, bladder cancer has a high recurrence rate (50%), and 15–40% of cases develop into a muscle-invasive form of the disease (3,4). Therefore, early diagnosis and consistent follow-up are necessary in order to improve patient quality of life.

Previously, the primary methods used to detect and follow up bladder cancer were cystoscopy and cytology (5). Cystoscopy is able to identify the majority of papillary and solid lesions and is therefore widely used. Cystoscopy combined with pathological biopsy is the gold standard for the diagnosis and follow-up of bladder cancer (6). However, it is not only an invasive procedure but also has limited accuracy in detecting certain lesions, particularly small areas of carcinoma in situ (7). While cytology has a specificity of >90%, its sensitivity is <44%, particularly in highly-differentiated tumor types (stages G1-G2) (8–10). Therefore, the invasive nature of cystoscopy and the low sensitivity of cytology limit the early diagnosis of bladder cancer in clinical practice. Consequently, a non-invasive, highly sensitive and specific alternative test is urgently required.

To identify a better method to diagnose bladder cancer, various urine-based tumor markers have been extensively investigated (7). These markers, including human complement factor H, cytokeratin 19 fragments and nuclear matrix protein 22, generally demonstrate a higher sensitivity but lower specificity compared with cytology (9,11,12). Biomarker diagnosis has not yet been recommended in the European Association of Urology guidelines (13). Recently, a novel non-invasive qualitative immunochromatographic test has been launched to identify the urinary bladder cancer antigen. The UBC® Rapid Test (Concile GmbH, Freiburg, Germany) is a point-of-care test, compliant with International Organization for Standardization 22870:2016, which may quantitatively measure fragments of cytokeratins 8 and 18 (14–16). These cytokeratins are located in the cytoskeleton of epithelial cells and tend to be overexpressed in urothelial tumor types including bladder cancer (17–19). Based on this, several trials have been performed to investigate the efficacy of the UBC® Rapid Test in the detection and follow-up of bladder cancer (20–22). However, differences in the design and enrollment of these studies have resulted in inconsistent conclusions, so its diagnostic accuracy remains unclear.

In the present study, these previous studies were systematically reviewed to assess the diagnostic value of the UBC® Rapid Test in the detection and follow-up of bladder cancer.

Materials and methods

Search strategy

The following databases were comprehensively searched for studies published between January 1, 1990 and June 1, 2017: Pubmed, Embase, the Cochrane Central Register of Controlled Trials (CENTRAL), Web of Science, China (WANFANG) and the China CNKI database. The search was performed using the following keywords in combination: (‘UBC’ OR ‘Cytokeratin 8’ OR ‘Cytokeratin 18’) AND (‘Bladder cancer’ OR ‘urinary bladder neoplasm’) as medical subject headings. Furthermore, the reference lists of all studies included in the meta-analysis were also reviewed for possible inclusion.

Inclusion and exclusion criteria

The inclusion criteria were as follows: i) Case-control or cohort design; ii) sufficient data for meta-analysis [true positive (TP), false positive (FP), false negative (FN) and true negative (TN)]; iii) if data or subsets of data were used in more than one article, the most recent article or the one with greater detail was selected; and iv) written in English or Chinese. The exclusion criteria were as follows: i) Reviews, case reports and letters to editors; ii) duplicate publications; iii) studies in languages other than English or Chinese; and iv) studies with insufficient data to construct a 2×2 table. All records were independently reviewed by Dr Pei Lu and Dr Rijin Song (Department of Urology, The First Affiliated Hospital of Nanjing Medical University, Nanjing, China). Consensus was normally reached for each eligible study and any disagreements were resolved by consultation with a third reviewer (Dr Min Gu; Department of Urology, The First Affiliated Hospital of Nanjing Medical University).

Data extraction and quality assessment

Relevant data were extracted from the full text of the included studies and included: First author, publication year, ethnicity, sample size, mean age, sex, specific details of index test used, sensitivity and specificity, TP, FP, FN and TN for various grades of bladder tumor types.

The quality assessment of diagnostic accuracy studies 2 (QUADAS-2) scale was used to evaluate the quality of the eligible studies (23). This contains four domains including patient selection, index test, reference standard and flow and timing. All domains were evaluated for the potential risk of bias and the first three were mainly concerned with applicability.

Statistical analysis

The accuracy indicators included pooled sensitivity, pooled specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic odds ratio (DOR) and their 95% confidence intervals (CIs). These were calculated using the random effects model (24). The summary receiver operative curve (SROC), which reveals the association between sensitivity and the false positive rate, was used to evaluate the consistency of results between all studies in addition to the accuracy of the test (25). The area under the curve (AUC) was also calculated. Heterogeneity was measured using a Q test and the inconsistency index (I2) (26). P<0.05 or I2>50% was considered to indicate significant heterogeneity and therefore the random effects model was applied (27); otherwise the fixed-effect model was used. One of the most important causes of heterogeneity in diagnostic tests is the threshold effect. This occurs when the sensitivity and specificity are negatively correlated (or sensitivity is positively correlated with 1-specificity), resulting in a typical ‘shoulder arm’ of the ROC plane distribution. A Spearman correlation analysis was performed. Subsequently, meta-regression and subgroup analyses were conducted to explore potential sources of inter-study heterogeneity. Furthermore, Deeks' funnel plots were used to detect any publication bias (28). All statistical analysis was conducted using Meta-Disc 1.4 software (Hospital Universitario Ramon y Cajal, Madrid, Spain) and STATA 12.0 software (StataCorp, LLC, College Station, TX, USA) (29,30).

Results

Study selection and characteristics

As presented in the flow chart (Fig. 1), a total of 14 potential relevant articles were identified initially, of which four were removed subsequent to reading the titles and abstracts in further detail. Following a full-text review, two studies were eliminated due to lack of sufficient data, leaving eight studies (14–16,22,31–34). The basic characteristics of the studies are summarized in Table I.

Figure 1.

Flow chart describing the systematic literature search and study selection process.

Table I.

Characteristics of the eight included studies in the present meta-analysis.

Study	Country	Year	Design	Blinding	Ethnicity	Mean age (years)	Male:Female	Sample size	TP	FP	FN	TN	(Refs.)
Mian et al, 2000	Austria	2000	Retrospective	Yes	Caucasian	65.8	NA	180	36	13	17	114	(31)
Babjuk et al, 2002	Czech	2001	Retrospective	Yes	Caucasian	66.3	141:77	107	38	6	40	23	(32)
Schroeder et al, 2004	Germany	2004	Prospective	Yes	Caucasian	64.3	80:35	135	21	19	38	57	(33)
Hakenberg et al, 2004	Germany	2004	Prospective	Yes	Caucasian	68.5	87:25	112	58	8	32	14	(34)
Ritter et al, 2014	Germany	2013	Prospective	Yes	Caucasian	70	151:47	198	37	41	24	96	(14)
Ecke et al, 2015	Germany	2015	Prospective	No	Caucasian	73	97:28	125	49	3	43	30	(15)
Styrke et al, 2017	Sweden	2017	Prospective	No	Caucasian	70	224:46	270	120	39	49	62	(16)
Ecke et al, 2017	Sweden	2017	Prospective	No	Caucasian	72	78:31	109	45	2	42	20	(22)

TP, true positive; FP, false-positive; TN, true negative; FN, false-negative; NA, data not available.

All eight studies were conducted in a European population, and the majority of the patients were male and >50 years old (Table I). Urinary sediment was used as a specimen, and cytology or cystoscopy was considered as the gold standard. The results of the quality assessment are presented in Fig. 2. The majority of articles included the majority of the QUADAS-2 domains, indicating that the overall quality of the included studies was moderate to high.

Figure 2.

Quality assessments of included studies using the QUADAS-2 tool. (A) Risk of bias summary: A review of the authors' judgments about the risk of each bias item for each included study. (B) Risk of bias graph: A review of the authors' judgments about each item presented as percentages across all included studies. QUADAS-2, quality assessment of diagnostic accuracy studies 2.

Threshold effect

The ROC curve of sensitivity against the specificity of each study (Fig. 3) revealed a non-typical shoulder arm appearance, indicating that there was no threshold effect. In addition, the calculated Spearman correlation coefficient value was 0.44 (P=0.27), also indicating no threshold effect.

Figure 3.

ROC curve for the assessment of the threshold effect in the UBC® Rapid Test. ROC, receiver operating characteristics.

Diagnostic accuracy

Overall, the sensitivity of the pooled data was 0.59 (95% CI=0.55–0.62) and the specificity was 0.76 (95% CI=0.72–0.80) (Fig. 4). The pooled PLR was 2.55 (95% CI=1.75–3.70), the NLR was 0.56 (95% CI=0.46–0.67) and the DOR was 4.88 (95% CI=2.82–8.45) (Figs. 5 and 6). The SROC curve for the eight studies is presented in Fig. 7. The overall AUC of the UBC® Rapid Test was 0.70 (95% CI=0.85–0.91). Significant heterogeneity was identified for pooled sensitivity (I2=78.8%, P<0.001; Fig. 4) and specificity (I2=82.1%, P<0.001; Fig. 4) so the random effects model was applied for further analysis.

Figure 4.

Forest plots of estimated (A) sensitivity and (B) specificity for the UBC® Rapid Test in the diagnosis of bladder cancer. CI, confidence interval.

Figure 5.

Forest plots of estimated (A) PLR and (B) NLR for UBC® Rapid Test in the diagnosis of bladder cancer. PLR, positive likelihood ratio; NLR, negative likelihood ratio; CI, confidence interval.

Figure 6.

Forest plots of the pooled DOR for the UBC® Rapid Test in the diagnosis of bladder cancer. DOR, diagnostics odds ratio; CI, confidence interval.

Figure 7.

SROC curve for UBC® Rapid Test in the diagnosis of bladder cancer of the included eight studies. SROC, summary receiver operative curve; AUC, area under curve; SE, standard error.

Meta-regression and subgroup analysis

Heterogeneity was identified in the estimates of sensitivity, specificity, PLR, NLR and DOR. Therefore, meta-regression was used to explore the source of heterogeneity on the basis of study design, double blinding and sample size. However, none of the above covariates were heterogeneous (all P>0.05; Table II). Although subgroup analysis, including design, blind and sample size, were performed, there was no difference in the diagnostic efficacy of this test, indicating none of the parameters were identified to be a source of heterogeneity (Table III).

Table II.

Results of the multivariable meta-regression model for the characteristics with backward regression analysis.

Variables	Coefficient	Standard error	P-value	RDOR	95% confidence interval
Cte.	1.179	0.7577	0.2174	–	–
S	−0.404	0.3753	0.3603	–	–
Design	0.798	0.7540	0.3673	2.22	(0.20–24.49)
Blinding	−0.750	0.6327	0.3210	0.47	(0.06–3.54)
Sample size	0.693	0.6791	0.3829	2.00	(0.23–17.35)

Cte., constant coefficient; S, statistic S; RDOR, relative diagnostic odds ratio. Inverse variance weights; variables were retained in the regression model if P<0.05.

Table III.

Summary results of diagnostic accuracy of UBC test for bladder cancer.

Subgroup	No. of studies	Sensitivity (95% CI)	Specificity (95% CI)	PLR (95% CI)	NLR (95% CI)	DOR (95% CI)	AUC
Design
Retrospective	2	0.56 (0.48, 0.65)	0.88 (0.82, 0.93)	4.08 (1.47, 11.33)	0.49 (0.25, 0.94)	8.45 (1.70, 42.05)	–
Prospective	6	0.59 (0.55, 0.63)	0.71 (0.67, 0.76)	2.0 (1.52, 2.64)	0.58 (0.47, 0.70)	3.81 (2.37, 6.12)	0.6945
Sample size
>150	3	0.68 (0.62, 0.74)	0.75 (0.70, 0.79)	2.78 (1.41, 4.03)	0.47 (0.37, 0.59)	6.11 (2.46, 15.16)	0.7271
≤150	5	0.52 (0.47, 0.57)	0.79 (0.72, 0.85)	2.39 (1.49, 5.17)	0.62 (0.50, 0.76)	4.09 (1.97, 8.49)	0.6392
Blinding
Yes	5	0.61 (0.56, 0.67)	0.72 (0.64, 0.79)	2.39 (1.44, 3.96)	0.59 (0.43, 0.79)	4.18 (1.87, 9.31)	0.7058
No	3	0.56 (0.48, 0.65)	0.88 (0.82, 0.93)	3.47 (1.26, 9.58)	0.51 (0.44, 0.59)	6.30 (2.82, 14.06)	0.7369
Total	8	0.59 (0.55, 0.62)	0.76 (0.72, 0.80)	2.55 (1.75, 3.70)	0.56 (0.46, 0.67)	4.88 (2.82, 8.45)	0.7046

CI, confidence interval; PLR, positive likelihood ratio; NLR, negative likelihood ratio; DOR, diagnostic odds ratio; AUC, area under the curve.

Publication bias

Deeks' funnel plot demonstrated no significant publication bias (P=0.70; Fig. 8).

Figure 8.

Deeks' Funnel Plot Asymmetry Test for the assessment of potential publication bias. ESS, effective sample size.

Discussion

To date, cystoscopy has been considered the gold standard for detecting bladder cancer and for following up patients who have undergone tumor resection (35). However, it is an invasive and expensive tool. Another test widely used in clinical practice is urine cytology; however, its low sensitivity limits its use (36). Therefore, it is necessary to identify a viable, reliable and minimally-invasive method to detect new or recurrent bladder cancer.

The UBC® Rapid Test is a quantitative method to determine the levels of urinary fragments of cytokeratin 8 and 18, and has recently been developed as a tumor marker to detect bladder cancer (37). In the present analysis, the pooled AUC of the UBC® Rapid test indicated that it was a better diagnostic tool compared with cystoscopy and cytology. The DOR value, the ratio of correct to false diagnosis, is a comprehensive indicator of the diagnostic efficiency index (38). The pooled DOR in the present study suggested that the UBC® Rapid Test is reliable compared with the overall accuracy of bladder cancer diagnosis.

The likelihood ratio, including PLR and NLR, is also a strong performance indicator for diagnostic experiments (39). Generally considered, a PLR>10 indicates the presence of disease, and a NLR<0.1 may rule out the possibility of disease. However, the present study revealed that the pooled PLR and NLR for the UBC® Rapid Test were 2.55 and 0.56, respectively. This suggests that the probability of the test providing a positive result in patients with bladder cancer was 2.55 times higher compared with patients without bladder cancer; and the probability of negative results was 0.56 times higher compared with in non-patients. Therefore, the performance of the UBC® Rapid Test in terms of pooled PLR and NLR did not meet clinical practice requirements and should be further modified prior to clinical use.

Exploring heterogeneity is crucial to understanding the factors that affect accurate estimates in addition to the appropriateness of combining the accuracy of different studies (40). Substantial heterogeneity was identified in the present meta-analysis in terms of the pooled sensitivity, specificity, PLR, NLR and DOR. The threshold effect remains an important cause of heterogeneity in diagnostic trials (41). In the present meta-analysis, a significant threshold effect was not observed. To further explore the source of heterogeneity, a meta-regression analysis was used based on design, blinding and sample size. The results suggested that none of these parameters were the cause, indicating that other variables contributed to the heterogeneity across the studies; these may have been publication and choice bias.

There are several limitations to the study. First, despite the extensive literature search, the number of studies and sample sizes included were small. Secondly, several papers published in different languages were excluded from the review, which may result in potential heterogeneity. Thirdly, all the trials included in this meta-analysis were retrospective, which may limit the conclusions due to the bias of choice.

In general, the present study suggests that the UBC® Rapid Test may be beneficial for the diagnosis of bladder cancer, since this non-invasive approach has a good overall diagnostic performance. However, further prospective, large-scale and multicenter assessments of clinical studies are required to fully assess the diagnostic role of the UBC® Rapid Test in patients with bladder cancer.