Diagnostic value of microRNA for pancreatic cancer: a meta-analysis.

INTRODUCTION: MicroRNAs have been reported to be aberrantly expressed in patients with pancreatic cancer. The aim of the present meta-analysis is to establish the overall diagnostic accuracy of the measurement of microRNA for diagnosing pancreatic cancer.
MATERIAL AND METHODS: After a systematic review of English language studies from Medline, Embase, and Cochrane Library, the sensitivity, specificity, and other measures of accuracy of microRNA in the diagnosis of pancreatic cancer were pooled using random-effects models. The methodological quality of each study was assessed by QUADAS (quality assessment for studies of diagnostic accuracy). Statistical analysis was performed by employing Meta-Disc 1.4 software and STATA. Summary receiver operating characteristic curves were used to summarize overall test performance. Deeks' test was used to test the potential publication bias.
RESULTS: Nine studies from seven publications met our inclusion criteria. The summary estimates for microRNAs in the diagnosis of pancreatic cancer in these studies were pooled sensitivity 0.89 (95% CI: 0.86-0.91), specificity 0.93 (95% CI: 0.90-0.95), positive likelihood ratio 11.62 (95% CI: 5.75-23.50), negative likelihood ratio 0.14 (95% CI: 0.08-0.24), diagnostic odds ratio 115.13 (95% CI: 33.73-351.28), and the area under the curve was 0.97.
CONCLUSIONS: MicroRNA assay plays an important role in the diagnosis of pancreatic cancer. The results of microRNA assays should be interpreted in parallel with clinical findings and the results of conventional tests.

Introduction

Pancreatic cancer (PaC) is a highly malignant cancer with increasing incidence and mortality worldwide. Pancreatic cancer is the fourth leading cause of cancer-related death in western countries and eighth in China, with a median survival of < 6 months and an average 5-year survival rate < 5% [1, 2]. Because of its insidious onset, the early diagnostic accuracy is very low, and the late diagnosis leads to a low resection rate and poor prognosis. Additionally, PaC and other tumor-free pancreatic diseases (e.g. chronic pancreatitis (CP)) may present with similar symptoms and similar imaging features, which often leads to incorrect interpretation [3, 4]. So, diagnosis of PaC remains a clinical challenge.

The common clinical diagnostic markers such as serum CA19-9, CA-125, CEA, MMP-9, K-ras mutation and many others are poorly effective in diagnosis of PaC [5-9]. Serum CA19-9 is the most common diagnostic marker, with a sensitivity of 70-80% but specificity less than 50% for diagnosing PaC, and CA19-9 correlates with the tumor size and stage of PaC, so it does not improve diagnostic efficacy for early PaC [5, 10]. Thus, finding an objective molecular test that can discriminate PaC from other tumor-free pancreatic diseases, especially CP, will be of fairly useful clinical significance.

MicroRNAs (miRNAs/miR) are a functional class of 18-24 nucleotide non-protein-coding RNA molecules that negatively regulate gene expression. Disturbance of miRNA expression may play a role in the initiation and progression of certain diseases. A lot of miRNAs have been found to correlate well with many human cancers since the first report of miR-15 and miR-16 expression abnormalities in chronic lymphocytic leukemia (CLL) [11-15]. Many studies have identified aberrant expression of miRNAs in human PaC [15-24]. Although the diagnostic accuracy of miRNA assays for PaC has been extensively studied, their exact role needs to be analyzed.

Therefore, we performed the present meta-analysis to establish the overall diagnostic accuracy of miRNA assays for diagnosing PaC.

Material and methods

Search strategy and study selection

Medline, Embase and Cochrane Library were searched for suitable studies in English language until June 2012 but without a lower date limit. Articles were also identified by using the related-articles function in PubMed. The search terms were “pancreatic cancer/pancreatic neoplasm(s)/pancreatic carcinoma/pancreatic adenocarcinoma/pancreatic ductal adenocarcinoma/intraductal papillary mucinous neoplasms”, “microRNA/miRNA”, “sensitivity”, “specificity”, and “diagnosis”.

A study was included if it met the following inclusion criteria: (1) published in English language; (2) e-clinical studies on evaluation of miRNAs in the diagnosis of PaC; (3) provided or can calculate both the sensitivity and specificity data; (4) each group contains more than 10 patients. Conference abstracts were excluded because of the limited data. No publication with evidence of possible overlap of patients with other studies. Two reviewers (Chun Wan and Yong-Chun Shen) independently judged study eligibility while screening the citations. Disagreements were resolved by consensus.

Data extraction and quality assessment

The final set of articles was assessed independently by two reviewers (Chun Wan and Yong-Chun Shen). Data retrieved from these articles included author, publication year, study of state, diagnostic standard, patient number, specimen, test method, miRNA expression signature, sensitivity and specificity data and methodological quality. The methodological quality of each study was assessed by QUADAS (quality assessment for studies of diagnostic accuracy, an evidence-based quality assessment tool for use in systematic reviews of diagnostic accuracy studies, maximum score 14) [25].

Statistical analysis

Standard methods recommended for diagnostic accuracy meta-analysis were used [26]. The true positive rate (TPR) and false positive rate (FPR) of each study were converted by constructing a 2 × 2 contingency table, and the patient numbers were used to calculate the overall diagnostic accuracy. The following indexes of test accuracy of each study were computed: sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic odds ratio (DOR), and the area under the curve (AUC). The AUC represents an analytical summary of test performance and displays the trade-off between sensitivity and specificity. An AUC of 1.0 (100%) indicates perfect discriminatory ability to distinguish cases from non-cases. The pooled sensitivity, specificity and other related indexes across studies were calculated using a random-effects model [27]. Chi-square test was used to detect statistically significant heterogeneity across studies. If there were enough reports, subgroup analyses would be performed to explore the potential between-study heterogeneity [28]. Since publication bias is of concern for meta-analyses of diagnostic studies, we tested for the potential presence of this bias using Deeks’ funnel plots [29]. All analyses were performed using two statistical software programs: Stata, version 12 (Stata Corporation, College Station, TX, USA) and Meta-Disc 1.4 for Windows (XI Cochrane Colloquium, Barcelona, Spain). All statistical tests were two-sided, and significance was set at p < 0.05.

Results

Characteristics and quality of the included studies

After independent review, seven English language publications dealing with miRNA expression for diagnosis of PaC were included in the present meta-analysis according to the above inclusion and exclusion criteria. The clinical characteristics of these studies, along with QUADAS score, were extracted and listed in Table I.

Table I

Summary of the studies included in the meta-analysis

Study	Author	Publication time	Country	Method	Patients		Test results				QUADAS score	Threshold	miRNA profile
					No.	Specimen	TP	FP	FN	TN
1	Lee et al. [19]	2007	America	RT-PCR	49	Tissue	28	4	0	17	10	No	miR-21, 100, 125b, 181a, 212, 221, 301, 367a, let-7i
2	Dillhoff et al. [20]	2008	America	ISH	137	Tissue	63	13	17	44	10	No	miR-21
3	Habbe et al. [21]	2009	America	RT-PCR	118	Tissue	52	1	12	53	10	No	miR-21
4	Habbe et al. [21]	2009	America	RT-PCR	118	Tissue	53	4	11	50	10	No	miR-155
5	Wang et al. [22]	2009	America	RT-PCR	47	Plasma	18	2	10	17	12	No	miR-21, 155, 196a, 210
6	Liu et al. [23]	2012	China	RT-PCR	245	Plasma	122	4	16	103	8	No	miR-16, 196a +CA19-9
7	Liu et al. [23]	2012	China	RT-PCR	206	Plasma	127	3	11	65	8	No	miR-16, 196a +CA19-9
8	Munding et al. [24]	2012	Germany	RT-PCR	75	Tissue	39	2	3	31	10	Yes	miR-135b
9	Liu et al. [25]	2012	China	RT-PCR	202	Serum	120	3	0	79	11	No	miR-20a, 21, 24, 25, 99a, 185, 191

TP – true positive, FP – false positive, FN – false negative, TN – true negative, QUADAS – quality assessment for studies of diagnostic accuracy (maximum score 14), ISH – in situ hybridization, no. – number, tissue – pancreatic tissue, no – the study doesn't mention threshold, yes – the study mentions a threshold

In the present meta-analysis, nine studies from the seven articles including 941 patients were available for analysis. All patients with PaC were diagnosed based on the histological and/or cytological evaluation of surgically resected tissue specimens or endoscopic ultrasound-guided fine-needle aspiration biopsy (EUS-FNAB). The categories of specimens included pancreatic tissue (5 studies, 55.6%), plasma (3 studies, 33.3%) and serum (1 study, 11.1%). More than 20 miRNAs involved in this study were analyzed by using the method of quantitative reverse transcription PCR (qRT-PCR) (8 studies), and in situ hybridization (ISH) (1 study). Only one study performed on pancreatic ductal adenocarcinoma (PDAC) and CP specimens mentioned the threshold of 5.14 for the Ct(miR-135b)-Ct(miR-24) biomarker pair, but others did not mention it. A score > 5.14 was derived as diagnostic negative (CP), and a score ≤ 5.14 as diagnostic positive (PDAC).

By use of QUADAS, quality scoring was compiled for every study on the basis of title, introduction, methods, results and discussion. When a criterion was fulfilled, a score of 1 was given, 0 if a criterion was unclear, and –1 if a criterion was not achieved. Overall, the quality of study design and reporting diagnostic accuracy of most studies were good to a certain extent and only seven studies had a higher QUADAS score (≥ 10).

Diagnostic accuracy

Figure 1 shows the forest plot of sensitivity and specificity for miRNA assays in the diagnosis of PaC of the nine studies. Pooled results for diagnostic accuracy are listed in Table II. The sensitivity ranged from 0.64 to 1.00 (pooled: 0.89; 95% CI: 0.86-0.91), and the specificity ranged from 0.77 to 0.98 (pooled: 0.93; 95% CI: 0.90-0.95). We also noted that pooled PLR was 11.62 (95% CI: 5.75-23.50), NLR was 0.14 (95% CI: 0.08-0.24), and DOR was 115.13 (95% CI: 37.73-351.28). These results indicated that the miRNA assays could differentiate PaC from those without PaC. Chi-squared values of sensitivity, specificity, PLR, NLR and DOR were 60.88 (p = 0.000), 25.55 (p = 0.0013), 36.42 (p = 0.000), 44.92 (p = 0.000) and 38.94 (p = 0.000), respectively, indicating significant heterogeneity between studies.

Figure 1

Forest plot of estimates of sensitivity and specificity for microRNA assays in the diagnosis of pancreatic cancer (PaC) of the included 9 studies. The point estimates of sensitivity and specificity from each study are shown as solid circles and the size of each study is indicated by the size of the solid circle. Error bars are 95% confidence intervals. Numbers indicate the 9 reference studies listed in Table I

Table II

Pooled diagnostic accuracy

	PaC vs. non-PaC#	Tissue-based miRNA	Blood-based (plasma and serum) miRNA
Number of studies	9	5	4
Sensitivity (95% CI)	0.89 (0.86-0.91)	0.85 (0.80-0.89)	0.91 (0.88-0.94)
Heterogeneity* (P)	60.88 (0.000)	14.61 (0.006)	38.88 (0.000)
Specificity (95% CI)	0.93 (0.90-0.95)	0.89 (0.84-0.93)	0.96 (0.93-0.98)
Heterogeneity* (P)	25.55 (0.001)	16.17 (0.003)	1.47 (0.690)
PLR (95% CI)	11.62 (5.75-23.50)	8.27 (3.56-19.23)	18.25 (10.36-32.12)
Heterogeneity* (P)	36.42 (0.000)	15.42 (0.004)	3.25 (0.355)
NLR (95% CI)	0.14 (0.08-0.24)	0.18 (0.11-0.28)	0.11 (0.03-0.34)
Heterogeneity* (P)	44.92 (0.000)	8.53 (0.074)	35.94 (0.000)
DOR (95% CI)	115.13 (37.73-351.28)	73.94 (18.97-288.21)	190.21 (34.56-1047.0)
Heterogeneity* (P)	38.94 (0.000)	15.21 (0.004)	13.73 (0.003)
AUC (SEM)	0.97 (0.013)	0.95 (0.025)	0.99 (0.007)

PaC – pancreatic cancer, non-PaC – patients without PaC

Q-value. AUC – area under the curve, DOR – diagnostic odds ratio, PLR – positive likelihood ratio, NLR – negative likelihood ratio

A graph of the summary receiver operating characteristic (SROC) curve for the miRNA assays showing true-positive rates vs false-positive rates from individual studies is shown in Figure 2. As a global measure of test efficacy, we used the Q-value, the intersection point of the SROC curve with a diagonal line from the left upper corner to the right corner of the ROC space, which corresponds to the highest common value of sensitivity and specificity for the test. This point does not indicate the only or even the best combination of sensitivity and specificity for a particular clinical setting but represents an overall measure of the discriminatory power of a test. Our data showed that the SROC curve is positioned near the desirable upper left corner of the SROC curve, and that the maximum joint sensitivity and specificity (i.e., the Q value) was 0.92, and the area under the curve (AUC) was 0.97, indicating a high level of overall accuracy.

Figure 2

Summary receiver operating characteristic (SROC) curve for microRNA assays in the diagnosis of pancreatic cancer (PaC) of the 9 included studies. Solid circles represent each study included in the meta-analysis. The size of each study is indicated by the size of the solid circle. The regression SROC curve summarizes the overall diagnostic accuracy

Table II also shows the pooled results for diagnostic accuracy of tissue-based and blood-based miRNA assays. For tissue-based miRNA assays, the pooled sensitivity was 0.85 (95% CI: 0.80-0.89), specificity was 0.89 (95% CI: 0.84-0.93), PLR was 8.27 (95% CI: 3.56-19.23), NLR was 0.18 (95% CI: 0.11-0.28), DOR was 73.94 (95% CI: 18.97-288.21) and the AUC was 0.95. For blood-based miRNA assays, the pooled sensitivity was 0.91 (95% CI: 0.88-0.94), specificity was 0.96 (95% CI: 0.93-0.98), PLR was 18.25 (95% CI: 10.36-32.12), NLR was 0.11 (95% CI: 0.03-0.34), DOR was 190.21 (95% CI: 34.56-1047.0) and the AUC was 0.99. Thus compared with the tissue-based miRNA assays, the blood-based miRNA assays showed a higher level of overall accuracy.

Publication Bias

Although the funnel plots for publication bias showed some asymmetry due to the limited number of studies (Figure 3), the Deeks’ test showed a statistically non-significant value (p = 0.30), indicating that there was no potential publication bias.

Figure 3

Funnel graph for the assessment of potential publication bias of the 9 included studies. The funnel graph plots the log of the diagnostic odds ratio (DOR) against the standard error of the log of the DOR (an indicator of sample size). Solid circles represent each study in the meta-analysis. The line indicates the regression line

Discussion

Development of sensitive and specific biomarkers for diagnosing pancreatic carcinoma is a critical problem because most of these patients remain asymptomatic at the early stage, and are surgically inoperable when diagnosed. Also, the conventional methods, such as enhanced-CT/MRI, serum CA19-9, CA-125, CEA, MMP-9, K-ras mutation and many others are not always helpful in making an early diagnosis, especially at the potentially resectable stage [3, 5–9, 30].

The present meta-analysis has shown that the pooled sensitivity was 0.89, specificity was 0.93, and AUC was 0.97, indicating a relatively high level of overall accuracy. The DOR is a single indicator of test accuracy [31] that combines the data from sensitivity and specificity into a single number. The DOR of a test is the ratio of the odds of positive test results if the subject has the disease to the odds of positive test results if the subject does not have the disease. The value of a DOR ranges from 0 to infinity, with higher values indicating better discriminatory test performance (higher accuracy). A DOR of 1.0 indicates that a test does not discriminate between patients with the disorder and those without it. In the present meta-analysis, we found that the pooled DOR was 115.13, also indicating a high level of overall accuracy.

The SROC curve and the DOR are not easy to interpret and use in clinical practice, while the likelihood ratio (PLR and NLR) is more clinically meaningful for our measures of diagnostic accuracy. A PLR value of 11.62 suggests that patients with PaC have about 11-fold higher chance of being miRNA assay-positive compared to patients without PaC, and this was high enough for the clinical purpose. The NLR was 0.14 in the present meta-analysis; it means that the probability of the patient having PaC is 14% if the miRNA assay is negative, which is not low enough to rule out PaC.

More than 20 miRNAs (miR-16, 20a, 21, 24, 25, 99a, 100, 135b, 125b, 155, 181a, 185, 191, 196a, 210, 212, 221, 301, 367a, let-7i, etc.) involved in this meta-analysis were analyzed. These miRNA biomarkers may provide new insight into the early detection of PaC. However, there is no standing panel of effective miRNAs. Lee et al. designed a study of more than 200 miRNA precursors, 20 of which expressed aberrantly in pancreatic adenocarcinoma [18]. Munding et al. identified 78 misregulated miRNAs and discovered that miR-135b performed best in discriminating pancreatic ductal adenocarcinoma (PDAC) from CP [23]. miR-21 was the most frequent one to be studied and its strong expression was expected to be an important indicator for predicting the poor survival of PaC [18–21, 24]. miR-155 has been identified as a biomarker of early pancreatic neoplasm based on its overexpression in ∼80% of precursor intraductal papillary mucinous neoplasms (IPMN) by Wang et al. [21]. Serum CA19-9 is the most common conventional diagnostic maker for PaC. Liu et al. found that the sensitivity and the specificity of combination (plasma miR-16+miR-196a+CA19-9) are better than CA19-9 alone for PaC screening of early tumors and differential diagnosis from CP [22]. Of the nine studies included, only one study mentioned the threshold from a training set of 20 specimens, suggesting that computing Ct(miR-135b)-Ct(miR-24) has the most discrimination potential. Therefore, more studies should be designed to define distinct miRNA expression profiling. And multi-center, prospective clinical studies on a large scale should be carried out to identify novel miRNA or the most effective combination of miRNA in the diagnosis of PaC.

Mitchell et al. showed that plasma-based miRNAs are remarkably stable for they are protected from endogenous RNase activity [32]. This meta-analysis also respectively analyzed the sensitivity and specificity of tissue-based and blood-based miRNA assays. Our results indicated that the blood-based level of miRNA can distinguish PaC with more significant sensitivity and specificity than tissue-based miRNA. Therefore, the development of blood-based (plasma or serum) biomarker assays may be the most desirable non-invasive method of choice for screening and diagnosing PaC in the future.

Our meta-analysis has several limitations. Firstly, although we used a comprehensive search strategy, and the screening, study selection, data extraction and quality assessment were done independently and reproducibly by two reviewers, there were only seven publications included, and the limited patient numbers may have influenced the outcomes, so it is still hard to make a definitive conclusion about the accuracy of diagnosis of PaC; hence, further studies on a large scale may be needed to confirm the diagnostic value of miRNA assay in PaC. Secondly, due to the limited studies included, we did not use QUADAS scores to perform a meta-regression analysis to assess the effect of study quality on relative DOR of miRNA assay in the diagnosis of PaC. And for the same reason, we could not explore whether the study design, such as blinded, cross-sectional, consecutive/random and prospective design, affects the diagnostic accuracy, either.

In conclusion, the present meta-analysis suggests a potential role for miRNA assays in screening and confirming a diagnosis of PaC. Further studies are required to confirm the predictive value. Since none of the biomarkers including miRNAs is special for PaC, the results of miRNA assays should be interpreted in parallel with clinical findings and the results of conventional tests.