Accuracy of interleukin-27 assay for the diagnosis of tuberculous pleurisy: A PRISMA-compliant meta-analysis.

BACKGROUND: The concentration of interleukin-27 (IL-27) in pleural effusions was found to be increased in tuberculous pleurisy and several studies have investigated the diagnostic value of IL-27 for tuberculous pleural effusions (TPEs), but the results varied a lot. We conducted the present study to comprehensively evaluate the diagnostic value of IL-27 for TPE.
METHODS: Primary diagnostic test studies of IL-27 for TPE was searched and identified from databases. The pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ration, diagnostic odds ratio, and receiver operating characteristic curves (SROCs) were computed or pooled to summarize the overall test performance.
RESULTS: Nine studies with a total number of 1226 patients were identified in our research. The main pooled estimates were as follows: sensitivity 0.92 [95% confidence interval (CI), 0.90-0.95], specificity 0.90 (95% CI, 088-0.92), and area under the SROC 0.97. No evidence of publication bias was detected.
CONCLUSION: Our research suggested the good diagnostic value of IL-27 for TPE and it could be used as a diagnostic biomarker.

Introduction

Pleural effusion is a common clinical problem, which can be caused by >50 diseases.[ Among them, pleural tuberculosis (TB) is the leading etiology of pleural effusion, especially in regions with high incidence of tuberculosis.[ For instance, in the largest series of 833 Chinese patients with undiagnosed pleural effusion who undergo medical thoracoscopy, 40.0% of cases are confirmed to suffer from tuberculous pleural effusion (TPE). In India, the proportion is 23.5% and in South Africa, the percentage is >80%.[

To date, the diagnosis of TPE remains a challenge. The conclusive diagnosis of TPE depends upon the isolation of Mycobacterium tuberculosis from sputum, pleural fluid, or pleural biopsy specimens. Positive rate of sputum acid-fast bacilli smear is merely 12% and the sensitivity of pleural fluid cultures is <40%.[ Medical thoracoscopy delivers a good diagnostic advantage for tuberculous pleurisy, but it is an invasive technique associated with major (1.8%) and minor (7.3%) complications,[ which may not be well accepted by certain patients especially the children and the elderly.[ Thus such methods are either deficient in some manner or too invasive. Because of the limitations of such conventional methods, the development of fluid biomarkers assay as an effective and noninvasive method to diagnose the tuberculous pleurisy was invigorated.

Interleukin-27 (IL-27), a recently discovered heterodimeric cytokine, has been found to be involved in TPE. It is a member of IL-12 cytokine family and is mainly produced by active antigen-presenting cells (APCs) under the stimulation of pathogen-associated molecular patters banding to toll-like receptors.[ Several studies found that along with IL-12, IL-27 plays an important role in the regulation of macrophage biological function during infection, and thus impending M tuberculosis growth.[ Tuberculous infection may cause the secretion of IL-27 and it can reach high localized concentrations in TPE.[ This raises the question that whether Il-27 can be a biomarker to differentiate TPE from non-TPE. Many researches focused on the question and a large portion of them showed an excellent diagnostic value and it was considered to be used as a diagnostic tool for TPE. Wu et al[ reported that with sensitivity of 95% and specificity of 97.6%, the diagnostic accurate of IL-27 was even better than INF-γ and adenylic deaminase (ADA). However several other studies showed the different results.[ The overall accuracy of IL-27 assay for the diagnosis of tuberculous pleurisy remains indecisive. Therefore, based on current available collective evidence, we performed the meta-analysis to establish the overall diagnostic accuracy of IL-27 for TPE.

Methods

Search strategy and study selection

We searched PubMed, Embase, Web of Science, China National Knowledge Infrastructure, Wangfang, and Weipu databases (up to May 30, 2017) to identify primary studies that relevant to the diagnostic value of IL-27 for TPE. The following search strings: “Interleukin-27,” “IL-27,” “pleurisy/pleuritis,” “pleural effusion/pleural fluid,” “tuberculosis,” “sensitivity and specificity,” and “accuracy” were used as medical headings or text words. References listed in the included articles or review articles were also examined manually to find more studies. A diagnostic study providing both sensitivity and specificity of IL-27 for TPE was included in our research. In order to reduce selection bias, we did not incorporate studies involving fewer than 30 participants. The languages were limited to English or Chinese. Two authors independently reviewed each included studies (ML and WYZ). Discrepancies in evaluation were resolved by consensus. Ethical approval is not required for a retrospective meta-analysis.

Data extraction and quality assessment

The included articles were assessed independently by 2 reviewers (ML and WYZ). The 2 reviewers were blind to each eligible article to extract the data on study characteristics and results of test accuracy, including publication year, first author, the country origin, patient number, test method, cut-off value, diagnostic standard of TPE, 2-by-2 tables of true positive, true negative, FP (false positive), and false negative. The methodology of eligible primary researches was evaluated by QUADAS-2 (quality assessment for studies of diagnostic accuracy) tool.[

Statistical analysis

Statistical Analysis was conducted by Stata software, version 11.0 (Stata Corp LP, College Station, TX), Meta-Disc for Windows (XI Cochrane Colloquium, Barcelona, Spain), and Review Manager 5.2 (The Cochrane Collaboration, Copenhagen, Denmark). Preferred Reporting Items for Systemic Reviews and Meta-analysis criteria and standard methods recommended for the diagnostic accuracy of meta-analyses were followed.[ We used Spearman rank correlation to test the threshold effect. The between-study heterogeneity was assessed by the Cochrane Q-statistic and I2 index. Substantial heterogeneity was considered when I2 exceeding 50%. For heterogeneous data, the DerSimonian-Laird model (random-effects model) was applied and subgroup analyses performed to explore potential between-study heterogeneity. Otherwise, the Mantel-Haenszel model (fixed-effects model) was employed.[ Meta-regression was applied to detect the potentially important covariates exerting substantial impact on between-study heterogeneity. Publication bias was assessed by Deeks's funnel plots.[

To evaluate the accuracy of IL-27 assay, we pooled the following diagnostic test indices: sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR). Area under the curve (AUC) on the summary receiver operating characteristic curves (SROCs) was applied to assess the overall diagnostic performance of the test. All statistical tests were 2 sided, with P values of .05 denoting statistical significance.

Results

Studies included in the meta-analysis

One hundred one potentially relevant studies were found from electronic databases with our search strategy. Ninety-eight of them were excluded based on title or abstract screening. The remaining 13 studies were retrieved by full texts. Among them, 2 were excluded because they did not report the sensitivity or specificity of IL-27 assay or they cannot be estimated with the data provided by the articles.[ One study was duplicate and only the best quality study was chosen.[ Subsequently, we identified and enrolled 9 articles that met our inclusion criteria.[ The process of selecting eligible studies is shown in Figure 1.

Figure 1

Flow chart of literature research.

Quality of the literatures and study characteristics

Basic characteristics of included studies were outlined in Table 1. Nine eligible studies with average sample size of 136 (range 44–431) provided a total population of 1226 patients. The 7 studies carried out in China[ and 2 in Europe[ from 2012 to 2017. In 2 studies, diagnosis of tuberculous pleurisy was made by histological or bacteriological examination, which are the “criterion standard” of tuberculosis infection.[ Although in the remaining 7 studies, 1 part of the patients were diagnosed by “criterion standard,” another part of the patients were diagnosed by clinical presentation, auxiliary examinations, and the responsiveness to anti-tuberculous chemotherapy (clinical course). All the studies mentioned the methods to detect IL-27. One study used liquid array technology[ and the remaining chosen enzyme-linked immunosorbent assay (ELISA). We used QUADAS-2 tool to evaluate the quality of the included studies. The tool covers 4 domains, including: patient selection, index test, reference standard, and flow and timing of samples. The results suggested that the quality of primary studies in our research was generally good except 4. They were judged to have high risk of bias in patient selection, reference standard, and flow and timing.[ In addition, 3 studies were judged to have uncertain risk of bias in patient selection.[Figure 2 (A and B) presents the QUADAS-2 assessment results of included studies.

Table 1

Summary of included studies.

Figure 2

Methodological quality assessment of studies of the interleukin-27 (IL-27) assay. A, Graph of risk of bias and applicability concerns. B, Summary of risk bias and applicability concerns.

Diagnostic accuracy

The Spearman rank correlation coefficient was −0.538 (P = .2603), which suggested the absence of threshold effect. Heterogeneity test showed the I2 value of sensitivity, specificity, PLR, NLR, and DOR were 0.9%, 79.5%, 83.8%, 0.0%, and 48.4%, respectively. So the fixed-effects model was applied to pool the sensitivity, NLR, and DOR (I2 < 50%). The specificity and PLR were pooled by the random-effects model due to the significant heterogeneity across the included studies (P < .05, I2 > 50%).

The forest plots of sensitivity and specificity for the IL-27 assay are shown in Figure 3. The sensitivity ranged from 0.80 to 1.0 [pooled 0.92, 95% confidence interval (CI), 0.90–0.95] and specificity ranged from 0.84 to 0.99 (pooled 0.90, 95% CI, 088–0.92). The pooled PLR and NLR were 15.78 (95% CI, 6.94–35.85) and 0.09 (95% CI, 0.06–0.12). The overall DOR was 132.23 (95% CI, 79.13–220.95).

Figure 3

Forest plots of sensitivity and specificity for interleukin-27 (IL-27) assay for the diagnosis of tuberculous pleural effusion (TPE). The point estimates of sensitivity and specificity from each study are shown as solid circles. Error bars indicate 95% CI. CI = confidence interval.

To summarize the global diagnostic performance of the test efficacy, we used the SROC and calculated the Q-value. The AUC was 0.9701 and the Q-value, the maximum joint sensitivity and specificity of our research, was 0.9200, indicating a good diagnostic value of IL-27 for TPE. Figure 4 shows the SROC.

Figure 4

SROC of interleukin-27 (IL-27) assay for the diagnosis of tuberculous pleural effusion (TPE). The size of each solid circle represents the sample size of each study. The regression SROC indicates the overall diagnostic accuracy. AUC = area under the curve, SROC = summary receiver operating characteristic curve.

Subgroup analysis

Base on the settings of tuberculosis incidence, we divided all the primary researches into 2 groups to perform subgroup analysis. Group A included 7 studies conducted in countries with high tuberculosis prevalences,[ whereas group B included the remaining 2 studies carried out in settings with low tuberculosis incidence rate.[ In group A, the pooled sensitivity, specificity, PLR, NLR, and DOR were 0.93 (95% CI, 0.90–0.95), 0.96 (95% CI, 0.94–0.98), 22.58 (95% CI, 13.5–38.32), 0.07 (95% CI, 0.90–0.95), and 289 (95% CI, 142.03–589.82) respectively. In group B, the pooled sensitivity, specificity, PLR, NLR, and DOR were 0.90 (95% CI, 0.81–0.96), 0.85 (95% CI, 0.82–0.88), 5.85 (95% CI, 4.66–7.35), 0.12 (95% CI, 0.06–0.23), and 52.21 (95% CI, 24.15–112.85).

Regression analysis and publication bias

Metaregression was performed to explore the possible covariates for the heterogeneity in our research. Based on the study characteristics, we chose 3 covariates: sample size, diagnostic standard, and ethnicity. Among them, ethnicity was likely to be the potential source of heterogeneity (P = .077). Sample size and diagnostic standard did not substantially affect the diagnostic performance of IL-27 assay (all P > .05). The outcomes of regression-analysis are shown in Table 2.

Table 2

Metaregression of potential heterogeneity within the included studies.

Publication bias analysis was conducted by Deek funnel plot test. The statistically nonsignificant value indicated a low likelihood of publication bias (P = .67). The Deek funnel plot test for our research is shown in Figure 5.

Figure 5

Deeks funnel graph of publication bias of included studies. The statistically insignificant value (P = .70) for the slope coefficient suggested symmetry in the data and a low likelihood of publication bias.

Discussion

Diagnosis of pleural tuberculosis remains a challenge with traditional methods. There is a need of reliable, efficient, and noninvasive methods to promote the development of fluid biomarkers assay as a kind of diagnostic tool. Previous studies on some potential biomarkers, such as interferon gamma (IFN-γ), neopterin, and leptin[ were not as beneficial as we expected. Therefore a few researchers turned their eyes toward IL-27.

IL-27, a member of the IL-12 cytokine family, is mainly produced by activated APCs. It is found to be involved in the immune response induced by M tuberculosis for its dual regulation function in intracellular infections.[ For one thing, by suppressing T-cell expansion and IFN-γ secretion, IL-27 prevents the maximal expression of antimycobacterial immunity. For another, IL-27 inhibits the production of tumor necrosis factor and IL-12, and thus preventing the pathological systemic hyperinflammatory response caused by M tuberculosis infection.[ In patients with TPE, Yang et al proved that pleural CD4+T cells, CD8+T cells, NK cells, NKT cells, B cells, monocytes, macrophages, and mesothelial cells can produce the IL-27 after tuberculous infection and it can reach a high localized concentrations in pleural fluid. Although the precise pathophysiological biofunction of IL-27 in TPE still needs further investigation, the fact of a local production of IL-27 in pleural space raises the possibility that IL-27 may have potential of being a promising candidate biomarker for differential diagnosis of TPE from pleural effusions with other causes. Several relevant diagnostic tests were performed, but the results remain controversial. Therefore we performed the present study to gain an overall result. To the best of our knowledge, it is the first meta-analysis to evaluate the diagnostic accuracy of IL-27 in TPE.

In our research, the pooled sensitivity of IL-27 assay is 0.92 (95% CI, 0.90–0.95), suggesting that approximately 92% of TPE patients would get a positive results after receiving the diagnostic test, which is helpful for the establishment of TPE diagnosis. A pooled specificity of 0.90 (95% CI, 088–0.92) means approximately 90% of patients without TPE would get a negative results, which is helpful to exclude the TPE patients. The PLR and NLR are easier to interpret and more clinically meaningful. In our study, the PLR was 15.78 (95% CI, 6.94–35.85) and NLR was 0.09 (95% CI, 0.06–0.12). The results mean that compared to patients without TPE, patients with tuberculous pleurisy have a 15-fold higher chance of being IL-27 assay positive. Similarly if a patient gains a negative result of IL-27 assay, he could have a 9% chance of being a tuberculous pleurisy patient. To present a global accuracy of the IL-27 test, SROC was employed and it showed a Q value of 0.9200 and an AUC of 0.9701. Since an AUC of 1.0 means a perfect ability for a diagnostic method to discriminate case from noncase, the present meta-analysis showed an excellent diagnostic value of IL-27 for TPE. Compared with other classic biomarkers, IL-27 appears to have a better diagnostic performance. ADA and IFN-γ are the most studied biomarkers for TPE and now they are widely used in clinical practice. According to the previous meta-analysis, the pooled sensitivity and specificity for ADA were 0.92 (95% CI 0.90–0.93) and 0.90 (95%CI 0.89–0.91)[ and for IFN-γ, the data were 0.82 (95% CI 0.79–0.85) and 0.87 (95% CI 0.84–0.90).[ Based on the related researches, IL-27 seems to have a better diagnostic accuracy than ADA and IFN-γ. Wu et al‘s diagnostic test directly compared the diagnostic performance of the 3 cytokines for TPE and the results were consistent with our conclusion.[ In addition, IL-27 appears to have a better diagnostic value in areas with high TB prevalence according to our research. Intense criticism against the ADA and INF-γ was often focused on the fact that their performance is highly dependent on the prevalence of TB.[ Previous studies have shown that the FP rate of these tests was higher in high TB prevalence settings. We blame the phenomenon partly to the high incidence rate of latent M tuberculosis infection in the areas. The latent infected patients live with superior immunologic function and smaller pathogen load can induce the response to tuberculosis antigen.[ By subgroup analysis, we found that unlike IFN-γ or ADA, diagnostic accuracy of IL-27 assay in high TB incidence group seems better than that in low TB incidence group. So the role of ADA and IFN-γ in combination with IL-27 is worth investigating further because this may overcome the problems with ADA or IFN-γ in countries with high TB incidence. One research carried out in China demonstrated that compared with using IL-27 alone, the combination of IL-27 with IFN-γ and ADA provide a better diagnostic accuracy. When at least 1 of these 3 tests is positive, the optimal sensitivity would be 100%. When all the 3 tests are positive, the specificity is to be 100%.[

The I2 value to detect heterogeneity exceeded 50% when pooling specificity and PLR, suggesting the significant heterogeneity across the included studies. Threshold effect due to the different cut-offs in the different primary researches is one of the most important causes of heterogeneity. But in our meta-analysis Spearman correlation test showed that it was not the source of heterogeneity. In subgroup analysis, we neither detect heterogeneity in high TB incidence group nor in low TB incidence group. This suggested that setting (area with high TB incidence or area with low TB incidence) was one sources of heterogeneity. As mentioned above, we found that unlike IFN-γ, diagnostic accuracy of IL-27 assay in high TB incidence group was better than that in low TB incidence group in subgroup analysis. It should be mentioned that this conclusion should be drawn with caution because merely 2 studies were included in low TB incidence group. Limited sample size may limit the interpretation of the meta-analysis. Further investigations are needed to provide a more reliable answer. In addition to subgroup analysis, we conducted meta-regression to explore the potential reason for heterogeneity. Sample size (≥100 or <100), diagnosis standard (by criterion standard or clinical course), and ethnicity (Asian or Caucasian) were included as variables. The outcomes of regression analysis indicated that ethnicity affected the diagnostic value of IL-27 for TPE. In our research, the primary studies of Asian patients were conducted in high TB incidence rate setting and the primary studies in which the Caucasian patients were involved were carried out in counties with low TB incidence rate. So, we think that it's probably the setting (area with high TB incidence or area with low TB incidence), not the ethnicity that affects the diagnostic performance of the IL-27 assay.

The findings of our research should be interpreted with caution due to several limitations. First, there are relatively small set of primary researches in our meta-analysis because of the strict inclusion and exclusion criteria. And we did not include studies published in languages other than English or Chinese and some unpublished literatures. Studies without sufficient data were also filtered such as abstracts and letters to the editors. So limited primary date may lead to inadequate statistical power to draw definitive conclusions and may limit the interpretation of the meta-analysis. Well-designed studies with larger sample are still needed in the future. In addition, misclassification bias may exist in the present meta-analysis because not all the tuberculous pleurisy in the included primary researches was diagnosed by criterion standard. The diagnosis in 7 primary studies was based on a mixture of bacteriological, histological, or clinical assessments.[

Conclusion

In conclusion, our study indicates that IL-27 assay in pleural effusion is a beneficial diagnostic tool to diagnose TPE.