Diagnostic accuracy of tumor necrosis factor-alpha assay for tuberculous pleurisy: A PRISMA-compliant meta-analysis.

BACKGROUND: The diagnosis of tuberculous pleurisy is difficult and traditional methods are not always helpful. Many studies have focused on the tumor necrosis factor-alpha (TNF-α) assay in pleural effusion for the diagnosis of tuberculous pleurisy, but the results remain controversial. This meta-analysis was conducted to determine the overall diagnostic accuracy of TNF-α.
METHODS: Relevant studies were searched from PubMed, Embase, Web of Science, Chinese National Knowledge Infrastructure (CNKI), Wangfang, and Weipu. We pooled the published results and computed the accuracy measures, including sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR). Receiver operating characteristic curves (SROC) and the area under the curve (AUC) were used to summarize the overall test performance.
RESULTS: Twelve studies with 1022 patients met the inclusion criteria. The pooled sensitivity and specificity were 0.85 (95%CI, 0.81-0.89) and 0.80 (95% CI, 0.77-0.83) respectively. The area under the SROC curve was 0.89.
CONCLUSIONS: The results of meta-analysis suggested that the TNF-α assay plays a vital role in the diagnosis of tuberculous pleurisy, whereas other test results or clinical findings should be interpreted together with the TNF-α assay to improve the overall diagnostic accuracy.

Introduction

Pleural effusion is widely found in patients with a variety of diseases. Pleural tuberculosis (TB) is one of the most important and common causes of pleural effusion, especially in the regions with high prevalence. Nevertheless, the establishment of tuberculous pleural effusion (TPE) diagnosis is difficult and challenging. Traditionally, it is based on the isolation of Mycobacterium tuberculosis from the pleural fluid or pleural biopsy specimens or the demonstration of caseating granulomas on histological examination by obtaining biopsy. These methods are far from perfection; they are either insufficient or too invasive. Only 10% to 35% of cultures and 20% to 81% of molecular tests reveal mycobacteria in pleural fluid.[ Pleural biopsy and culture of biopsy material provide a relative high sensitivity, but these procedures are invasive and may not be well accepted by all the patients.[ Therefore, some new effective and efficient methods should be identified to aid the diagnosis of tuberculous pleurisy.

TNF-α is an important pro-inflammatory cytokine which is synthesized by lymphocytes and macrophages.[ Previous study showed the diagnostic value of TNF-α assays in many diseases. Kleine et al[ evaluated the diagnostic value of TNF-α assay in cerebrospinal fluid for bacterial meningitis, which reached a sensitivity of 85% and a specificity of 91.3%. Surbatovic and Padkovic study showed the correlation between TNF-α and severe acute pancreatitis (SAP). In patients with SAP, the lower level of TNF-α indicated higher probability to develop multiple organ dysfunction syndrome (MODS), with a sensitivity and specificity of 83% and 77.4% respectively.[ Tebruegge et al[ investigated the diagnostic value of peripheral blood TNF-α assay to differentiate between TB-uninfected and TB-infected children (up to 18 year), the results showed that the assay achieved an excellent diagnostic value, with a sensitivity of 95.5% and a specificity of 88.0%. Related studies proved that mycobacterial proteins can cause the secretion of TNF-α and it can reach high localized concentrations in pleural effusion to eliminate the bacilli and granuloma formation.[ Therefore, various studies focused on whether TNF-α can be used as a diagnostic tool for tuberculous pleurisy. Some studies detected a high level of TNF-α in the tuberculous pleural effusion and many of them consider TNF-α assay can be used for the differential diagnosis of tuberculous pleural effusion. Several other studies, however, showed that there is no difference of TNF-α level between TB pleural effusion and Non-TB pleural effusion.[ Due to the controversial results of the previous studies, this meta-analysis established the overall accuracy of pleural TNF-α test in the diagnosis of tuberculous pleurisy.

Methods

Search strategy and study selection

Multiple data base including Medline (using PubMed as the search engine), Embase, Web of Science, CNKI, Wangfang and Weipu (up to December 30, 2015) were searched using the following search strings: “tumor necrosis factor,” “tuberculosis,” “pleurisy/pleuritis,” “pleural effusion/pleural fluid,” “tuberculosis,” “sensitivity and specificity,” and “accuracy” to identify relevant literatures. References listed in the selected articles were also examined. The languages were limited to English or Chinese. Conference abstracts and letters to the journal editors were excluded because of limited information and data. A study providing both sensitivity and specificity of TNF-α was incorporated into the meta-analysis. Each included study was reviewed independently by 2 authors (ML and WZ). Disagreement was resolved by consensus. Ethical approval is not required because meta-analysis is based on the published articles.

Data extraction and quality assessment

The included literatures were evaluated independently by 2 reviewers (ML and WZ). The reviewers were blinded to publication to retrieve the data, including: the author, publication year, the country, patient number, test method, cut-off value, sensitivity, specificity, and methodological quality. To assess the trial methodology, we used the QUADAS-2(quality assessment for studies of diagnostic accuracy) tool to evaluate the quality of diagnostic accuracy of the primary studies we used.[

Statistical analysis

We used standard methods recommended for the diagnostic accuracy of meta-analyses and followed the Preferred Reporting Items for Systemic Reviews and Meta-analysis (PRISMA) criteria.[ We computed the following indices of test accuracy: sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR).

The sensitivity and specificity of each single test were used to plot a summary ROC (SROC).[ Spearman's rank correlation was performed as a test for the threshold effect. Heterogeneity among the primary studies was detected with Chi-squared and Fisher's exact tests. The average sensitivity, specificity, and other related measurements of the studies were calculated by random-effects model or fixed-effects model.[ If there were enough studies, subgroup analyses were performed to explore potential between-study heterogeneity. Meta-regression analysis was conducted to explore the possible reasons for the heterogeneity. We also assessed the potential publication bias by using Deeks's funnel plots.[

The analysis was performed by 2 kinds of statistical software: Stata, version 11 (Stata Corporation; College Station, TX) and Meta-Disc for Windows (XI Cochrane Colloquium; Barcelona, Spain). All statistical tests were 2-sided, with P values of 0.05 denoting statistical significance.

Results

Studies included in the meta-analysis

After literature review, 20 studies were selected. Among them, 7 were excluded because sensitivity and specificity of TNF-α test could not be estimated with the data provided by the articles.[ One study was excluded because the same authors published the research on the same patient; thus, only the best quality study was chosen.[ Consequently, the remaining 12 studies were eligible for the meta-analysis,[ with a total number of 1022 patients. The flowchart of study selection is shown in Fig. 1.

Figure 1

Flow chart of literature research.

Quality of the literatures and study characteristics

In our meta-analysis, the average sample size of the included studies was 85 (range 50–127). Diagnosis of TPE in 7 studies was made based upon histological or bacteriological confirmation, which are considered “gold standard.”[ Whereas in the other remaining 5 studies, TPE patients were diagnosed based upon “gold standard” or on their clinical course which included clinical presentation, pleural fluid analysis, radiology, and the responsiveness to anti-tuberculous therapy.[ All the included studies mentioned the TNF-α assay method. One study used the RIA assay as the test method,[ whereas the remaining 11 studies used ELISA assay. Among the 12 included studies, 2 studies reported blinded interpretation of TNF-α assay independent of the reference standard.[ All the studies except 4[ were designed as prospective studies. The clinical characteristics and other information are outlined in Table 1. By the QUADAS-2 tool, it was found that the quality of studies for our research was generally good. The results are presented in Fig. 2A and B.

Table 1

Summary of included studies.

Figure 2

Methodological quality assessment of studies of the TNF-αassay. (A) Graph of risk of bias and applicability concerns. (B) Summary of risk bias and applicability concerns. TNF-α = tumor necrosis factor-alpha.

Diagnostic accuracy

The forest plots of sensitivity and specificity of TNF-α test for the diagnosis of TP are shown in Fig. 3. The sensitivity varied between 0.71 and 0.96 (pooled 0.85, 95% CI, 0.81–0.89), whereas specificity ranged from 0.63 to 0.93 (pooled 0.80, 95% CI, 0.77–0.83). The PLR was 4.12 (95% CI, 3.07–5.53), the NLR was 0.20 (95% CI, 0.15–0.27), and the DOR was 21.93 (95% CI, 12.85–37.43). The I values of sensitivity, specificity, PLR, NLR, and DOR were 13.4%, 66.5%, 67.1%, 33.4%, and 52.7% respectively. So the overall sensitivity and NLR were pooled by the mixed-effects model (I < 50%), and the others were pooled by the random-effects model to perform the analysis because the heterogeneity across studies showed significant difference (P < 0.05, I > 50%).

Figure 3

Forest plots of sensitivity and specificity for TNF-α assay for the diagnosis of TPE. The point estimates of sensitivity and specificity from each study are shown as solid circles. Error bars indicate 95% CI. CI = confidence interval, TNF-α = tumor necrosis factor-alpha, TPE = tuberculous pleural effusion.

We also performed subgroup analysis. In the 5 studies performed in area with high TB incidence,[ the pooled sensitivity was 0.86 (95% 0.80–0.90) and specificity was 0.78 (95% 0.72–0.84). The remaining 7 studies conducted in area with low TB incidence, the pooled sensitivity and specificity was 0.84 (95% 0.78–0.89) and 0.81 (95% 0.77–0.85) respectively. Subgroup analysis difference was not statistically significant compared with pooled results from all included studies (P = 0.978).

To summarize the global diagnostic performance of the test, the summary receiver operating characteristic (SROC) curve was generated and the Q-value, the point represents the maximum polymerization spot of sensitivity and specificity in the SROC curve, was also calculated.[ As is shown in Fig. 4, in our study, the area under the curve (AUC) was 0.89 and the Q-value was 0.82, suggesting a relatively high diagnostic accuracy.

Figure 4

Summary receiver operating characteristic curve (SROC) of TNF-α assay for the diagnosis of TPE. The size of each solid circle represents the sample size of each study. The regression SROC curve indicates the overall diagnostic accuracy. SROC = Summary receiver operating characteristic curve, TPE = tuberculous pleural effusion.

Meta-regression analysis

To explore the possible reasons for the heterogeneity, a meta-regression analysis based on sample size (≥100 or <100), setting (area with high TB incidence or area with low TB incidence) and diagnosis standard (by gold standard or clinical course) was performed. In our study, none of the above covariates included in the meta-regression was found to be the significant source of heterogeneity (P < 0.05) (Table 2).

Table 2

Meta-regression of potential heterogeneity within the included studies.

Publication bias

The Deeks’ test was performed to test the publication bias and it was insignificant (P = 0.71). The funnel plot for publication bias (Fig. 5) is also basically symmetry. The results suggested a very low likelihood of publication bias.

Figure 5

Deeks's funnel graph of publication bias of included studies. The statistically insignificant value (P = 0.70) for the slope coefficient suggested symmetry in the data and a low likelihood of publication bias.

Discussion

With the high prevalence of tuberculosis and the complexities in diagnosing pleural tuberculosis, it is crucial to develop new methods that are rapid, efficient, and noninvasive. Detection of TNF-α level in pleural fluid has been widely studied throughout the world. Some researchers proposed that the TNF-α assay was a useful noninvasive tool to aid the diagnosis of pleural tuberculosis, whereas others hold the opposite opinion.

The present meta-analysis investigated the overall accuracy of pleural TNF-α test in the TPE with a sensitivity of 0.85 (95%CI, 0.81–0.89) and a specificity of 0.80 (95% CI, 0.77–0.83) and, which means nearly 15% of non-TPE patients would be missed and almost 20% of patients without tuberculosis would be inaccurately treated. So the positive results may not confirm TPE entirely and negative tests do not indicate the absence of TPE. To present a global summary of the test performance, the SROC curve was applied. The present meta-analysis based on the SROC curve has shown the maximum joint sensitivity and specificity (Q value) was 0.82, and the AUC was 0.89, indicating that the level of the overall accuracy was relatively high.

DOR, the ratio of the odds of positive test results in the diseased relative to the odds of positive test results in the nondiseased, is another indicator of test accuracy. DOR combines the data from sensitivity and specificity into a single number, reflecting the correlation between the diagnostic method results and the likelihood of the disease. A higher DOR value indicates a better discriminatory ability of the diagnostic method.

The mean DOR value was 21.93 (95% CI, 12.93–37.43) in the present study, indicating that the TNF-α test plays a role for the diagnosis of TPE. Meanwhile, both PLR and NLR are also presented as measures of diagnostic accuracy in our study because likelihood ratios are considered to be more clinically expressive.[ A greater PLR or a lower NLR means a better diagnostic value of the diagnostic test. The pooled PLR was 4.12 and NLR was 0.20 in our study, indicating that compared to patients without TPE, TPE patients have an ∼4-fold higher chance of being TNF-α assay positive. However, a patient could have a 20% chance of having TPE if he gets a negative TNF-α assay result.

Exploration of heterogeneity is an important part of the meta-analysis. In our study, the I test for the pooled specificity, PLR, and DOR suggested that the heterogeneity among the studies was significant. Because of the different cut-offs between the included studies, the threshold effect is one of the most important causes of heterogeneity; therefore, we used the Spearman correlation coefficient to analyze the threshold effect. The results found that there is no correlation between the sensitivity and specificity (P < 0.05), suggesting that the threshold effect is not the source of heterogeneity. Meta-regression was performed to find the potential reason for heterogeneity. Variables like sample size (≥100 or < 100), setting (area with high TB incidence or area with low TB incidence), and diagnosis standard (by gold standard or clinical course) were included in the meta-regression analysis but none of them were observed to substantially affect the diagnostic accuracy of TNF-α for TPE.

The present meta-analysis suggests that the TNF-α assay play a vital role for TPE but it is far from perfection. Combination with other markers may be a reliable way to improve the diagnostic accuracy. Momi et al[ found that combined sensitivity and specificity of TNF-α plus VEGF is better than testing TNF-α alone, with a higher sensitivity of 88.9%. Wong et al[ reported that the combination of TNF-α with IFN-γ gained a high-diagnostic performance. The combination of TNF-α and ADA has been found to improve the diagnostic value of TPE with a high specificity of 96.3%.[

Our results suggest that the accuracy of TNF- α test was relatively high, which was in accordance with numerous previous studies. However, it is in disagreement with those of Soderblom et al[ and Chomej et al[ who found that TNF-α is not a good marker to separate tuberculous from malignant or parapneumonic pleural fluid. This discrepancy may be due to the inhomogeneity of the patients included in their nontuberculous group.[

It should be emphasized that this meta-analysis still has some limitations although comprehensive search strategy and data extraction were carried out. First, studies published in languages other than English or Chinese were excluded, and unpublished studies or abstracts from conferences were also filtered due to insufficient data, which may lead to publication bias, although the present meta-analysis shows no publication bias. Second, only 12 studies with 1022 cases were included. The pooled results generated from the limited samples may limit the interpretation of the meta-analysis. Third, the misclassification bias may occur since not all the tuberculous pleurisy patients in the studies were diagnosed by bacteriological or histological assessments or on the gold standard combination of both. In fact, 5 studies included in the meta-analysis used a mixture of bacteriological, histological, or clinical assessments.[ The issue regarding accuracy of diagnosis can cause nonrandom misclassification, contributing to biased results.

Conclusion

In conclusion, our study demonstrated that the TNF-α assay play a vital role in the diagnosis of TPE, whereas other test results or clinical findings should be interpreted together with the test to improve the diagnostic accuracy.