Diagnostic accuracy of pleural fluid tumor necrosis factor-α in tuberculous pleurisy: A meta-analysis.

BACKGROUND: Pleurisy is a common extra pulmonary complication of tuberculosis, but current methods for diagnosing it are fairly crude. Here we product a meta-analysis for the available evidence on the ability of tumor necrosis factor-α (TNF-α) in pleural fluid to serve as a diagnostic marker of tuberculous pleurisy (TP).
MATERIALS AND METHODS: We searched the PubMed, EMBASE, and Google Scholar databases systematically for studies measuring sensitivity, specificity and other measures of diagnostic accuracy of pleural fluid TNF-α in the diagnosis of TP were meta-analyzed by Stata, version 12 and meta-disc.
RESULTS: A total of six publications reporting seven case-control studies were identified. Pooled results indicated that pleural fluid TNF-α showed a diagnostic sensitivity of 0.89 (95% confidence interval [95% CI] 0.83-0.93; range, 0.42-1.0) and a diagnostic specificity of 0.82 (95% CI: 0.78-0.86; range, 0.58-0.98). The pooled positive likelihood ratio was 4.78 (95% CI: 3.32-6.89); the negative likelihood ratio, 0.16 (95% CI: 0.1-0.27); the diagnostic odds ratio, 32.43 (95% CI: 14.48-72.6); and the area under the curve was 0.8556 (standard error of mean 0.0559).
CONCLUSION: Pleural fluid TNF-α levels shows relatively high sensitivity but insufficient specificity for diagnosing TP. Pleural fluid TNF-α measurement may be useful in combination with clinical manifestations and conventional tests such as microbiological examination or pleural biopsy.

INTRODUCTION

Tuberculosis remains a frequent and important infectious disease worldwide,[1] it occurs with, in approximately 30% of all cases, extrapulmonary involvement in the form of tuberculous pleurisy (TP).[2] And TP increases the morbidity and mortality associated with tuberculosis, and it is a major economic and health burden all over the world.[3]

The differential diagnosis of TP reliably is challenging although many conventional diagnostic methods are used, including microscopic analysis of pleural fluid or sputum smears, culturing for mycobacteria and even performing a pleural biopsy. Each of these methods has significant limitations: Microscopic analysis of pleural fluid is rarely positive (<5%),[456] culture of pleural fluid shows poor sensitivity (24-58%), culturing for Mycobacterium tuberculosis requires several weeks,[67] and pleural biopsy, which is considered the best method for confirming TP diagnosis,[7] is invasive and technically difficult.[8] Even more invasive procedures such as thoracoscopy or thoracotomy have been used for differential diagnosis of TP, but these complex procedures can cause complications and even increase morbidity.[9]

These led investigators to explore several biomarkers as possible diagnostic indicators. TNF-α is a small polypeptide with pleiotropic effects on biological and immunological processes.[10] Its release by mesothelial cells in pleura contributes to the occurrence and development of TP through three mechanisms: It acts as a proinflammatory cytokine to attract neutrophils to kill mycobacteria directly as part of an innate immune response; it acts synergistically with interferon (IFN-γ) to activate macrophages as part of an adaptive immune response; and it recruits naïve T cells to the granuloma.[11] Tumor necrosis factor-α (TNF-α) levels in the pleural fluid are significantly higher in patients with tuberculosis than in patients with pulmonary malignancy.[10] Some studies have reported that levels of TNF-α in pleural fluid provide high diagnostic sensitivity (96.0%) and specificity (93.0%).[12] Other studies, however, have reported much lower corresponding values of 70% and 66%.[10] However, the comprehensive picture of the diagnostic usefulness of TNF-α levels in pleural fluid is not obvious. Therefore in this article we will product a meta-analysis to explore the diagnostic accuracy of pleural fluid TNF-α in TP.

MATERIALS AND METHODS

We conducted this meta-analysis according to the guidelines of the PRISMA[13] and the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) checklist.[14]

Search strategy and selection criteria

We searched PubMed and EMBASE to identify eligible studies through May 15, 2015. The following search terms were used: “Tuberculosis” or “TB disease” or “active tuberculosis” or “pleural effusions” or “TP” and “TNF-α” or “TNF-alpha” or “cytokines” and “sensitivity” and “specificity” and “diagnosis”. Only English-language articles were considered. Reference lists of articles identified in these searches were also searched manually.

To be included in our study, studies had to

Evaluate the sensitivity and/or specificity of pleural fluid TNF-α for diagnosis of TP using.

A case-control design involving.

Case and control groups classified using clear diagnostic criteria and.

The same method to assay TNF-α.

Apply an adequate experimental method.

Unpublished data, abstracts, review articles, and letters to the editor were excluded.

Data extraction and quality assessment

Two reviewers (Z.L. and W.Q.) independently assembled a final set of eligible studies, and a third author (X.C.) was consulted to resolve disagreements. The same procedure was followed to extract data from the included studies using a standardized form. Extracted data included first author, publication year, country of the study, number and characteristics of participants, TNF-α assay method, cut-off value for TNF-α detection, sensitivity and specificity, and numbers of true positives, false positives, true negatives and false negatives. We assessed the methodological quality of the studies using the QUADAS-2 checklist, with a maximum score of 11.[14]

Statistical analysis

Standard methods recommended for meta-analyses of diagnostic test evaluations[14] were used. Analyses were performed using Stata and Meta-DiSc (XI Cochrane Colloquium, Barcelona, Spain).[15] The following measures of test accuracy were computed for each study: Sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR) and diagnostic odds ratio (DOR). Overall diagnostic performance was assessed from summary receiver operating characteristic (SROC) curves.[1617] These curves were plotted for each study using the sensitivity and specificity based on the single-test threshold identified within the same study.[1618]

Then we performed a meta-regression analysis to assess the effect of the baseline in each study on the relative DOR of pleural fluid TNF-α in TP diagnosis.

We used a random-effect model to meta-analyze sensitivity, specificity, and other diagnostic measures across multiple studies.[1920] To assess statistically significant variability (heterogeneity) across studies, we used Chi-squared and Fisher's exact tests. We tested for the potential presence of publication bias using Deeks’ funnel plots.[21]

RESULTS

Literature searches turned up 96 potentially eligible studies, and 86 were excluded based on review of titles and abstracts. The remaining 10 articles were read in full, and 4[15171819] were excluded because they did not apply an adequate experimental method. In the end, six publications[101112222324] assessing the diagnostic performance of pleural fluid TNF-α assay in TP were included in our analysis. One study[12] involved two control groups, and sufficient data were reported for both that we were able to treat the groups as two independent studies in the meta-analysis [Figure 1]. Thus, our review included seven studies from six publications [Table 1].

Figure 1

Flow diagram of included and excluded studies

Table 1

Characteristics of studies included in the meta-analysis

Study characteristics

The seven studies involved 159 patients with TP and 338 without it. Average sample size in the seven studies was 71 (range: 50-97). TP was diagnosed by bacteriology[1012222324] or bacteriology and histology.[11] TNF-α levels in all studies were assayed by enzyme-linked immunosorbent assay (ELISA). The control groups were patients with pulmonary malignancy or parapneumonic effusions or both.

Diagnostic accuracy

Sensitivity for pleural fluid TNF-α in TP diagnosis ranged from 0.71 to 0.96 in the seven studies, and meta-analysis of sensitivity and specificity indicated a pooled sensitivity of 0.89 (95% confidence interval [CI]: 0.83-0.93) [Figure 2]. Specificity ranged from 0.71 to 0.93, and meta-analysis showed a pooled specificity of 0.82 (95% CI: 0.78-0.86) [Figure 2]. PLR was 4.78 (95% CI: 3.32-6.89) and NLR was 0.16 (95% CI: 0.1-0.27) [Figure 3]. DOR was 32.43 (95% CI: 14.48-72.6) [Figure 4]. Significant heterogeneity (I2 > 50%) was present among the seven studies with respect to PLR data but not data for other diagnostic measures: I2 was 0% for sensitivity, 47.5% for specificity, 53.8% for PLR, 25% for NLR, and 46.9% for DOR.

Figure 2

Forest plot of estimates of sensitivity/specificity for pleural fluid TNF-α in the diagnosis of tuberculous pleurisy. Point estimates of sensitivity and specificity from each study are shown as solid circles, the size of which reflects the total number of cases and controls. Error bars show 95% confidence intervals. Numbers indicate the reference numbers of the studies

Figure 3

Forest plot of estimates of positive and negative likelihood ratios (PLR and NLR) for pleural fluid TNF-α in the diagnosis of tuberculous pleurisy. Point estimates of PLR and NLR from each study are shown as solid circles, the size of which reflects the total number of cases and controls. Error bars show 95% confidence intervals. Numbers indicate the reference numbers of studies

Figure 4

(a) Forest plot of estimates of diagnostic odds ratios for pleural fluid tumor necrosis factor-α (TNF-α) in the diagnosis of tuberculous pleurisy. Point estimates of diagnostic odds ratios from each study are shown as solid circles, the size of which reflects the total number of cases and controls. Error bars show 95% confidence intervals. Numbers indicate the reference numbers of studies. (b) Summary receiver operating characteristic curves for pleural fluid TNF-α. Each study is depicted as a solid circle, the size of which reflects the total number of cases and controls

Summary receiver operating characteristic curves were generated by plotting sensitivity against (1-specificity) for individual studies [Figure 4]. The curves did not lie near the desired upper left corner, and the maximum joint sensitivity and specificity was 0.89, with an area under the curve (AUC) of 0.8556 (standard error of mean 0.0559). On the other hand, DOR was 32.43, suggesting relatively high overall accuracy.

Multiple regression analysis

Across the seven studies, TNF-α cut-off values in the ELISA and different diseases in the control group were different significantly [Table 1]. These levels did not appear to affect significantly diagnostic accuracy [Table 2].

Table 2

Weighted meta-regression of the effects of diseases in control groups and cut-off values on diagnostic accuracy of pleural fluid TNF-α

Publication bias

Funnel plots showed some asymmetry [Figure 5], presumably reflecting the small number of studies included in our meta-analysis. Nevertheless, Deeks’ test gave a P = 0.11, suggesting that our analysis did not carry significant risk of publication bias.

Figure 5

Funnel plot for evaluating publication bias among the six articles (seven studies) included in the meta-analysis. The log of the diagnostic odds ratio (DOR) is plotted against the standard error of log DOR; the latter serves as an indicator of sample size. Each article is shown as a solid circle, and the regression line is shown

DISCUSSION

Given the numerous limitations associated with current methods for diagnosing TP, researchers have explored whether pleural fluid biomarkers such as TNF-α can serve as diagnostic markers.[2526272829] These studies have given conflicting results about the diagnostic performance of pleural fluid TNF-α, so here we meta-analyzed the available evidence. Our analysis suggests that pleural fluid TNF-α measurements by themselves are not sufficiently sensitivity (0.89) and specificity (0.82) to diagnose TP, but they can provide complementary diagnostic information when used in combination with assays of other pleural fluid biomarkers and conventional tests such as bacteriological examination or pleural biopsy.

A meta-analysis of the seven included studies indicated a pooled DOR of 32.4 for the pleural fluid TNF-α assay, indicating a relatively high accuracy. DOR, which combines sensitivity and specificity data that serves as an aggregate indicator of test accuracy,[30] is the ratio of the odds of positive test results in individuals with disease relative to the odds of positive test results in individuals without disease.[31]

The SROC curve and its AUC present an overall summary of test performance and display the trade-off between sensitivity and specificity.[31] In the present meta-analysis, we found the sensitivity of the pleural fluid TNF-α assay to be 0.89; specificity, 0.82; maximum joint sensitivity and specificity, 0.89; and AUC, 0.8556. These results also indicate relatively high accuracy.

Diagnostic odds ratio and SROC curve analysis are not easy to interpret and use in clinical practice,[32] and likelihood ratios are considered more clinically meaningful.[3233] Therefore, we meta-analyzed the data to determine pooled PLR and NLR as measures of diagnostic accuracy. The PLR value of 4.78 suggested that the probability of being positive for pleural fluid TNF-α was nearly 5-fold higher for patients with TP than for patients without it. Although this is insufficient to serve as the sole basis for diagnosing TP, it is likely to be sufficient to allow a clinician to decide whether to initiate or continue anti-tuberculosis treatment of TP in individuals who do not present evidence of malignancy or inflammation. At the same time, NLR was 0.16 in our meta-analysis, indicating that patients negative for pleural fluid TNF-α still have a 16% chance of having TP. This provides further evidence that such an assay is inadequate, on its own, for ruling out TP. Thus, a negative pleural fluid TNF-α assay is not sufficient cause to deny or discontinue anti-tuberculosis therapy.

The reliability of meta-analyses depends on heterogeneity among the included studies, and we found significant heterogeneity in the data for PLR. Since the causes of heterogeneity can reveal systematic factors affecting the accuracy and reliability of meta-analyses,[34] we examined the seven studies more carefully. In all studies, TP was diagnosed based on bacteriology, histology or both; TNF-α was determined using an ELISA kit according to the manufacturer's guidelines (OxfordImmunotec Ltd., Abingdon, UK); and the QUADUS-2 score in each study was relatively high. In addition, inter-study variation in TNF-α cut-off values and baseline TNF-α levels in the control groups did not substantially affect diagnostic accuracy. Therefore, the basis for the heterogeneity in PLR data in our meta-analysis is unclear, and in any case, further large studies are needed to verify our findings, especially since we excluded possibly relevant studies that were not published in English or that were published only as conference abstracts or letters to the editor.

Analyses of other pleural fluid biomarkers suggest that they, like TNF-α, cannot be used alone as diagnostic indicators. A meta-analysis of the diagnostic accuracy of IFN-g in TP reported a sensitivity of 0.75, specificity of 0.82, PLR of 3.49, NLR of 0.24, and DOR of 19.04.[9] Diagnostic performance was better in meta-analysis of adenosine deaminase, which reported a sensitivity of 0.86, specificity of 0.88, PLR of 6.32, NLR of 0.15, and DOR of 45.25.[35] These results suggest none of the biomarkers is sufficient on its own. Assaying for multiple biomarkers may improve the accuracy of TP diagnosis, and such combined approaches should include TNF-α because the peptide plays such an important role in tuberculosis pathogenesis, especially in patients with auto-immune diseases who are taking TNF-α antagonists.

CONCLUSION

The available evidence suggests that the pleural fluid TNF-α assay should not be used on its own to diagnose TP or guide treatment decisions, but it can be used to complement other tests including microscopic smear examination, culture for M. tuberculosis, pleural tissue histology and response to anti-tuberculosis therapy. The diagnostic performance of TNF-α may improve by changing the detection platform from the current ELISA to more advanced flow cytometry, ELISPOT and Luminex methods.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

AUTHOR'S CONTRIBUTIONS

LZ searched the literature and assessed articles, extracted data, performed meta-analyses, and wrote the draft. WZ searched the literature assessed articles, and revised the draft. LL extracted data, checked the procedure of meta-analysis. WQ checked the study procedure and revised the draft. CX contributed in the conception of the work, revised the draft, approval of the final version of the manuscript, and agreed for all aspects of the work.