Accuracy of the Bronchoalveolar Lavage Enzyme-Linked Immunospot Assay for the Diagnosis of Pulmonary Tuberculosis: A Meta-analysis.

Assessing of local immune response may improve the accuracy of pulmonary tuberculosis (PTB) diagnosis. Many studies have investigated diagnosing PTB based on enzyme-linked immunospot (ELISPOT) assay of bronchoalveolar lavage (BAL) fluid, but the results have been inconclusive. We meta-analyzed the available evidences on overall diagnostic performance of ELISPOT assay of BAL fluid for diagnosing PTB.A systematic literature search was performed using PubMed, Embase, Wangfang, Weipu, and CNKI. Data were pooled on sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR). Overall test performance was summarized using summary receiver operating characteristic curves and the area under the curve (AUC). Deeks test was used to test for potential publication bias.Seven publications with 814 subjects met our inclusion criteria and were included in this meta-analysis. The following pooled estimates for diagnostic parameters were obtained: sensitivity, 0.90 (95% CI: 0.85-0.94); specificity, 0.80 (95% CI: 0.77-0.84); PLR, 5.08 (95% CI: 2.70-9.57); NLR, 0.13 (95% CI: 0.06-0.28); DOR, 49.12 (95% CI: 12.97-186.00); and AUC, 0.96. No publication bias was identified.The available evidence suggests that ELISPOT assay of BAL fluid is a useful rapid diagnostic test for PTB. The results of this assay should be interpreted in parallel with clinical findings and the results of conventional tests.

INTRODUCTION

Tuberculosis continues to be one of the world's biggest threats; it now ranks as a leading cause of infection-related mortality worldwide.[1,2] In 2014, tuberculosis was diagnosed in an estimated 9.6 million people around the world, and it was linked to an estimated 1.5 million deaths.[1] Pulmonary tuberculosis (PTB) is the major manifestation of the disease.[3] Despite numerous diagnostic improvements, identification of Mycobacterium tuberculosis remains the gold standard for PTB diagnosis. However, culturing the pathogen is time-consuming, and false negative results are obtained for approximately 20% of patients with PTB.[4]

A rapid alternative is microscopic detection of acid-fast bacilli (AFB) in bronchial secretions or bronchoalveolar lavage (BAL) fluid. However, suitable specimens are difficult to obtain, sensitivity of microscopic detection has been shown to vary across 3 consecutive specimens,[5] and the test gives false negative results in up to 50% of patients with PTB.[6] Nucleic acid amplification tests (NAATs) can provide results several weeks earlier than culture, but these tests are often too expensive and complex for routine use in resource-limited settings. In addition, although NAATs show high specificity, their sensitivity can be variable and low.[7-10] The automated, self-contained NAAT known as the Xpert MTB/RIF assay shows higher sensitivity and specificity than conventional NAATs[11,12] for detecting M. tuberculosis and rifampicin resistance. However, it is expensive and available only in specialized institutions.[13,14] The tuberculin skin test, while simple, can show false positive results in the presence of Bacille Calmette Guérin and many bacteria that are not M. tuberculosis, increasing the risk of misdiagnosis.[15] The limitations of these diagnostic approaches highlight the need to identify new diagnostic tools.

Interferon-gamma release assays (IGRAs) are increasingly used to detect M. tuberculosis infection.[16] IGRAs, known as the enzyme-linked immunospot (ELISPOT) assay and enzyme-linked immunosorbent assay (ELISA), are based on interferon-gamma (IFN-γ) secretion by lymphocytes exposed to M. tuberculosis-specific antigens: early secreted antigenic target 6 (ESAT-6) and culture filtrate protein 10 (CFP-10). Although IGRAs are now recommended as an alternative to the tuberculin skin test for targeted testing,[17,18] the diagnostic accuracy of IGRAs show suboptimal diagnostic accuracy with both blood and pleural fluid samples.[19,20] Studies suggest that applying IGRAs to BAL fluid may improve diagnostic performance, particularly for diagnosis of smear-negative PTB, since this fluid comes from the site of infection (lungs). After stimulation with ESAT-6 and CFP-10, the concentrations of T-cells secreting IFN-γ are 10- to 100-fold higher in BAL fluid than in peripheral blood in patient with PTB.[7,21]

Numerous studies have aimed to extend the use of BAL ELISPOT to the diagnosis of PTB, especially smear-negative PTB. The results have been variable, leading us to meta-analyze the available evidence to comprehensively assess the overall accuracy of BAL ELISPOT for diagnosing PTB.

METHODS

Search Strategy and Study Selection

A systematic search was performed using PubMed, Embase, Wanfang, Weipu, and CNKI to identify studies of the usefulness of BAL ELISPOT to diagnose PTB that were published up to July 2015. The following search terms were used: “bronchoalveolar lavage” OR “bronchoalveolar lavage fluid” AND “Enzyme-linked Immunospot” OR “ELISPOT” OR “Interferon-gamma release assays” OR “IGRA” AND “tuberculosis.” Articles were also identified using the “related-articles” function in PubMed. References within identified articles were also searched manually.

A study was included in this meta-analysis when it fulfilled the following inclusion criteria: it was an original research article published in English or Chinese; it applied the ELISPOT assay to BAL fluid for the diagnosis of PTB; it provided sufficient data about true positive, false positive, false negative, and true negative results; and it involved at least 20 participants to reduce selection bias. Institutional review board approval was not required for this retrospective meta-analysis.

Data Extraction and Quality Assessment

Two reviewers independently assessed articles for eligibility to be included in this meta-analysis; discrepancies in assessment were resolved by consensus. The following data were retrieved from each study: 1st author, publication year, country, number of patients, diagnostic standard, test method, sample, numbers of true positive, numbers of false positive, numbers of false negative, numbers of true negative, indeterminate results, and methodological quality. The methodological quality of each study was assessed using the QUADAS-2 instrument.[22] QUADAS-2 assesses risk of bias in 4 parts: patient selection, index test, reference standard, flow, and timing. It assesses applicability concerns in 3 parts: patient selection, index test, and reference standard.

Statistical Analysis

Standard methods recommended for diagnostic accuracy meta-analysis were used.[23,24] The following indices of test accuracy were computed for each study: sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic odds ratio (DOR), and the area under the curve (AUC). The analysis was based on a summary receiver operating characteristic (SROC) curve.[23,25] Pooled estimates of sensitivity, specificity, and related indices were calculated across studies using a random-effects model or a fixed-effects model based on the results of heterogeneity tests. Heterogeneity between studies was evaluated using the χ2 test and Fisher exact tests. Deeks funnel plot was used to detect the potential presence of publication bias.[26] Meta-analysis was carried out using STATA 12.0 (Stata Corp., College Station, TX), Meta-DiSc 1.4 (XI. Cochrane Colloquium, Barcelona, Spain), and RevMan 5.2 (Cochrane Collaboration, Oxford, UK). All statistical tests were 2-sided, and P < 0.05 was considered significant.

RESULTS

Characteristics and Quality of the Included Studies

Seven studies assessing BAL ELISPOT for PTB diagnosis were included in the meta-analysis based on the inclusion criteria (Figure 1, Table 1).[27-33] The average sample size was 116 (range, 32–347). All patients had been enrolled consecutively and prospectively. Patients were diagnosed with tuberculosis using a recently developed algorithm,[34] and M. tuberculosis culture served as the reference standard. In all 7 studies, the ELISPOT-based T-SPOT-TB assay was performed. In 6 studies, the BAL ELISPOT result was considered positive when the test well contained at least 5 more spot-forming cells (SFCs) as the control well and when the test well contained twice the number of SFCs as the control well.[27-32] The BAL ELISPOT result was considered negative if it did not meet the criteria for a positive result and if the positive control well contained at least twice the number of SFCs as the control well. Results that met neither of these definitions were considered to be indeterminate. In 1 study, in contrast, a BAL ELISPOT result was considered positive when the test well contained at least 6 more SFCs and had twice the number of SFCs as the control well.[33]

FIGURE 1

Flow diagram of included and excluded studies.

TABLE 1

Clinical Summary of Included Studies Examining the Diagnostic Accuracy of Bronchoalveolar Lavage Enzyme-Linked Immunospot

QUADAS-2 was proposed in 2011 and was integrated into RevMan 5.2 in 2012.[22,35] This instrument assesses methodological quality in terms of patient selection, index test, reference standard, and flow and timing. When a criterion is fulfilled, an answer of Yes is given; if it is unclear whether a criterion is fulfilled, Unclear is reported; and if a criterion is not fulfilled, No is reported. These responses for each criterion are then converted into risk of bias and applicability concerns. The quality of studies in our meta-analysis was generally good, but 3 studies were judged to have unclear risk of bias (Figure 2). This risk of bias was related to the index test in all 3 studies[27,28,33] as well as to the reference standard in 2 of them.[27,33] Two studies were judged to have high risk of bias,[29,31] related to flow and timing.

FIGURE 2

Quality assessment of studies of the bronchoalveolar lavage enzyme-linked immunospot assay. (A) Graph of risk of bias and applicability concerns. (B) Summary of risk of bias and applicability concerns.

Diagnostic Accuracy

Figure 3 shows estimates of sensitivity and specificity for diagnostic accuracy of BAL ELISPOT. Sensitivity ranged from 0.73 to 1.00 (pooled: 0.90; 95% CI: 0.85–0.94), and specificity ranged from 0.48 to 1.00 (pooled: 0.80; 95% CI: 0.77–0.84). The following pooled parameters were also calculated: PLR, 5.08 (95% CI: 2.70–9.57); NLR, 0.13 (95% CI: 0.06–0.28) (Figure 4); and DOR, 49.12 (95% CI: 12.97–186.00). Chi-squared values for these pooled estimates were 10.03 (P = 0.123) for sensitivity, 40.62 (P = 0.000) for specificity, 44.24 (P = 0.000) for PLR, 18.05 (P = 0.006) for NLR, and 28.21 (P = 0.000) for DOR. These values indicate significant heterogeneity among the studies.

FIGURE 3

Forest plot showing estimates of sensitivity and specificity for the bronchoalveolar lavage enzyme-linked immunospot assay. Point estimates of sensitivity and specificity from each study are shown as solid circles. Error bars indicate 95% confidence interval (CI).

FIGURE 4

PLR and NLR for the bronchoalveolar lavage enzyme-linked immunospot assay. NLR = negative likelihood ratio, PLR = positive likelihood ratio.

In the subset of 3 studies conducted in settings with low tuberculosis incidence,[27,28,30] pooled sensitivity was 0.92 (95% CI: 0.85–0.97) and specificity was 0.83 (95% CI: 0.79–0.87). Among the remaining studies, conducted in settings with high tuberculosis incidence,[29,31-33] pooled sensitivity was 0.88 (95% CI: 0.81–0.94) and specificity was 0.74 (95% CI: 0.66–0.81). Comparing these subgroup analyses with results pooled from all 7 studies suggests that BAL ELISPOT performs better in settings of low tuberculosis incidence, but the difference in performance in the 3 cases was not statistically significant (P = 0.585).

An SROC curve was calculated for BAL ELISPOT, this curve plots sensitivity against 1-specificity from individual studies (Figure 5). As a global measure of test efficacy, we determined the Q-value; this is the point of intersection between the SROC curve and the diagonal running from the left upper corner to the lower right corner of the ROC space. The Q-value corresponds to the highest common value of sensitivity and specificity, as well as the point where sensitivity equals specificity. The SROC curve was positioned near the desirable upper left corner of the plot, suggesting good performance, and Q-value for sensitivity and specificity was 0.90 (SEM 0.036), with an AUC of 0.96 (SEM 0.025). These data indicated high overall accuracy for the BAL ELISPOT assay in PTB diagnosis.

FIGURE 5

Summary receiver operating characteristic (SROC) curves for the bronchoalveolar lavage enzyme-linked immunospot assay. Solid circles represent each study included in the meta-analysis. The size of each study is indicated by the size of the solid circle. The regression SROC curves summarize the overall diagnostic accuracy.

Multiple Regression Analysis and Publication Bias

The effect of study quality on the relative DOR of BAL ELISPOT was assessed by performing meta-aggression involving setting (low vs high tuberculosis incidence), sample size, blinding design, publication year, and rate of indeterminate results (Table 2). Three studies were performed in areas with a low tuberculosis incidence[27,28,30] and 4 studies in areas with high tuberculosis incidence.[29,31-33] Two studies showed rates of indeterminate results >30%.[29,31] Meta-regression showed that setting, sample size, blinding design, and publication year did not substantially affect the diagnostic accuracy of BAL ELISPOT (P > 0.05). However, indeterminate results did affect diagnostic accuracy (P = 0.013). In addition, we found that the inclusion of HIV patients in 2 studies[29,31] significantly affected heterogeneity (P = 0.013).

TABLE 2

Meta-Regression on Diagnostic Accuracy of Bronchoalveolar Lavage Enzyme-Linked Immunospot

Deeks funnel plot asymmetry test was used to assess likelihood of publication bias. Although the funnel plots for publication bias showed some asymmetry due to the limited number of studies (Figure 6), the P value associated with Deeks test was not significant (P = 0.48). This suggests symmetry in the data and low likelihood of publication bias.

FIGURE 6

Funnel graph to assess risk of publication bias in studies of the bronchoalveolar lavage enzyme-linked immunospot assay. The funnel graph plots the log of the diagnostic odds ratio (DOR) against the standard error of the log of the DOR (an indicator of sample size). Solid circles represent each study in the meta-analysis. The regression line is shown.

DISCUSSION

Currently, 2 types of IGRAs are commercially available: the ELISA-based QFT-G or QFT-IT, and the ELISPOT-based T-SPOT-TB. Both ELISPOT and ELISA measure IFN-γ release after T cell stimulation with the M. tuberculosis-specific antigens ESAT-6 and CFP-10. However, ELISPOT has been reported to be more stable and sensitive.[15,36] This has led investigators to examine whether ELISPOT may be useful for tuberculosis screening. Those studies have focused on BAL fluid, since IGRAs show inadequate sensitivity and specificity with pleural and blood samples.[19,20,24] Assaying BAL fluid may boost sensitivity since M. tuberculosis-specific lymphocytes are concentrated at the site of infection (lungs).[27] Given the rationality of local immunodiagnosis using BAL ELISPOT,[37,38] a growing number of studies have evaluated the diagnostic accuracy of BAL ELISPOT for PTB, but results have been mixed.

Our meta-analysis of the available evidence showed a pooled sensitivity of 0.90 and specificity of 0.80. The relatively high sensitivity indicates that it would be possible to exclude PTB if a patient's BAL ELISPOT result was below the cut-off value. However, the moderate specificity limits the usefulness of the test. At the same time, our meta-analysis indicated an AUC of 0.96 for the SROC curve, which presents a global summary of test performance and shows the trade-off between sensitivity and specificity. Since an AUC of 1.0 (100%) indicates perfect ability to discriminate cases from noncases, our meta-analysis suggests a relatively high level of overall diagnostic accuracy. Consistent with this, we calculated a pooled DOR of 49.12 for BAL ELISPOT. The DOR indicates the ratio of the odds of positive test results in the diseased group to the odds of positive test results in the nondiseased group. Higher values of DOR, which ranges from 0 to infinity, indicate better discriminatory performance. The pooled DOR in our meta-analysis suggests that the assay is helpful for diagnosing PTB. Our results are consistent with reports that BAL ELISPOT is a more sensitive supplementary test than blood or pleural ELISPOT.[36] In addition, BAL fluid is easy to obtain in patients who are unable to produce sputum. Thus, BAL ELISPOT may afford an alternative for patients without sputum.

Since the PLR and NLR are more clinically meaningful and easier to interpret than the AUC, we used these indices to assess the diagnostic accuracy of BAL ELISPOT. PLR indicates how much the odds of a condition are increased by a positive test, while NLR indicates how much they are decreased by a negative test. Larger PLR means greater diagnostic accuracy, whereas a smaller NLR is better. Our meta-analysis indicated a pooled PLR of 5.08, meaning that patients with PTB had a 5-fold higher chance of giving a positive BAL ELISPOT result than did patients without PTB. This is not a large enough difference for clinical use. Similarly, the pooled NLR was 0.13, indicating 13% probability that a patient with a negative BAL ELISPOT result actually had tuberculosis, which is not low enough to exclude PTB in the clinic.

Based on World Bank criteria, 5 studies in our meta-analysis were conducted in high-income countries,[27-31] while 2 were conducted in low- or middle-income countries.[32,33] Five of 7 studies used the tuberculin skin test, which gave results consistent with BAL ELISPOT but with lower sensitivity and specificity.[27-30,32] A recent meta-analysis concluded that the overall sensitivity of the tuberculin skin test for active TB is 77%,[39] which is lower than the 90% calculated here for BAL ELISPOT. In addition, the tuberculin skin test is influenced by previous tuberculosis history, while BAL ELISPOT is not. However, BAL ELISPOT is not available in resource-limited settings, where the budget and infrastructure for IGRAs are lacking. This helps to explain why, even though IGRAs have been available for several years, the tuberculin skin test is still much more widely used to screen people with a positive immune response against M. tuberculosis.[40] Studies suggest that if an individual gives negative results on both the tuberculin skin test and an IGRA, active tuberculosis can be ruled out with greater than 95% confidence.[41,42]

The gold standard for tuberculosis diagnosis remains culture of M. tuberculosis in liquid or solid media. Both types of culture require weeks, and this assay fails to detect approximately 20% of PTB cases.[4] The pathogen can be detected microscopically in an AFB assay, which is rapid but of limited sensitivity.[6] Although an M. tuberculosis-specific NAAT is frequently used for rapid PTB diagnosis, the sensitivity of NAAT with sputum or BAL fluid is good in patients with positive results on AFB sputum smears,[43] but it gives false negative results for more than 50% of patients with negative smear results.[8,44] Similarly, the Xpert MTB/RIF assay, an automated NAAT that performs much better in patients with PTB than conventional NAATs,[11,12] shows higher sensitivity in smear-positive patients (98%) than in smear-negative ones (67%).[13] Thus, studies have concluded that although the Xpert assay can diagnose PTB better than smear microscopy, its overall sensitivity is less than that of the culture test.[45-47] In addition, Xpert MTB/RIF requires a stable and uninterrupted electrical power supply, temperature control, and annual calibration of the cartridge modules.[14] These considerations indicate that no existing microbiological or immunological method on its own is adequate to diagnose or exclude PTB.

Since guidelines recommend bronchoscopy for patients with suspected tuberculosis who give negative results on AFB sputum smears,[30,34] we suggest that a negative smear result should lead to prompt bronchoscopy and BAL ELISPOT. Indeed, a comparison of various methods to diagnose active PTB (BAL fluid microscopy, BAL fluid culture, transbronchial biopsy for histology, BAL NAAT, and BAL ELISPOT) concluded that BAL ELISPOT was the best method for rapid diagnosis.[30]

Two of 7 studies included in this meta-analysis used BAL NAAT for the diagnosis of PTB.[28,30] The percentage of patients with PTB not diagnosed by NAAT was 64.8%, compared to only 14.1% for BAL ELISPOT.[28] The BAL NAAT showed lower sensitivity and specificity than BAL ELISPOT. These results suggest that BAL ELISPOT shows superior, more consistent diagnostic performance than BAL NAAT, especially for patients with smear-negative PTB. Thus, BAL ELISPOT provides an alternative technique for detecting PTB associated with negative results in the AFB smear test and culture test. At the same time, the expense, invasiveness and risk of obtaining BAL fluid, and assaying it means that this alternative may not be appropriate for all patients in all settings.

Five of 7 studies included in this meta-analysis also examined blood ELISPOT for PTB diagnosis.[27-31] The diagnostic accuracy of BAL-ELISPOT is superior to that of blood ELISPOT, probably reflecting the fact that in active PTB, antigen-specific T cells clonally expand and concentrate at the site of infection (lungs). In addition, a positive BAL ELISPOT result effectively discriminates between active and latent tuberculosis.[28-30]

Significant heterogeneity was detected among the included studies, so we performed meta-regression analysis to investigate the possible sources of this heterogeneity. Although diagnostic accuracy of BAL ELISPOT tended to be higher in settings of low tuberculosis incidence than in settings of high incidence, the difference was not significant. In contrast, the relatively high rate of indeterminate results did affect the diagnostic accuracy of BAL ELISPOT (P = 0.013), as did inclusion of HIV patients[29,31] (P = 0.013). These results are consistent with reports that the large proportion of indeterminate results makes the BAL ELISPOT less than ideal in areas of high tuberculosis incidence where immunodiagnosis focuses on the lung.[29,31] They are also consistent with a report that BAL ELISPOT performed poorly in an area with high HIV burden.[31] The presence of immunosuppressive cytokines or cells, including immunosuppressive pulmonary macrophages and regulatory T cells, may help explain the high proportion of inconclusive results.[48] Another contributor may be high spot count in the negative control.[29] A 3rd reason may be that the IFN-γ detected in the assay is secreted by terminally differentiated effector T-cells rather than by M. tuberculosis-specific effector T-cells. The accumulation of terminally differentiated effector T-cells is a hallmark of early immune senescence in advanced HIV/AIDS in South Africa,[31,49] and terminally differentiated effector T-cells may concentrate at the site of disease in patients with active PTB.[50] The 4th reason may be low numbers of cells harvested at bronchoscopy, particularly in technically difficult procedures. Multicenter clinical studies should be performed to rigorously assess the diagnostic performance of BAL ELISPOT in patients with PTB.

We used the QUADAS-2 instrument[22] to assess the quality of included studies on the basis of information in the title, Introduction, Methods, Results, and Discussion. This is an improved version of the original QUADAS instrument[51] that takes into account more details of the study, such as the explanation of indeterminate results. Since the studies often reported inadequate information, assessment of several criteria was “unclear,” increasing risk of bias and applicability concerns.

Our meta-analysis is limited by several factors. First, only 7 publications were included, reflecting the strict search strategy and study selection. The limited patient numbers may have influenced the outcomes, so statistical power may be inadequate for drawing definitive conclusions about the ability of BAL ELISPOT to diagnose PTB. Further, larger studies may be needed to confirm the diagnostic value of BAL ELISPOT. Second, because of the few studies included, we were unable to perform subgroup analysis to assess studies that included HIV patients separately from other studies. Further study of BAL ELISPOT for diagnosing PTB in HIV patients is needed, which will allow rigorous meta-analysis to be carried out. Third, all the studies in our meta-analysis were of modest methodological quality, based on QUADAS-2 assessment. This was due mainly to lack of reporting of key information.

CONCLUSIONS

The present meta-analysis suggests that BAL ELISPOT may significantly aid the diagnosis of PTB. Its relatively high sensitivity may make it suitable for PTB screening. Further study and optimization are required to reduce the rate of inconclusive results. Although BAL ELISPOT shows better diagnostic performance than both the tuberculin skin test and blood ELISPOT, it is not accurate enough on its own to diagnose or exclude PTB. Effective combination of different diagnostic tests remains the most critical element in controlling tuberculosis.