Commercial serological tests for the diagnosis of active pulmonary and extrapulmonary tuberculosis: an updated systematic review and meta-analysis.

BACKGROUND: Serological (antibody detection) tests for tuberculosis (TB) are widely used in developing countries. As part of a World Health Organization policy process, we performed an updated systematic review to assess the diagnostic accuracy of commercial serological tests for pulmonary and extrapulmonary TB with a focus on the relevance of these tests in low- and middle-income countries. METHODS AND
FINDINGS: We used methods recommended by the Cochrane Collaboration and GRADE approach for rating quality of evidence. In a previous review, we searched multiple databases for papers published from 1 January 1990 to 30 May 2006, and in this update, we add additional papers published from that period until 29 June 2010. We prespecified subgroups to address heterogeneity and summarized test performance using bivariate random effects meta-analysis. For pulmonary TB, we included 67 studies (48% from low- and middle-income countries) with 5,147 participants. For all tests, estimates were variable for sensitivity (0% to 100%) and specificity (31% to 100%). For anda-TB IgG, the only test with enough studies for meta-analysis, pooled sensitivity was 76% (95% CI 63%-87%) in smear-positive (seven studies) and 59% (95% CI 10%-96%) in smear-negative (four studies) patients; pooled specificities were 92% (95% CI 74%-98%) and 91% (95% CI 79%-96%), respectively. Compared with ELISA (pooled sensitivity 60% [95% CI 6%-65%]; pooled specificity 98% [95% CI 96%-99%]), immunochromatographic tests yielded lower pooled sensitivity (53%, 95% CI 42%-64%) and comparable pooled specificity (98%, 95% CI 94%-99%). For extrapulmonary TB, we included 25 studies (40% from low- and middle-income countries) with 1,809 participants. For all tests, estimates were variable for sensitivity (0% to 100%) and specificity (59% to 100%). Overall, quality of evidence was graded very low for studies of pulmonary and extrapulmonary TB.
CONCLUSIONS: Despite expansion of the literature since 2006, commercial serological tests continue to produce inconsistent and imprecise estimates of sensitivity and specificity. Quality of evidence remains very low. These data informed a recently published World Health Organization policy statement against serological tests. Please see later in the article for the Editors' Summary.

Introduction

Despite impressive advances in tuberculosis (TB) control over the last decade [1], missed diagnoses continue to fuel the global epidemic, leading to more severe illness for patients and enabling further transmission of Mycobacterium tuberculosis [2]. Smear microscopy and chest radiography, the primary tools used in resource-limited countries for identifying TB, often perform poorly, especially in HIV-coinfected patients [3]–[5]. Improved techniques, such as liquid culture for M. tuberculosis and nucleic acid amplification tests, are often too expensive and complex for routine use in resource-limited settings. The Xpert MTB/RIF (Cepheid), a new technology recently endorsed by the World Health Organization (WHO), provides high sensitivity for detection of TB and drug resistance [6]. WHO has issued a blueprint for Xpert's implementation [7]; however, high cost may be a barrier for scaling up this technology in many areas where the epidemic is most severe [2].

Serological tests have a long history and have been used successfully for the rapid diagnosis of many infectious diseases (e.g., HIV, syphilis, and viral hepatitis). In this paper, “serological tests” refers to blood tests that detect the humoral immune (antibody) responses to M. tuberculosis antigens. Serological tests are not to be confused with interferon-gamma release assays that measure the T-cell-based interferon-gamma response to M. tuberculosis antigens. In comparison with microscopy, serological tests appear to offer several advantages: (1) the result from a serological test using the enzyme-linked immunosorbent assay (ELISA) format could be available within hours, and the result using an immunochromatographic assay format, within minutes; (2) a serological test, if developed into a point-of-care test, could potentially replace microscopy or extend testing to lower levels of health services; and (3) in children, for whom sputum is difficult to obtain, and in patients suspected of having extrapulmonary TB, a blood test may be more practical.

Although currently the International Standards for TB Care discourages the use of serological tests in routine practice [8] and no international guideline recommends their use, dozens of commercial serological tests for TB diagnosis are offered for sale in many parts of the world [9], including Afghanistan, Bangladesh, Brazil, Cambodia, China, India, Indonesia, Kenya, Myanmar, Nigeria, Pakistan, Philippines, Russia, South Africa, Thailand, Uganda, and Viet Nam, as was recently found in a survey of 22 high TB burden countries [10]. For example, in India, numerous products with claims of high accuracy in their package inserts are available for purchase (Table S1), and an estimated 1.5 million serological tests are performed every year [10].

We are aware of four systematic reviews and one laboratory-based evaluation on this topic. The first review included only studies with a cohort or case series design and searched the literature through 2003 [11]. Performance of the tests was modest, and sensitivity decreased when only studies meeting at least two design-related criteria were included (seven studies, pooled sensitivity of 34%) [11]. Two subsequent reviews evaluating commercial serological tests for pulmonary TB (68 studies) [12] and extrapulmonary TB (21 studies) [13] found the sensitivity and specificity of these tests to be highly variable. The fourth review, a meta-analysis of in-house serological tests for the diagnosis of pulmonary TB (254 studies including 51 distinct single antigens and 30 distinct multiple-antigen combinations), identified potential candidate antigens for inclusion in an antibody-detection-based TB test in patients with and without HIV infection; however, no single antigen achieved sufficient sensitivity to replace smear microscopy [14]. A laboratory-based evaluation of 19 rapid commercial tests conducted by the WHO Special Programme for Research and Training in Tropical Diseases found that, in comparison with culture plus clinical follow-up, serological tests provided low and variable sensitivity (1% to 60%) and specificity (53% to 99%) [15].

Since the publication of the previous reviews, the evidence base has grown and approaches to meta-analysis of diagnostic tests have evolved. This updated systematic review was commissioned by WHO to guide policy recommendations on serological tests for TB, with a special focus on the relevance of these assays in low- and middle-income countries. The objective of this review is to synthesize new evidence since 2006 in order to address the following question: what is the diagnostic accuracy of commercial serological tests for active TB (pulmonary and extrapulmonary TB) in adults and children, with and without HIV infection? Specifically, we were interested in evaluating the use of a serological assay as a replacement test for, or an additional test after, smear microscopy.

Methods

We followed methods for conducting and reporting systematic reviews and meta-analyses recommended by the Cochrane Collaboration Diagnostic Test Accuracy Working Group and the PRISMA statement (Text S1), including the preparation of a protocol and analysis plan (Text S2) [16]–[18].

Selection Criteria and Definitions

Types of studies

Diagnostic studies (with any study design) were included that evaluated serological tests for active TB (pulmonary and extrapulmonary TB) in patients who provided sera before or within 14 d of starting antituberculous treatment.

Participants

The participants constituted adults and children, with and without HIV infection, with suspected or confirmed active TB, from all clinical settings (clinic or hospital). The protocol for the current review included studies with at least ten TB cases. Studies could be performed in any country regardless of TB incidence or income status.

Index test

The index test was any commercial serological test for the diagnosis of active TB.

Comparator tests

There was either no test or smear microscopy used for comparison.

Target conditions

The target conditions were pulmonary and extrapulmonary TB.

Reference standards

Pulmonary TB required positivity on mycobacterial culture. (The previous review accepted positivity on either culture or smear microscopy as the reference standard [12].) Extrapulmonary TB required positivity on at least one of the following tests: culture, smear, or histopathological examination.

Outcomes

The outcomes were sensitivity and specificity. Sensitivity refers to the proportion of patients with a positive serological test result among patients with TB confirmed by the reference standard. Specificity refers to the proportion of participants with a negative serological test result among participants without TB according to the reference standard. To estimate specificity, we selected only one non-TB group if a study had more than one such group. The preferred non-TB participants were those in whom active TB was initially suspected but later ruled out (“other respiratory disease” or “mixed disease” groups), and who were from the same population as TB patients.

Extrapulmonary TB

Extrapulmonary TB was classified as lymph node, pleural, meningeal and/or central nervous system, bone and/or joint, genitourinary, abdominal, skin, other sites, disseminated, and multiple sites (extrapulmonary TB cases from different sites are combined to obtain at least ten extrapulmonary TB cases).

Country income status

Country income status was classified according to the World Bank List of Economies [19].

Exclusion criteria

The following studies were excluded: (1) studies published before 1990; (2) animal studies; (3) conference abstracts and proceedings; (4) studies on the detection of latent TB infection; (5) studies on nontuberculous mycobacterial infection; (6) studies that used non-immunological methods for detection of antibodies; and (7) basic science literature that focused on detection/cloning of new antigens or their immunological properties (i.e., early pre-clinical studies).

Search Methods

We updated the database searches (MEDLINE [1 May 2006 to 29 June 2010], BIOSIS [1 January 2005 to 10 February 2010], EMBASE [1 October 2005 to 10 February 2010], and Web of Science [1 January 2005 to 10 February 2010]) that were carried out in previous systematic reviews (MEDLINE [1 January 1990 to 30 May 2006], BIOSIS [1 January 1990 to 6 December 2005], EMBASE [1 January 1990 to 11 October 2005], and Web of Science [1 January 1990 to 6 December 2005]) for relevant studies that reported data on commercial serological tests for active TB. The original search was limited to English, and the updated search was performed without a language restriction. The search field tags used in database searching were MeSH terms (mh), title/abstract words (tiab), and title (ti). The terms used included: tuberculosis[mh] OR mycobacterium tuberculosis[mh], “sensitivity and specificity”[mh] OR diagnostic*[tiab] OR predictive value of tests[mh] OR immunologic tests[mh] OR immunochemistry[mh] OR serology[ti] OR serological[ti] OR serodiagnosis[tiab] OR serodiagnostic[tiab] OR immunodiagnosis[tiab] OR immunodiagnostic[tiab] OR antibody[tiab] OR antibodies[tiab] OR elisa[tiab] OR immunosorbent[tiab] OR (western[tiab] AND blot*[tiab]) OR immunoassay[tiab] OR “humoral immune” OR “humoral immunity” OR “humoral antibody” OR “immune based” OR “antibody detection” (Text S3).

In addition to database searches, we also searched reference lists of eligible papers and related reviews, and contacted authors and researchers in the field to identify additional potentially relevant published studies. For lack of time, we did not specifically seek to identify unpublished studies.

Study Selection

Initially, two reviewers (KRS and LLF) independently screened the accumulated citations for relevance and then independently reviewed full-text articles using prespecified eligibility criteria. Disagreements about study selection were resolved by discussion.

Data Extraction

A data extraction form was created and pilot-tested with a subset of eligible studies and then finalized. Two reviewers independently extracted data from included studies with the standardized form on the following characteristics: study design; age group (children <15 y of age); HIV status; case country of residence; sputum smear status (pulmonary TB); site of TB (extrapulmonary TB); assay type (e.g., ELISA, immunochromatographic test); antibody class detected (IgG, IgM, and IgA); serological test name; antigen composition; condition of the specimen (fresh or frozen); and sensitivity and specificity (data were extracted as true positives, false positives, false negatives, and true negatives). In some cases, study investigators evaluated more than one diagnostic test with the same set of participants. In these situations, we extracted data for each test and considered each dataset to be an independent study. For example, Anderson et al. [20] contributed three studies evaluating three serological tests: (1) InBios Active TbDetect IgG ELISA (InBios International); (2) IBL M. tuberculosis IgG ELISA (IBL-Hamburg), and (3) anda-TB IgG (Anda Biologicals) [20]. The agreement between reviewers on data extraction for sensitivity and specificity was 100%. Other differences between the reviewers (these differences mainly concerned the methodological quality assessment) were resolved by discussion. When necessary, we contacted authors of papers identified through the updated literature search for additional information.

While extracting data, we looked for studies that considered the added value of serological tests to determine if they contributed to active TB diagnosis beyond that ascertained by conventional tests such as symptoms, sputum smears, and chest radiographs. In particular, we looked for studies comparing microscopy with microscopy plus serology or studies that performed multivariable analysis. Since we did not identify any studies of this type, we considered studies in smear-negative patients to provide indirect evidence of the use of serology as an add-on test to microscopy. In addition, we looked for information on patient-important outcomes. Patient-important outcomes for this review could include an increased number of TB patients detected, decreased time to starting treatment, increased number of patients starting TB treatment, decreased number of false-positive TB patients treated, and decreased number of patients lost because of a reduced number of visits. Finally, we looked for information on the values and preferences of patients associated with these tests.

Assessment of the Methodological Quality of Individual Studies

Two reviewers (KRS and LLF) independently assessed study quality using the core set of 11 items from Quality Assessment of Diagnostic Accuracy Studies (QUADAS), a validated tool to evaluate the presence of bias and variation in diagnostic accuracy studies [21]. As recommended, we scored each item as “yes,” “no,” or “unclear.” We considered representative patient spectrum (i.e., was the spectrum of patients representative of the patients who will receive the test in practice?) to be persons suspected of having active TB who were consecutively or randomly enrolled. For pulmonary TB, a score of “yes” for representative spectrum also required that patients were evaluated in an outpatient setting. For all studies, we scored the following six items as “yes”: acceptable reference standard (the reference standard was a criterion for inclusion); acceptable delay between serological test and reference standard; partial verification avoided; incorporation avoided; reference standard results blinded (as culture result was considered to be entirely objective in interpretation); and relevant clinical information. “Differential verification avoided” was scored as “yes” if all participants suspected of having TB were evaluated with the same reference standard or if participants without TB were reported to be asymptomatic and healthy. “Index test (serological test) result blinded” (to reference standard result) was scored as “yes” if this was explicitly stated in the paper. “Uninterpretable results reported” was scored as “yes” if uninterpretable results were described or there was a statement about the absence of uninterpretable results. “Withdrawals explained” was scored as “yes” if a flow diagram or statement was included making it clear what happened to all participants in the study. Conflicts of interest are known to be a concern in diagnostic studies [22]. Therefore, we evaluated the involvement of test manufacturers. Finally, we grouped studies according to the type of serological test or site of TB (in the case of extrapulmonary TB) and assessed study quality separately for each subgroup.

GRADE Quality of Evidence

We used the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach, a transparent and systematic process for making judgments about quality of the evidence [23]. GRADE specifies four categories for quality: high, moderate, low, and very low. These categories are then applied to the body of evidence for each outcome, rather than to individual studies. In the GRADE approach, the categories reflect the extent of confidence that an estimate of effect is correct [24] . Quality begins with a consideration of study design (e.g., randomized controlled trials and cross-sectional studies in patients with diagnostic uncertainty are considered high quality) and may be compromised by five factors: limitations (risk of bias assessed by QUADAS, such as absence of consecutive or random selection of participants and lack of blinding of test results), indirectness (lack of generalizability and use of test results as surrogates for patient-important outcomes), inconsistency (unexplained heterogeneity), imprecision (wide confidence intervals for estimates of test accuracy), and risk of publication bias [23].

Data Analysis

Descriptive analyses were performed using SPSS (version 14.0.1.366). For each study, the sensitivity and specificity of the serological test along with the 95% CIs were calculated, and forest plots were generated to display sensitivity and specificity estimates using Review Manager 5.0 (The Nordic Cochrane Center). Heterogeneity was assessed visually using forest plots (Review Manager 5.0).

Selection of Subgroups for Meta-Analysis

We recognized that studies were heterogeneous in many respects, particularly concerning the serological test used, antibody class detected, sputum smear status (pulmonary TB), and site of extrapulmonary TB. Therefore, in order to address heterogeneity and combine study results, subgroups of “comparable” tests and extrapulmonary sites were prespecified. When possible, studies were stratified by smear and HIV status. For meta-analysis, at least four studies were required to be available for inclusion in a subgroup, in order to strengthen results and reduce the possibility of finding a significant result by chance. This classification resulted in seven subgroups for meta-analysis (four pulmonary and three extrapulmonary TB subgroups). As noted above, in some cases, study investigators evaluated more than one diagnostic test with the same participants; therefore, some meta-analyses included the same individuals multiple times.

To summarize test performance within each subgroup, we carried out bivariate meta-analyses that jointly modeled sensitivity and specificity. These models weighted studies according to the sampling variability within studies as well as the unexplained heterogeneity between studies using a random effects approach [25]. Subgroups were considered homogeneous with respect to a number of observed variables. However, within a subgroup, it is likely that the heterogeneity between studies could be explained by measurable but unobserved quantities, e.g., the positivity cutoff, that we could not address. Therefore, for the pooled results of studies in the meta-analysis, we did not attempt to quantify unobserved heterogeneity using statistics such as I 2 or chi-squared [18]. The model was estimated using a Bayesian approach with nonsubjective prior distributions and implemented using WinBUGS (version 1.4.1) [26]. We used Wilson's method for estimating the credible interval as this method performs well even when the probability or the sample size is small [27]. Finally, a hierarchical summary receiver operating characteristic (HSROC) curve was plotted for selected meta-analyses. The HSROC curve plots sensitivity versus specificity and provides information on the overall performance of a test across different thresholds. The closer the curve is to the upper left-hand corner of the plot (sensitivity and specificity are both 100%), the better the performance of the test [28]. The plots were made using R (version 2.6.1) [29].

Results

Pulmonary TB

Results of the search

Initially, 4,256 citations were identified (Figure 1). After screening titles and abstracts, 160 potentially relevant full-text papers were retrieved. Thirty-one papers (20 from the original review and 11 from the update) describing 67 studies and involving 5,147 participants (sample size = 8,318) were included in the review [20],[30]–[59]. A list of excluded articles with their reasons for exclusion is provided in Text S4.

Figure 1

Flow of studies in the review of commercial serological tests for the diagnosis of active tuberculosis.

Included studies

Of 67 total studies, six (9%) were reported in languages other than English: Spanish (2), Turkish (1), Chinese (1), Bosnian (1), and Russian (1). Thirty-two (48%) studies were conducted in low- and middle-income countries. No studies were randomized controlled trials; 55% of studies used a cross-sectional study design, and 45% of studies used a case-control study design. All but one study reported recruiting TB and non-TB patients from the same underlying population [56]. One study involved HIV-infected individuals, and no studies involved children. Thirty-one (46%) studies involved smear-positive patients, 28 (42%) studies involved smear-negative patients, and eight (12%) studies involved patients with unspecified smear status. Fifty-four (81%) studies used ELISA, 12 (18%) studies used an immunochromatographic assay, and one study used a kaolin precipitation test. The majority of studies detected only IgG antibody (44 studies) and used frozen serum (51 studies). The median number of TB patients included in each study was 41 (interquartile range 33 to 54). Eighteen serological tests were included; anda-TB (IgG, IgA, and IgM) was the test most frequently evaluated (16/67 [24%]) (Table 1). The antigen composition for five (28%) of the total 18 tests was considered proprietary information. Of the tests with known antigens, all had unique antigenic compositions except for anda-TB and Hexagon, which both contained antigen A60.

Table 1

Commercial serological tests included in this systematic review for pulmonary TB.

Name of Test (Number of Studies)	Name of Manufacturera, City, Country	Antigenic Composition	Antibodies Detected	Laboratory Technique
Detect-TB (2)	Adaltis—Advanced Laboratory Diagnostics Systems, Rome, Italy	Proprietary, a cocktail of three M. tuberculosis recombinant proteins and two synthetic peptides from five different proteins	IgG	ELISA
anda-TB ELISAb (16)	Anda Biologicals, Strasbourg, France	A60	IgG, IgM, IgA, IgG plus M	ELISA
Tuberculosis Specific Antigen (1)	Chengdu Pharmaceutical, Chengdu, China	Proprietary	Unknown	ELISA
Kaolin Agglutination Test (1)	Hitech Laboratories, Bombay, India	Tuberculophosphatide	IgG	Agglutination test
Hexagon TB (1)	Human Gesellschaft für Biochemica und Diagnostica, Wiesbaden, Germany	A60	IgG, IgA, IgM	ICT
Mycobacterium tuberculosis IgG (1)	IBL, Hamburg, Germany	18 kDa, 36 kDa, and 40 kDa recombinant antigens	IgG, IgA, IgM	ELISA
ICT TB (3)	ICT Diagnostics, Sydney, Australia	38 kDa and four proprietary antigens; all five antigens are recombinant	IgG	ICT
ActiveTBDetect (1)	InBios International, Seattle, US	Mtb81, Mtb8, Mtb48, DPEP (MPT32), 38 kDa protein, and two additional proprietary antigens	IgG	ELISA
SEVA (1)	Jamnalal Bajaj Tropical Disease Research Centre, Mahatma Gandhi Institute of Medical Sciences, Maharashtra, India	31 kDa, native glycoprotein antigen from culture filtrate of MTB H37Rv	IgG	ELISA
TB Enzyme Immunoassay (5)	Kreatech, Amsterdam, The Netherlands	Kp-90 antigenic compound: LAM, 10 kDa, 16 kDa, 21 kDa, 30 kDa, 34 kDa, 65 kDa, and 95 kDa	IgG, IgA	ELISA
Determiner TB Glycolipid Assay (6)	Kyowa Medex, Tokyo, Japan	Contains trehalose 6,6′-dimycolate, trehalose monomycolate, diacyltrehalose, phenolic glycolipid, 2,3,6,6-tetraacyl-trehalose-2-sulfate, and 2,3,6-triacyl-trehalose	IgG	ELISA
Assure TB (2)	MedTek, Paranque City, Philippines; and Genelabs Diagnostics, Singapore	Proprietary, two recombinant antigens	IgG	ICT
MycoDot (3)	Mossman Associates, Milford, Massachusetts, US	LAM	IgG	ICT
Pathozyme Mycoc (11)	Omega Diagnostics, Alva, Scotland	LAM, recombinant 38 kDa	IgG, IgA, IgM	ELISA
Pathozyme TB Complex (2)	Omega Diagnostics, Alva, Scotland	Recombinant 38 kDa	IgG	ELISA
Pathozyme TB Complex Plusd (9)	Omega Diagnostics, Alva, Scotland	Recombinant 38 kDa and 16 kDa	IgG	ELISA
SDHO MTB (1)	SDHO Laboratories, Saint-Sauveur des Monts, Canada	Proprietary	IgG	ICT
Serocheck-MTB (1)	Zephyr Biomedicals, Verna, India	Recombinant 14 kDa, 38 kDa, 16 kDa, and 6 kDa	Unknown	ICT

ICT, immunochromatographic test; LAM, lipoarabinomannan.

Some manufacturers may no longer provide the serological tests listed above. It is also possible that the same test may be marketed under various names by different companies.

anda-TB: IgG (13 studies), IgM (one study), IgA (two studies).

Pathozyme Myco: IgG (three studies), IgM (two studies), IgA (two studies), IgG plus IgM (one study), IgG plus IgA (one study), IgM plus IgA (one study), IgG plus IgM plus IgA (one study).

Pathozyme TB Complex Plus as a single test (three studies) or in combination with Pathozyme Myco G (one study), Pathozyme Myco M (one study), Pathozyme Myco A (one study), Pathozyme Myco G and A (one study), Pathozyme Myco M and A (one study), Pathozyme Myco G, M, and A (one study).

One study directly compared a serological test to sputum microscopy. No studies evaluated the incremental value of adding a serological test after smear microscopy. However, as noted, 28 (42%) studies involved smear-negative patients. These studies were considered a proxy for a diagnostic strategy using serological tests in addition to microscopy. No studies reported on patient-important outcomes or patient values and preferences concerning these tests. Characteristics of included studies are described in Table S2.

Methodological quality, all included studies

As assessed with QUADAS, studies had very serious limitations. Of the total 67 studies only 19 (28%) were considered to include a representative patient population (we scored this item as “yes” when ambulatory patients suspected of having active TB were randomly or consecutively selected), and 34 (51%) studies reported blinding of the serological test result. The majority (60%) of studies reported industry involvement, mainly the donation of test kits (Figure 2). We downgraded two points for limitations in the GRADE Evidence Profile (Table 3).

Figure 2

Methodological quality graph, all studies, pulmonary TB.

Review authors' judgments about each methodological quality item.

Table 3

GRADE evidence profile: should commercial serological tests be used as a replacement test for conventional tests such as smear microscopy in patients of any age suspected of having pulmonary tuberculosis?

Outcome	Number of Studies (Participants)	Study Design	Limitations	Indirectness	Inconsistency	Imprecision	Publication Bias	Final Quality	Effect per 1,000a	Importanceb
True Positives	67 (5,147)	Cross-sectional and case-control	Very seriousc (−2)	No serious indirectnessd	Very seriouse (−2)	Seriousf	Likelyg	Very low ⊕◯◯◯	Prevalence 10%: 64; prevalence 30%: 192	Critical
True Negatives	67 (5,147)	Cross-sectional and case-control	Very seriousc (−2)	No serious indirectnessd	Very seriouse (−2)	Seriousf	Likelyg	Very low ⊕◯◯◯	Prevalence 10%: 819; prevalence 30%: 637	Critical
False Positives	67 (5,147)	Cross-sectional and case-control	Very seriousc (−2)	No serious indirectnessd	Very seriouse (−2)	Seriousf	Likelyg	Very low ⊕◯◯◯	Prevalence 10%: 81; prevalence 30%: 63	Critical
False Negatives	67 (5,147)	Cross-sectional and case-control	Very seriousc (−2)	No serious indirectnessd	Very seriouse (−2)	Seriousf	Likelyg	Very low ⊕◯◯◯	Prevalence 10%: 36; prevalence 30%: 108	Critical

Based on sample size = 8,318, sensitivity median = 64%, specificity median = 91%. The quality of evidence was rated as high (no points subtracted), moderate (one point subtracted), low (two points subtracted), or very low (>2 points subtracted) based on five factors: study limitations, indirectness of evidence, inconsistency in results across studies, imprecision in summary estimates, and likelihood of publication bias. For each outcome, the quality of evidence started at high when there were randomized controlled trials or high-quality observational studies (cross-sectional or cohort studies enrolling patients with diagnostic uncertainty) and at moderate when these types of studies were absent. No points were subtracted when there were negligible issues identified; one point was subtracted when there was a serious issue identified; two points were subtracted when there was a very serious issue identified in any of the criteria used to judge the quality of evidence. Points subtracted are in parentheses. Publication bias was rated as “not likely,” “likely,” or “very likely” [23].

What do these results mean given 10% or 30% disease prevalence among individuals being screened for TB?

Outcomes were ranked by their relative importance as critical, important, or of limited importance. Ranking helped to focus attention on those outcomes that were considered most important.

The majority of studies lacked a representative patient population and were not blinded.

Although diagnostic accuracy is considered a surrogate for patient-important outcomes, we did not downgrade.

There was considerable heterogeneity in study results.

We did not pool accuracy estimates. The 95% CIs were wide for many individual studies. We did not downgrade as there were a large number of studies and we had already taken off two points for inconsistency.

Data included in the review did not allow for formal assessment of publication bias using methods such as funnel plots or regression tests. Therefore, publication bias cannot be ruled out. It is prudent to assume some degree of publication bias as studies showing poor performance of serological tests were probably less likely to be published. No points were deducted.

Test performance, all studies

As seen from the forest plots in Figure 3, studies displayed considerable heterogeneity, with sensitivity values ranging from 0% to 100% and specificity values, from 31% to 100%. We did not pool accuracy estimates because of the heterogeneity among studies. Similarly, when restricted to studies conducted in low- and middle-income countries, sensitivity (16% to 91%) and specificity (31% to 100%) were highly variable (data not shown).

Figure 3

Forest plots of sensitivity and specificity, all studies, pulmonary TB.

Specificity data for Mizusawa et al. [49] were not reported. FN, false negative; FP, false positive; TN, true negative; TP, true positive. 95% CIs are included in brackets. On the right, the sensitivity and specificity estimates for individual studies are indicated by blue squares and the 95% CIs are indicated by black horizontal lines.

Methodological quality, studies in smear-negative patients

As assessed with QUADAS, only 14 (50%) of the total 28 studies were considered to include a representative patient population. A majority (75%) of studies reported blinding of the serological test result (Figure 4). We downgraded one point for limitations in the GRADE Evidence Profile (Table 4).

Table 4

GRADE evidence profile: should commercial serological tests be used as an “add on” test to smear microscopy in patients of any age suspected of having pulmonary TB?

Outcome	Number of Studies (Participants)	Study Design	Limitations	Indirectness	Inconsistency	Imprecision	Publication Bias	Final Quality	Effect per 1,000a	Importanceb
True Positives	28 (1,961)	Mainly cross-sectional	Seriousc (−1)	Seriousd (−1)	Very seriouse (−2)	Seriousf	Likelyg	Very Low ⊕◯◯◯	Prevalence 10%: 61	Critical
True Negatives	28 (1,961)	Mainly cross-sectional	Seriousc (−1)	Seriousd (−1)	Very seriouse (−2)	Seriousf	Likelyg	Very Low ⊕◯◯◯	Prevalence 10%: 828	Critical
False Positives	28 (1,961)	Mainly cross-sectional	Seriousc (−1)	Seriousd (−1)	Very seriouse (−2)	Seriousf	Likelyg	Very Low ⊕◯◯◯	Prevalence 10%: 72	Critical
False Negatives	28 (1,961)	Mainly cross-sectional	Seriousc (−1)	Seriousd (−1)	Very seriouse (−2)	Seriousf	Likelyg	Very Low ⊕◯◯◯	Prevalence 10%: 39	Critical

This table includes studies conducted in smear-negative patients as a proxy for a diagnostic strategy using serological tests in addition to smear microscopy. Based on sample size = 3,433, sensitivity median = 61% and specificity median = 92%. The quality of evidence was rated as high (no points subtracted), moderate (one point subtracted), low (two points subtracted), or very low (>2 points subtracted) based on five factors: study limitations, indirectness of evidence, inconsistency in results across studies, imprecision in summary estimates, and likelihood of publication bias. For each outcome, the quality of evidence started at high when there were randomized controlled trials or high-quality observational studies (cross-sectional or cohort studies enrolling patients with diagnostic uncertainty) and at moderate when these types of studies were absent. No points were subtracted when there were negligible issues identified; one point was subtracted when there was a serious issue identified; two points were subtracted when there was a very serious issue identified in any of the criteria used to judge the quality of evidence. Points subtracted are in parentheses. Publication bias was rated as “not likely,” “likely,” or “very likely” [23].

What do these results mean given 10% disease prevalence among individuals being screened for TB?

Outcomes were ranked by their relative importance as critical, important, or of limited importance. Ranking helped to focus attention on those outcomes that were considered most important.

Only 14/28 (50%) studies were considered to include a representative patient population; 75% of studies reported blinding of the serological test result.

We downgraded for indirectness because these studies were used as a proxy for a diagnostic strategy using serological tests in addition to smear microscopy.

There was considerable heterogeneity in study results.

We did not pool accuracy estimates. The 95% CIs were wide for many individual studies. We did not downgrade as there were a large number of studies and we had already taken off two points for inconsistency.

Data included in the review did not allow for formal assessment of publication bias using methods such as funnel plots or regression tests. Therefore, publication bias cannot be ruled out. It is prudent to assume some degree of publication bias as studies showing poor performance of serological tests were probably less likely to be published. No points were deducted.

Test performance, studies in smear-negative patients

For individual studies involving smear-negative patients, sensitivity values ranged from 29% to 77%, and specificity values, from 77% to 100% (Figure 5). We did not pool accuracy estimates because of the considerable heterogeneity among studies. As noted above, studies involving smear-negative patients were considered to provide indirect evidence of a diagnostic strategy using microscopy plus serology. Hence, as an add-on test, serological tests provided inconsistent sensitivity and specificity.

Figure 4

Methodological quality summary, pulmonary TB, studies of smear-negative patients.

Review authors' judgments about each methodological quality item.

Analysis by subgroups

According to our prespecified analysis plan, there was a sufficient number of studies to perform a meta-analysis for only one serological test, anda-TB IgG, with results stratified by smear status (seven studies of smear-positive and four studies of smear-negative patients). In studies of smear-positive patients, one study was conducted in a low-income country [43]. In studies of smear-negative patients, no studies were conducted in a low- or middle-income country.

Methodological quality of studies evaluating anda-TB IgG

In studies of smear-positive patients, no studies were considered to have a representative patient population (participants were known TB cases rather than suspected cases, were inpatients, and/or were enrolled by convenience), and only two studies reported blinding of the serological test result [43],[52] (Figure S1). In studies of smear-negative patients, no studies were considered to have a representative patient population (participant selection was by convenience or not reported), and only one study reported blinding of the serological test result [52] (Figure S2).

Meta-analysis

In studies involving smear-positive patients, anda-TB IgG yielded a pooled sensitivity of 76% (95% CI 63–87) and a pooled specificity of 92% (95% CI 74–98). In studies involving smear-negative patients, the pooled sensitivity of anda-TB IgG decreased to 59% (95% CI 10–96); the 95% CI was very wide, reflecting the imprecision of the sensitivity estimate. Pooled specificity was 91% (95% CI 79–96) (Table 2). The HSROC curves show the decreased performance of the test in smear-negative patients compared with smear-positive patients (Figure 6).

Table 2

Bivariate meta-analyses: pooled sensitivity and specificity estimates by subgroup.

Subgroup	Number of Studies	Number of Participants (Sample Size)	Sensitivity	Specificity
Pulmonary TB
anda-TB IgG, smear-positive	7	870 (870)	76 (63–87)	92 (74–98)
anda-TB IgG, smear-negative	4	700 (700)	59 (10–96)	91 (79–96)
ELISA	54	3,696 (6,434)	60 (6–65)	98 (96–99)
Immunochromatographic Testa	12	1,231 (1,512)	53 (42–64)	98 (94–99)
Extrapulmonary TB
Lymph node TB	6	640 (922)	64 (28–92)	90 (76–97)
Pleural TB	5	322 (572)	46 (29–63)	87 (51–99)
anda-TB IgG	10	1,055 (1,637)	81 (49–97)	85 (77–92)

Sensitivity and specificity estimates given as posterior means (percent) with 95% credible intervals in parentheses.

Serological tests included: ICT TB (three studies), Assure TB (two studies), MycoDot (three studies), SDHO (two studies), Hexagon (one study), Serocheck-MTB (one study).

Figure 5

Forest plots of sensitivity and specificity, pulmonary TB, studies of smear-negative patients.

FN, false negative; FP, false positive; TN, true negative; TP, true positive. 95% CIs are included in the brackets. On the right, the sensitivity and specificity estimates for individual studies are indicated by blue squares, and the 95% CIs are indicated by black horizontal lines.

We explored whether test technique had an impact on accuracy. Compared with ELISA (pooled sensitivity 60% [95% CI 6–65]; pooled specificity 98% [95% CI 96–99]), immunochromatographic assays yielded lower pooled sensitivity (53%, 95% CI 42–64) and comparable pooled specificity (98%, 95% CI 94–99) (Table 2; Figure 7). The probability that the pooled sensitivity of ELISA tests exceeds that of immunochromatographic assays was estimated at 0.88.

Figure 6

Summary HSROC plots of sensitivity and specificity for anda-TB IgG in smear-positive and smear-negative pulmonary TB patients.

Smear-positive (line A; open circles) and smear-negative (line B; gray circles) pulmonary TB patients. The width of the circles is proportional to the number of patients in each study. The red squares are the summary values for sensitivity and specificity.

TB in HIV-infected patients

The only study identified involving HIV-infected patients compared the performance of the SDHO MTB test (SDHO Laboratories) head-to-head with smear microscopy in 55 individuals suspected of having pulmonary TB residing in the Central African Republic [44]. TB was confirmed by culture. Compared with smear microscopy (sensitivity 68% [95% CI 49–83]), SDHO yielded a sensitivity of only 16% (95% CI 5–34). Specificity of SDHO was 90% (95% CI 74–98), lower than the specificity of smear microscopy (100% [95% CI 89–100]).

Extrapulmonary TB

Twelve papers (nine from the original review and three from the update) describing 25 studies and involving 1,809 participants (sample size = 3,776) were included in the review (Figure 1) [45],[48],[60]–[69].

Of 25 total studies, ten (40%) studies were conducted in low- and middle-income countries. All papers were written in English. Only one study involved HIV-infected individuals. The vast majority (88%) of studies involved adults. In two studies (reported from one paper), 13 of 35 extrapulmonary TB cases occurred in children; however, data were not provided separately for the children [67]. One study specified 13 y as the minimum age for eligibility; however, the age range for the enrolled participants was not reported [64]. Of 25 total studies, serological tests were evaluated for diagnosis of the following forms of extrapulmonary TB: lymph node, six studies; pleural, five studies; multiple sites, five studies (see Table S3 for a list of sites involved); genitourinary, two studies; disseminated, four studies; and meningeal, one study. In two studies, the site of extrapulmonary involvement was not reported. Six distinct serological tests were evaluated; 17 (68%) of the total 25 studies used anda-TB (IgG, ten studies; IgM, five studies; IgA, one study; IgM plus IgA, one study). ELISA was used in 21 (84%) studies, and immunochromatographic assays, in four studies. The majority (72%) of studies detected IgG antibodies. The condition of the specimen was frozen in six (24%) studies and not reported in 19 studies. The median number of TB patients included in each study was 35 (interquartile range 30 to 56). No studies reported on patient-important outcomes or patient values and preferences concerning these tests. Characteristics of included studies are described in Table S3.

Of the 25 total studies, only one (4%) study was considered to include a representative patient population, and only four (16%) studies reported blinding of the serological test result (Figure 8).

Figure 7

Summary HSROC plots of sensitivity and specificity by assay technique.

ELISA (line A; open circles) and immunochromatographic test (line B; gray circles). The width of the circles is proportional to the number of patients in each study. The red squares are the summary values for sensitivity and specificity.

As seen from the forest plots in Figure 9, studies displayed considerable heterogeneity, with sensitivity values ranging from 0% to 100%, and specificity values, from 59% to 100%. We did not pool accuracy estimates because of the heterogeneity among studies.

Figure 8

Methodological quality summary, all studies, extrapulmonary TB.

Review authors' judgments about each methodological quality item.

Forest plots of sensitivity and specificity, all studies, extrapulmonary TB.

There was a sufficient number of studies to perform a meta-analysis for only one serological test, anda-TB IgG (ten studies). Eight studies were conducted in high-income countries, one study was conducted in India [61], and one study, in Turkey [66].

As assessed with QUADAS, studies had very serious limitations. No studies were considered to have a representative population (no studies reported selecting participants in a consecutive or random manner), and no studies reported interpreting the results of the serological test result without knowledge of the results of the reference standard.

We determined pooled sensitivity and specificity estimates for studies evaluating any serological test for the diagnosis of lymph node or pleural TB and for studies using anda-TB IgG (all extrapulmonary sites). For lymph node TB, pooled sensitivity was 64% (95% CI 28–92) and pooled specificity was 90% (95% CI 76–97). For pleural TB, pooled sensitivity was 46% (95% CI 29–63) and pooled specificity was 87% (95% CI 51–99). For anda-TB IgG, pooled sensitivity was 81% (95% CI 49–97) and pooled specificity was 85% (95% CI 77–92) (Table 2). For all three meta-analyses, the 95% CIs for sensitivity were wide, reflecting the imprecision of the estimates.

The only study identified involving HIV-infected patients evaluated the performance of the MycoDot test (Mossman Associates) in a cross-sectional study of patients suspected of having TB in Thailand [68]. In all, 142 HIV-infected (mean CD4 cell count = 188 cells/mm3 [range 7 to 632]) and 144 HIV-uninfected patients with newly diagnosed TB participated in the study, of whom 50 patients (40 HIV-infected and ten HIV-uninfected patients) had a diagnosis of lymph node TB established by culture or histopathological examination. Compared with the sensitivity of MycoDot in HIV-uninfected TB patients (80%, 95% CI 44–98), the sensitivity of the test in HIV-infected TB patients was considerably lower (33%, 95% CI 19–39). The specificity in both groups was 97% (95% CI 93–99) (Table S3).

GRADE Evidence Profiles

For the pulmonary TB studies, the quality of the body of evidence supporting TB serology's estimates of sensitivity and specificity was graded as “very low” (Tables 3 and 4). Thus, regardless of the width of the 95% CIs (which reflects the size of studies and the standard deviation of their measured results), we have very low confidence in the estimates obtained from pooling studies in the meta-analysis [23]. For the extrapulmonary TB studies, the final quality grades were also very low (data not shown).

Discussion

This updated systematic review assessing the diagnostic accuracy of commercial serological tests for pulmonary and extrapulmonary TB summarizes the current literature and includes 14 new papers (approximately 30% of the included papers) identified since our previous reviews [12],[13]. Unlike the earlier reviews, in the update, we performed a meta-analysis using a bivariate random effects model to account for the variability in test accuracy across studies. Findings from the current review are similar to those of the previous review: studies of current serological tests show that these tests provide inaccurate and imprecise estimates of sensitivity and specificity.

In the earlier systematic reviews, we recommended the use of guidelines such as STARD (Standards for the Reporting of Diagnostic Accuracy Studies) [70] and QUADAS [21] to improve methodological study quality. In the current review, within-study quality continues to be a concern. For example, in the pulmonary TB group there were 16 new studies. Six of these studies (three papers) were published subsequent to the previous reviews [20],[37],[49]. Four of the six studies selected participants by convenience or did not report the manner of selection (selection bias), and no studies reported that the serological test result was interpreted without knowledge of the reference standard. Selection bias and absence of blinding are features of study design that have been associated with exaggerated accuracy estimates [71],[72].

A substantial contribution of the current review is the use of the GRADE approach. This framework enabled us to synthesize data on the quality of the body of evidence in a way that was not possible for the previous systematic reviews [12],[13] because GRADE was not well developed for diagnostic studies at that time. The very low quality of evidence for the studies evaluating anda-TB IgG in smear-negative patients decreases our confidence in the pooled sensitivity and specificity estimates. In this subgroup, applying the GRADE approach, quality was compromised by three factors: (1) risk of bias: no studies recruited participants in a random or consecutive manner, and only one study reported blinded interpretation of the serological test result; (2) indirectness: no studies were conducted in low- or middle-income countries, limiting generalizability to these settings; and (3) imprecision. If the pooled estimates of test accuracy had been derived from high-quality studies, then the serological test might have been shown to have some clinical utility for contributing to diagnostic algorithms for smear-negative TB, especially since the tests are relatively inexpensive, rapid, and easy to perform. However, the very low quality of the evidence implies that the serological test cannot be recommended.

Strengths and Limitations

Strengths of our review include the use of a standard protocol and comprehensive search strategy, two independent reviewers at all stages of the review process, the assessment of methodological quality of individual studies with the QUADAS tool, and the use of the GRADE approach. Heterogeneity is to be expected in results of diagnostic test accuracy studies [73]. Therefore, we prespecified subgroups to limit heterogeneity and, as noted above, used a bivariate random effects model. Our review also had limitations, notably, the majority of studies were not considered to include patients with a representative spectrum of disease severity. Differing criteria for patient selection and greater duration and severity of illness of the study populations may have introduced variability in findings among studies. In addition, the majority of studies were not performed in a blinded manner, or blinding was not explicitly stated. Also, the meta-analysis was limited by the small number of studies for a particular serological test. anda-TB IgG was the only test with enough studies for meta-analysis. Clearly, having more studies would have allowed us to examine observed, study-level covariates that could be sources of heterogeneity. An additional limitation was that, in some cases, we assumed that multiple results carried out on the same sample were independent. By doing so, our meta-analysis model may have underestimated heterogeneity and overestimated precision of the pooled sensitivity and specificity estimates by including a larger number of participants. Subgroup analyses in a meta-analysis, like subgroup analyses in a clinical trial, are vulnerable to bias; therefore, the findings of this meta-analysis should be interpreted with caution [74]. Although we tried to address language bias by performing the updated literature search in all languages, the original literature search was limited to studies published in English, and language bias remains a possibility. Finally, our review did not allow for formal assessment of publication bias using methods such as funnel plots or regression tests because such techniques have not been adequately evaluated for diagnostic data [75]. Therefore, publication bias cannot be ruled out. However, it is prudent to assume some degree of publication bias, as studies showing poor performance of serological tests may have been less likely to be published, especially because several studies were industry supported.

This systematic review focused on test accuracy (i.e., sensitivity and specificity). Although, we looked for information on patient-important outcomes (meaning a serological test used in a given situation results in a clinically relevant improvement in patient care and/or outcomes), we did not find this information in the literature reviewed. We did not identify studies with the specific aim of detecting the value of serology over and above conventional tests such as smears. However, the WHO Special Programme for Research and Training in Tropical Diseases report on rapid serological tests for TB mentioned above did evaluate the added value of smear plus serology and reported a gain equivalent to the detection of 57% of the smear-negative, culture-positive TB cases. There was, however, a corresponding unacceptable decrease in specificity to 58% [15].

In conclusion, published data on commercial serological tests produce inconsistent and imprecise estimates of sensitivity and specificity, and the quality of the body of evidence on these tests remains disappointing. This systematic review included evaluations of only commercially available antibody-based detection tests. Considerable research is underway on new approaches to the serological diagnosis of TB. These approaches include the use of newly identified selected purified recombinant antigens and antigen combinations [14]. Recent studies from a number of laboratories have reported several new potential candidate antigens that may be expected to lead to improved antibody detection tests for TB in the future. These conclusions should be reconsidered if, in the future, methodologically adequate research evaluating serological tests becomes available.

The findings from this systematic review were used as the input for a cost-effectiveness study of serological testing for active TB in India [76]. In comparison with sputum microscopy, serological testing resulted in fewer disability-adjusted life years averted and more false-positive diagnoses and secondary infections, while increasing costs to the Indian TB control sector approximately 4-fold. This cost-effectiveness study and the findings from our updated systematic review were considered by a WHO Expert Group on Serodiagnostics, and in July 2011, the WHO published a policy statement on commercial serodiagnostic tests for diagnosis of TB. The policy states that “Commercial serological tests provide inconsistent and imprecise estimates of sensitivity and specificity. There is no evidence that existing commercial serological assays improve patient-important outcomes, and high proportions of false-positive and false-negative results adversely impact patient safety. Overall data quality was graded as very low, with harms/risks far outweighing any potential benefits (strong recommendation). It is therefore recommended that these tests should not be used in individuals suspected of active pulmonary or extra-pulmonary TB, irrespective of their HIV status.” The WHO policy strongly encourages targeted further research to identify new/alternative point-of-care tests for TB diagnosis and/or serological tests with improved accuracy [77].

Methodological quality summary, studies of anda-TB IgG, smear-positive patients. Review authors' judgments about each methodological quality item.

(TIF)

Click here for additional data file.

Methodological quality summary, studies of anda-TB IgG, smear-negative patients. Review authors' judgments about each methodological quality item.

(TIF)

Click here for additional data file.

TB serological assays on the Indian market with accuracy estimates from package inserts.

(DOC)

Click here for additional data file.

Characteristics of included studies evaluating serological tests for the diagnosis of pulmonary TB.

(DOC)

Click here for additional data file.

Characteristics of included studies evaluating serological tests for the diagnosis of extrapulmonary TB.

(DOC)

Click here for additional data file.

PRISMA statement.

(DOC)

Click here for additional data file.

Protocol.

(DOC)

Click here for additional data file.

PubMed literature search, 1 May 2006 to 10 February 2010. A weekly search was run through 29 June 2010.

(DOC)

Click here for additional data file.

List of excluded studies and reasons for their exclusion.

(DOC)

Click here for additional data file.