Performance of the TB-LAMP diagnostic assay in reference laboratories: Results from a multicentre study.

OBJECTIVE: To evaluate the diagnostic performance of TB-LAMP, a manual molecular tuberculosis (TB) detection method, and provide comparison to the Xpert MTB/RIF assay.
METHODS: In a large multicentre study, two sputum samples were collected from participants with TB symptoms in reference laboratories in Peru, South Africa, Brazil, and Vietnam. Each sample was tested with TB-LAMP. The reference standard consisted of four direct smears, four cultures, and clinical and radiological findings. Individuals negative on conventional tests were followed up after 8 weeks. The Xpert MTB/RIF assay was performed on fresh or frozen samples as a molecular test comparison.
RESULTS: A total of 1036 adults with suspected TB were enrolled. Among 375 culture-confirmed TB cases with 750 sputum samples, TB-LAMP detected 75.6% (95% confidence interval (CI) 71.8-79.4%), including 97.9% (95% CI 96.4-99.4%) of smear-positive TB samples and 46.6% (95% CI 40.6-52.7%) of smear-negative TB samples. Specificity in 477 culture-negative participants not treated for TB (954 sputum samples) was 98.7% (95% CI 97.9-99.6%). TB-LAMP test results were indeterminate in 0.3% of cases.
CONCLUSIONS: TB-LAMP detects nearly all smear-positive and half of smear-negative TB cases and has a high specificity when performed in reference laboratories. Performance was similar to the Xpert MTB/RIF assay.

Introduction

In 2016, an estimated 10.4 million people worldwide developed tuberculosis (TB) and 1.3 million HIV-negative people died from the disease (World Health Organization, 2017). The lack of rapid and accurate diagnostic tools contributed to the estimated 4.1 million cases of TB that went unreported (TDR, FIND SA, 2006). Smear microscopy, the first line in TB detection, has a low and variable sensitivity (30–70%) (Steingart et al., 2006, Aber et al., 1980, Urbanczik, 1985). Conventional culture on solid media has a sensitivity of 80–90%, but requires 2–8 weeks for a result (Lee et al., 2003; Somoskövi et al., 2000). Although conventional liquid culture (Mycobacteria Growth Indicator Tube (MGIT); Becton Dickinson Microbiology Systems) typically has sensitivity greater than 95%, the time to result is still up to 4 weeks. This results in individuals being lost to follow-up and treatment delays (Stall et al., 2011).

The use of molecular methods such as nucleic acid amplification tests (NAAT) may promise faster results with high sensitivity and specificity. The Xpert MTB/RIF assay (Cepheid, Sunnyvale, CA, USA) is an automated molecular method based on PCR. Initial demonstration studies found the sensitivity to be 77% in those with smear-negative, culture-positive TB and specificity to be >98% (Boehme et al., 2010, Boehme et al., 2011). A Cochrane review in 2014 found sensitivity in those with smear-negative, culture-positive TB to be 67% and specificity to be 99% (Steingart et al., 2014). However, Xpert MTB/RIF requires a continuous supply of electricity, hefty investment in equipment, and long-term maintenance plans.

TB-LAMP is a new, commercially available, manual molecular TB detection method based on the novel loop-mediated isothermal amplification platform (LAMP), manufactured by Eiken Chemical Co. in Japan. LAMP is an attractive diagnostics platform because it takes less than 2 h to perform, requires minimal instrumentation, and generates a fluorescent result that can be detected with the naked eye (Notomi et al., 2000). Furthermore, it has shown potential as a cross-disease platform, with assays developed for malaria, African trypanosomiasis, severe acute respiratory syndrome, and influenza (Kuboki et al., 2003, Poon et al., 2005a, Poon et al., 2005b, Poon et al., 2006). The TB-LAMP assay requires only minimal instrumentation in the form of a heating block. Additionally, it has the potential for higher throughput as it can test up to 14 samples per test run (Boehme et al., 2011, Vassall et al., 2011). A full description of the TB-LAMP procedure can be found in the recent World Health Organization (WHO) policy guidance (WHO Policy Guidance, 2016).

This study aimed to determine the accuracy of TB-LAMP in a single raw sputum sample in comparison to conventional methods and the Xpert MTB/RIF assay when performed in quality-assured TB reference laboratories.

Study population and methods

TB reference laboratories were selected in four urban centres: Cape Town in South Africa, Lima in Peru, Ho Chi Minh City in Vietnam, and Rio de Janeiro in Brazil. The reference laboratories selected were enrolled in national or international quality assurance programmes and had each undergone a laboratory assessment by FIND prior to selection. The study was approved by the institutional review boards in all countries. Informed consent was obtained from all participants. Additionally, while participants may have received better than standard diagnostic care through the use of additional cultures, clinicians were blinded to the results of TB-LAMP so as not to impact patient care.

Adults (≥18 years) with symptoms suggestive of pulmonary TB, as defined by national TB programmes, were enrolled consecutively if they were able to provide two sputum samples of at least 1.5 ml and had not received TB treatment in the preceding 60 days.

Each of the two sputum samples obtained had 60 μl removed for TB-LAMP and approximately 10 μl for direct Ziehl–Neelsen (ZN) and/or light-emitting diode (LED) fluorescence microscopy (FM) smear (Figure 1 ). The TB-LAMP technician was blinded to the smear results and vice versa. The remaining sputum was then processed with n-acetyl-l-cysteine–sodium hydroxide (NALC—NaOH) and used for solid culture (Löwenstein–Jensen medium) and MGIT liquid culture (WHO, 1998, Siddiqi and Ruesch-Gerdes, 2006). The first positive culture from the two performed per sample (two samples per individual) underwent confirmation of Mycobacterium tuberculosis (MTB) complex by MPT64 antigen detection (Capilia TB; Tauns Laboratories) (Hillemann et al., 2005).

Figure 1

Sample flow (Abbreviations: ZN, Ziehl–Neelsen microscopy; FM, fluorescence microscopy; NALC—NaOH, n-acetyl-l-cysteine–sodium hydroxide; PBS, phosphate-buffered saline; Xpert, Xpert MTB/RIF; User A, the technician performing TB-LAMP and blinded to the smear results; User B, the technician performing smears and blinded to the TB-LAMP results).

A scanty positive culture was defined as a positive MGIT at >28 days from inoculation or a Löwenstein–Jensen with <20 colonies. A culture-positive TB case was diagnosed if a participant had either two scanty or any non-scanty positive MTB culture result, consistent with previous analysis of molecular TB diagnostics (Boehme et al., 2010, Boehme et al., 2011). Non-tuberculous mycobacteria (NTM) or mixed cases were analyzed separately.

Samples from culture-positive TB cases were further classified as smear-positive or smear-negative based on a single smear test performed on that sputum. A smear-positive TB sample required one or more acid-fast bacilli per 100 fields for ZN or per 400 fields for FM (scanty grade or higher) following WHO recommendations (World Health Organization, 2007). A smear-negative TB sample had no acid-fast bacilli detected by ZN or FM whatsoever, but was found through culture testing to be a culture-positive TB case. Samples from participants with all cultures negative and any positive smear were analyzed separately.

Any participant with all smear results negative and all four cultures negative (or up to two cultures contaminated and the remaining negative), was considered bacteriologically negative for TB. Bacteriologically negative participants found to be TB-LAMP-positive on either sputum specimen were sought for clinical and laboratory follow-up after 8 weeks. Bacteriologically negative participants found to be TB-LAMP-negative were also followed up wherever programmatically possible.

A non-TB case was defined as a participant who was bacteriologically negative and for whom no TB treatment was prescribed (based on adjunct diagnostics such as chest X-ray and/or symptoms) at enrolment or during follow-up after 8 weeks. Bacteriologically negative participants who were treated empirically at enrolment or follow-up were considered to have clinically diagnosed TB. Such decisions were at the discretion of each physician and were subject to local variation.

Those initially bacteriologically negative with any positive culture at follow-up were reclassified as culture-positive TB. Those with no laboratory confirmation but either unimproved chest X-ray (i.e., physician compared a follow-up chest X-ray to the initial chest X-ray and found it to be ‘same abnormal’ or ‘worse’) or non-remitting symptoms (i.e., physician described symptoms as ‘same’ or ‘worse’) at follow-up were reclassified as possible TB and analyzed separately.

If the TB-LAMP technician could not determine from the fluorescent read-out whether TB-LAMP was positive or negative, a second reader was asked to make the determination. If the second read was also indeterminate, a second TB-LAMP test was repeated on the same sputum sample wherever possible. Each run of up to 14 TB-LAMP tests at one time included a negative and a positive control. If the negative control is positive, this indicates potential DNA contamination and all tests in that run must be repeated after decontamination procedures. If the positive control is negative, this indicates probable reagent degradation and all tests in that run must be repeated with new reagents. The overall indeterminate rate indicates the lack of a clear result and the need for a repetition of the test from any of these causes.

The performance of TB-LAMP on a single raw sputum sample was evaluated against the reference standard of two direct ZN smears, two direct FM smears, and four cultures per patient. Because a TB-LAMP test was performed on each of the two samples submitted per participant, correlations in the results were accounted for by using bootstrapping with sampling by cluster, where each set of two samples was the cluster, to obtain exact standard errors to calculate correct 95% confidence intervals (95% CI) for a single raw sample.

The performance of the Xpert MTB/RIF assay was evaluated as a molecular test comparison to TB-LAMP. In Peru and South Africa, where the assay was already available and in use, the Xpert MTB/RIF assay was performed on freshly concentrated sputum, whereas in Brazil and Vietnam, the Xpert assay was performed on sputum that had been frozen for 2–6 months. The Xpert MTB/RIF assay was performed after NALC—NaOH concentration from the first sputum sample collected, according to the manufacturer’s instructions (Figure 1). In Peru, the assay was performed from a third sputum sample if provided and from the first sputum sample if not. This third sample underwent only direct ZN and FM smears before concentration with NALC—NaOH then Xpert MTB/RIF testing. In Brazil and Vietnam, after NALC—NaOH concentration, >0.5 ml of the remainder of the first sample collected was frozen at −70 °C and tested by Xpert MTB/RIF after 2–6 months.

Any sample reported as positive for MTB by the automated output from the Xpert MTB/RIF was considered positive, while samples reported as MTB-negative were considered negative. Any error, invalid, or no result value was considered indeterminate and ineligible for sensitivity/specificity analysis. 95% CI were obtained using the binomial distribution. Blinding was deemed unnecessary given the automated nature of the test.

Results

Between January and December 2012, 1036 eligible participants were enrolled across the four sites (Table 1 ). Results from 25 participants were excluded from analysis for the following reasons: two for having more than two contaminated cultures; five were culture-positive but missing results of MPT64; 13 had a single positive culture after >28 days or <20 colonies; two were missing smear results; three were found to be culture-positive and NTM but only at follow-up.

Table 1

Demographic characteristics.

	Peru	South Africa	Brazil	Vietnam	Overall
Included/enrolled	196/199	245/259	224/266	299/312	964/1036
Male, n (%)	98 (50%)	161 (66%)	138 (62%)	210 (70%)	607 (63%)
Age, years, median (range)	41 (18–86)	38 (18–77)	48 (18–81)	37 (18–81)	40 (18–86)
HIV-positive, n (%)	2 (1.0%)	87 (36%)	1 (0.4%)	6 (2.0%)	96 (10%)
Culture-positive TB prevalence, n (%)	42 (21%)	63 (26%)	72 (32%)	198 (66%)	375 (39%)
Smear-negative, culture-positive TB prevalencea, n (%)	22 (11%)	32 (13%)	14 (6%)	88 (29%)	156 (16%)
Clinical TB prevalence, n (%)	4 (2.0%)	14 (5.7%)	3 (1.3%)	22 (7.4%)	43 (4.5%)
NTM only, n (%)	1 (0.5%)	0 (0.0%)	22 (8.3%)	6 (1.9%)	29 (2.8%)
Follow-up rate among TB-LAMP-positive, culture-negative (%)	1/1 (100%)	0/1 (0%)	11/14 (79%)	3/4 (75%)	15/20 (75%)
Follow-up rate among TB-LAMP-negative, culture-negative (%)	152/153 (99%)	169/181 (93%)	0/140 (0%)	60/98 (61%)	381/572 (67%)

NTM, non-tuberculous mycobacteria; TB, tuberculosis.

Based on two direct Ziehl–Neelsen smears.

Across all sites, there were 375 culture-positive TB cases, 477 non-TB participants, 43 with clinically diagnosed TB, and 69 with possible TB. These were the 964 participants and 1928 sputum samples in the primary analysis (Figure 2 ). There were 38 participants with NTM (29/38) or mixed MTB–NTM infections (9/38) who were analyzed separately (Table 2 ). Also analyzed separately were nine FM smear-positive, culture-negative individuals.

Figure 2

Participants and sputum samples—diagnostic breakdown (Abbreviations: S+C+, smear-positive, culture-positive; S−C+, smear-negative, culture-positive; CI, confidence interval; TB, tuberculosis).

Table 2

TB-LAMP samples analyzed separately among those with NTM or mixed MTB–NTM infections.

	Peru	South Africa	Brazil	Vietnam	Overall
Clinical TB (+/n)	0/8	0/28	1/6	1/44	2/86
Possible TB (+/n)	0/20	0/72	5/8	3/38	8/138
ZN smear-positive, culture negative (+/n)	0/0	0/0	0/0	0/0	0/0
FM smear-positive, culture-negative (+/n)	0/1	0/0	0/7	0/1	0/9
NTM sample only (−/n)	1/1	0/0	25/35	6/15	32/51
Mixed infection, MTB sample only (+/n)	0/0	0/0	2/5	4/4	6/9

+/n, number of positive samples out of the total number of samples; −/n, number of negative samples out of the total number of samples; FM, fluorescence microscopy; MTB, Mycobacterium tuberculosis; NTM, non-tuberculous mycobacteria; TB, tuberculosis; ZN, Ziehl–Neelsen.

The follow-up rate among individuals with a positive TB-LAMP and all negative smear/culture results was 75% (15/20), while the follow-up rate among participants with negative results on all tests was 67% (381/572) (Table 1). The overall follow-up rate for those with negative smear/culture results in South Africa was 93% (169/182) and in Peru was 99% (153/154), while in Vietnam it was only 62% (63/102). In Brazil, 79% (11/14) of participants with a positive TB-LAMP and negative smear/culture were followed up, but none of those who were negative on all tests were followed up due to programmatic restrictions.

The overall TB-LAMP indeterminate rate was 0.3%. Two out of 2081 TB-LAMP tests performed could not be interpreted after two readings and four had a positive result from the negative control, indicating likely DNA contamination for that run of the assay. All had clear positive or negative results after repetition of the TB-LAMP assay on the same sputum.

For the 375 participants with culture-positive TB and with two sputum samples each (750 samples), 567 of these samples tested positive by TB-LAMP, giving a sensitivity of 75.6% (567/750; 95% CI 71.8–79.4%). Sensitivity was 97.9% (415/424; 95% CI 96.4–99.4%) and 46.6% (152/326; 95% CI 40.6–52.7%) in direct ZN smear-positive and smear-negative TB samples, respectively (Table 3 ). Overall results classified using direct FM smears were similar (direct FM results include only Peru, Vietnam, and Brazil).

Table 3

TB-LAMP performance in a single direct sputum sample per participant (using two samples per participant).

		Peru	South Africaa	Brazil	Vietnam	Overall
Sensitivity
Culture-positive TB		89.3%	68.3%	86.8%	71.0%	75.6%
	95% CI	80.0–98.5	57.6–78.9	80.1–93.5	65.6–76.3	71.8–79.4
	(m/n)	(75/84)	(86/126)	(125/144)	(281/396)	(567/750)
Direct ZN smears	Smear-positive TB	100%	98.2%	100%	96.2%	97.9%
	95% CI	92.1–100	94.8–100	96.7–100	93.4–99.1	96.4–99.4
	(m/n)	(45/45)	(56/57)	(109/109)	(205/213)	(415/424)
	Smear-negative TB	76.9%	43.5%	45.7%	41.5%	46.6%
	95% CI	58.6–95.2	29.2–57.7	28.9–62.5	34.0–49.1	40.6–52.7
	(m/n)	(30/39)	(30/69)	(16/35)	(76/183)	(152/326)
Direct FM smears	Smear-positive TB	98.4%	–	100%	92.4%	95.0%
	95% CI	95.2–100		95.6–100	88.6–96.2	92.5–97.5
	(m/n)	(61/62)		(82/82)	(218/236)	(361/380)
	Smear-negative TB	63.6%	–	69.4%	39.4%	49.2%
	95% CI	33.3–94.0		56.1−82.6	31.2–47.5	42.0–56.4
	(m/n)	(14/22)		(43/62)	(63/160)	(120/244)
Specificity
Non-TB, including follow-up		99.6%	99.6%	97.2%	98.3%	98.7%
95% CI		98.9–100	98.9–100	95.2–99.3	95.0–100	97.9–99.6
(m/n)		(279/280)	(263/264)	(282/290)	(118/120)	(942/954)
Non-TB, excluding follow-up		99.7%	99.7%	94.7%	96.9%	97.9%
95% CI		99.0–100	99.1–100	91.8–97.6	93.2–100	96.9–98.9
(m/n)		(299/300)	(335/336)	(286/302)	(155/160)	1075/1098

CI, confidence interval; m, TB-LAMP-positive or negative samples; n, total number of samples; TB, tuberculosis; ZN, Ziehl–Neelsen microscopy; FM, fluorescence microscopy.

South Africa performed only direct Ziehl–Neelsen smears.

Nine hundred and forty-two of 954 sputum samples from 477 non-TB participants tested negative on TB-LAMP, giving an overall specificity of 98.7% (95% CI 97.9–99.6%) (Table 3). Inclusion of follow-up information made the greatest impact in Brazil where specificity was 97.2% (282/290; 95% CI 95.2–99.3%) but only 94.7% (286/302; 95% CI 91.8–97.6%) exclusive of follow-up (i.e., inclusive only of initial clinical and laboratory findings and whether the individual was immediately put on treatment). In Vietnam, specificity also improved with the inclusion of follow-up (98.3%; 118/120; 95% CI 95.0–100%). Specificity in Peru was 99.6% (279/280; 95% CI 98.9–100%) and in South Africa was also 99.6% (263/264; 95% CI 98.9–100%); at these sites, specificity was similar irrespective of whether follow-up data were considered or not.

Only South Africa had a significant number of HIV-positive individuals (35.5%). The overall sensitivity of TB-LAMP in sputum samples from culture-positive, HIV-positive participants was 51.9% (28/54; 95% CI 34.1–69.6%). By smear status, in HIV-positive participants, sensitivity was 100% (19/19; 95% CI 82.4–100%) in smear-positive samples and 26.3% (9/35; 95% CI 8.6–42.9%) in smear-negative samples.

The performance of TB-LAMP was compared to that of Xpert MTB/RIF (Table 4 ). In this study, overall sensitivity in smear-positive TB cases was 97.9% (415/424; 95% CI 96.4–99.4%) for TB-LAMP and 99.0% (207/209; 95% CI 96.6–99.9%) for Xpert MTB/RIF (p  = 0.32, Pearson’s χ2 test). Sensitivity in smear-negative TB cases was 46.6% (152/326; 95% CI 40.6–52.7%) for TB-LAMP and 52.3% (81/158; 95% CI 43.2–59.3) for Xpert MTB/RIF (p  = 0.63). Specificity was 98.7% (942/954; 95% CI 97.9–99.6%) for TB-LAMP and 97.2% (457/470; 95% CI 95.3–98.5%) for Xpert MTB/RIF (p  = 0.07). The Xpert MTB/RIF indeterminate rate was 0.9% (9/973).

Table 4

Xpert MTB/RIF performance in a single concentrated sputum sample per study participant.

	Peru	South Africa	Brazil	Vietnam	Overall
% of included individuals with a valid Xpert MTB/RIF result (m/n)	95.9%	100%	100%	97.3%	98.3%
	(188/196)	(245/245)	(224/224)	(291/299)	(948/964)
Sensitivity
Culture-positive TB	87.2%	66.7%	86.1%	77.7%	78.5%
95% CI	72.6–95.7	53.7–78.0	75.9–93.1	71.2–83.4	73.9–82.6
(m/n)	(34/39)	(42/63)	(62/72)	(150/193)	(288/367)
ZN direct smear-positive TB	95.7%	100%	100%	99.0%	99.0%
95% CI	78.1–99.9	88.1–100	93.3–100	94.8–100	96.6–99.9
(m/n)	(22/23)	(29/29)	(53/53)	(103/104)	(207/209)
ZN direct smear-negative TB	75.0%	38.2%	47.4%	52.8%	52.3%
95% CI	47.6–92.7	22.2–56.4	24.4–71.1	41.9–63.5	43.2–59.3
(m/n)	(12/16)	(13/34)	(9/19)	(47/89)	(81/158)
Clinical TB (m/n)	0/4	1/14	1/3	2/21	4/42
Possible TB (m/n)	0/10	0/36	1/4	1/19	2/69
Specificity
Non-TB, including follow-up	100%	100%	91.7%	98.3%	97.2%
95% CI	97.3–100	97.2–100	86.0–95.7	90.8–100	95.3–98.5
(m/n)	(135/135)	(132/132)	(133/145)	(57/58)	(457/470)
Non-TB, excluding follow-up	93.1%	78.6%	88.1%	73.1%	84.3%
95% CI	87.7–96.6	71.6–84.5	81.8–92.8	61.8–82.5	81.0–87.3
(m/n)	(135/145)	(132/168)	(133/151)	(57/78)	(457/542)

CI, confidence interval; m, Xpert MTB/RIF-positive or negative samples; n, total number of samples; TB, tuberculosis; ZN, Ziehl–Neelsen microscopy.

Discussion

This study was successful in demonstrating the performance of the TB-LAMP assay. TB-LAMP can reliably detect nearly all smear-positive TB cases and roughly half of smear-negative TB cases from a single raw sputum sample. In this study in TB reference laboratories, no significant difference in sensitivity and specificity was found for TB-LAMP compared to Xpert MTB/RIF using four direct smears and four cultures as the reference standard.

Brazil had the lowest specificity rates for TB-LAMP and Xpert MTB/RIF, as well as the least follow-up information. Given programmatic capabilities, follow-up was only asked of participants positive on TB-LAMP and with bacteriologically negative results (79% follow-up rate); this resulted in significant improvement in the TB-LAMP specificity (94.7% excluding follow-up and 97.2% including follow-up). Directed follow-up was not performed for participants with Xpert MTB/RIF-positive and bacteriologically negative results; furthermore, the use of an imperfect standard will have a more deleterious effect on an assay with higher specificity. Although it could not be confirmed, it is possible that specimen decontamination before culture in Brazil was performed using 2% NaOH (where 1–1.5% is recommended), which would have destroyed weakly growing mycobacteria, thereby lowering culture sensitivity (Kent and Kubica, 1985, Global Laboratory Initiative, 2014). This hypothesis is further supported by the high number of smear-positive, culture-negative results found in Brazil. Finally, the laboratory in Brazil had lower sensitivity using FM (45.7%) than the more standard ZN (69.4%) – the opposite of what would be expected based on a previous meta-analysis of the two methods (Steingart et al., 2006).

Excluding Brazil, specificity would be 99.4% for TB-LAMP across the other three countries and 99.7% for Xpert MTB/RIF. This specificity for Xpert MTB/RIF agrees with data reported in the literature (Steingart et al., 2014).

Peru had an unusually high TB-LAMP sensitivity in ZN smear-negatives (76.9%), while Vietnam had unusually low TB-LAMP sensitivity in FM smear-negatives (39.4%). These variations may reflect the variability of smear performance from one laboratory to another. The same variability from site to site is seen for Xpert MTB-RIF as well, supporting this conclusion.

In conclusion, while this study was limited in that complete follow-up and standardization of the protocol was not possible, the findings contribute to the growing literature on the performance of molecular assays in the intended populations. Given the finding in this study that laboratory performance of TB-LAMP approaches that of Xpert MTB/RIF, further studies are recommended in settings of intended use to evaluate these expected benefits.

The use of a manual technique may give TB-LAMP the potential to enter the market at a lower cost than its automated counterpart (Vassall et al., 2011). It is hoped that the limited infrastructure needed to reliably power a heating block for 40 min will make it possible to run this assay in decentralized settings. Furthermore, the higher throughput of this assay, with up to 14 tests per batched 2-h run, may accommodate laboratories with moderate to high workloads at any level of the health system.

Conflict of interest

Clinical evaluation and assay development was supported by FIND. The authors have no other conflicts of interest to declare.