Diagnostic accuracy of the loop-mediated isothermal amplification assay for extrapulmonary tuberculosis: A meta-analysis.

BACKGROUND: Loop-mediated isothermal amplification (LAMP) is used to detect pulmonary tuberculosis (PTB); however, the diagnostic accuracy of the LAMP assay for extrapulmonary tuberculosis (EPTB) is unclear. We performed a meta-analysis to evaluate the performance of LAMP in the detection of EPTB.
METHODS: We searched PubMed, EMBASE, the Cochrane Library, China National Knowledge Infrastructure (CNKI), and the Wanfang database for studies published before Sep 16, 2017. We reviewed studies and compared the performance of LAMP with that of a composite reference standard (CRS) and culture for clinically suspected EPTB. We used a bivariate random-effects model to perform meta-analyses and used meta-regression and subgroup analysis to analyze sources of heterogeneity.
RESULTS: Fourteen articles including 24 independent studies (16 compared LAMP to CRS, 8 to culture) of EPTB were identified. LAMP showed a pooled sensitivity of 77% (95% confidence interval (CI) 68-85), specificity of 99% (95% CI 96-100), and area under SROC curves (AUC) of 0.96 (95% CI 0.94-0.97) against CRS. It showed a pooled sensitivity of 93% (95% CI 88-96), specificity of 77% (95% CI 64-86), and AUC of 0.94 (95% CI 0.92-0.96) against culture. The pooled sensitivity, specificity, and AUC of MPB64 LAMP were 86% (95% CI 86-86), 100% (95% CI 100-100), and 0.97 (95% CI 0.95-0.98), respectively, and those of IS6110 LAMP were 75% (95% CI 64-84), 99% (95% CI 90-100), and 0.91 (95% CI 0.88-0.93), respectively, compared with CRS.
CONCLUSIONS: These results suggest good diagnostic efficacy of LAMP in the detection of EPTB. Additionally, the diagnostic efficacy of MPB64 LAMP was superior to that of IS6110 LAMP.

Introduction

Tuberculosis (TB) is an infectious disease caused by Mycobacterium tuberculosis (MTB) and is one of the most serious challenges to public health [1]. The most common site of tuberculosis infection is the lung, but bacteria can also spread to extrapulmonary sites, causing extrapulmonary tuberculosis (EPTB). EPTB accounts for approximately 22% of total TB cases [2]. Diagnosis of EPTB is very challenging, because specimens of EPTB are not as easy to obtain by noninvasive methods as are sputum samples. Invasive procedures requiring special expertise are often required to obtain specimens such as cerebrospinal fluid and pleural effusion. Additionally, culture of EPTB specimens has low sensitivity. Biopsy along with histopathological examination and culture is required to diagnose EPTB.

There are many methods for diagnosis of tuberculosis. PCR tests require an expensive thermal cycler to amplify DNA fragments in multiple temperature-dependent steps. Therefore, some PCR assays, such as Xpert MTB/RIF, are very costly, which is an obstacle to application in low-income areas. Loop-mediated isothermal amplification (LAMP) is an isothermal DNA method that relies on two or three sets of primers to amplify minute quantities of DNA within a shorter period of time. Compared with other nucleic acid amplification tests, LAMP is very economical. This is a new assay with high accuracy for pulmonary TB detection [3], but there are no systematic studies assessing its diagnostic accuracy for EPTB.

For this purpose, we performed a meta-analysis to reveal the diagnostic test accuracy of the LAMP assay for EPTB using data from previous studies of the LAMP assay compared with that of a composite reference standard (CRS) and culture reference in the detection of EPTB. We analyzed the pooled sensitivity and specificity of this assay against different references. Moreover, the diagnostic efficiency of the test according to different target genes, types of samples, incubation times, condition of samples, and types of LAMP were evaluated by subgroup analysis.

Methods

We followed the standard guidelines to perform this meta-analysis [4-6].

Data sources and search strategy

We searched PubMed, EMBASE, the Cochrane Library, China National Knowledge Infrastructure (CNKI), and the Wanfang database for studies evaluating LAMP accuracy in TB published before Sep 16, 2017. The search formula ((“Loop-Mediated Isothermal Amplification” OR LAMP) AND (“Tuberculosis”[Mesh] OR “Tuberculoses” OR “Kochs Disease” OR “Disease, Kochs” OR “Koch's Disease” OR “Disease, Koch's” OR “Koch Disease” OR TB)) was used for PubMed without any language restrictions. The search formulas for EMBASE, the Cochrane Library, CNKI, and the Wanfang database were similar to the PubMed search formula. The search strategies for each database were shown in the S1 File. References of included articles and published reviews were also reviewed for possible candidate studies. We extracted data including author, year, country, true positive (TP), false positive (FP), false negative (FN), true negative (TN) values for the assay, reference standard, target gene, and specimen type, as well as other parameters.

Inclusion criteria

We included full text original studies assessing the diagnostic accuracy of the LAMP assay for EPTB using extrapulmonary site specimens. Reference standards were defined in the studies and were appropriate. Articles directly provided TP, FP, FN, and TN values for the assay, or included the data necessary to calculate these measures. Case reports, studies of fewer than 10 samples, abstracts, and conference reports without full articles were excluded.

Reference standard

A composite reference standard (CRS) or mycobacterial culture was defined as the reference standard in the studies. Clinical manifestation, biochemical testing results, smears, histopathology, other nucleic acid amplification tests (NAATs), culture, or a response to anti-tuberculosis treatment constituted the reference standards in the CRS.

Literature screening and selection

Two investigators independently assessed candidate articles by reviewing titles and abstracts, then full text for inclusion. Discrepancies between the two decisions were resolved by discussion with a third investigator.

Data extraction

The same two investigators independently extracted the necessary information from each of the included articles. We then cross-checked the information obtained by the two investigators. Discrepancies between the two data sets were settled by discussion with a third investigator, just as in the literature selection phase. Data from studies against two different reference standards or target genes were treated separately.

Assessment of study quality

According to the two reference standards (CRS and culture), the two investigators independently divided studies into two groups and used a revised tool for Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) to assess study quality separately [7]. Publication bias was not assessed, because these methods are not applicable for studies of diagnostic accuracy [8].

Data synthesis and statistical analysis

We first obtained the numbers of TP, FP, FN, and TN in each included study, and then calculated the estimated pooled sensitivity and specificity of LAMP associated with 95% CI against CRS or culture using bivariate random-effects models. Forest plots of the sensitivity and specificity as well as summary receiver operating characteristic (SROC) curves were generated for each study. The area under SROC curves (AUC) was likewise calculated. The I2 statistics were calculated to assess the heterogeneity between studies compared with a standard reference. A value of 0% indicated no observed heterogeneity, and values greater than 50% were considered substantially heterogeneous [9, 10]. We explored targeted genes, types of samples, incubation time, condition of samples and types of LAMP as potential sources of heterogeneity using subgroup and meta-regression analyses. At least four available studies were needed to carry out the meta-analysis for a predefined variable type. Data from studies against CRS and culture were analyzed separately. Stata version 14.0 (Stata Corp, College Station, TX, USA) with the MIDAS command packages was used to analyze the results.

Imperfect reference standard

Imperfect reference standards may lead to misclassification of samples in diagnostic validity studies [11, 12]. For the paucibacillary nature of EPTB, culture is an imperfect reference standard and leads to an underestimation of the true specificity of LAMP. A CRS is a composite standard that comprises the results of several tests; however, a CRS itself may have reduced specificity that could result in apparent FN LAMP results, also leading to an underestimation of the true sensitivity of LAMP [13, 14]. Therefore, a study comparing LAMP with both culture and CRS might provide a more credible range for sensitivity and specificity.

Results

Identification of studies and study characteristics

Eight hundred candidate articles were identified by searching the relevant databases using our search strategy, and 14 qualified articles were included according to the inclusion criteria (Fig 1, S2 File) [15-28]. The number of specimens evaluated in each article ranged from 27 to 315 with a median of 118. Twelve articles were written in English, and 2 in Chinese. All studies were conducted in countries with high tuberculosis burdens (India and China). We excluded two studies [29, 30] that had the same data as other included studies [15, 25], and one study [31] whose data was part of another study [28].

Fig 1

Literature retrieval flow chart.

Searches of PubMed, the Cochrane Library, Embase, Wanfang database and CNKI returned 264, 11, 328, 98, and 99 articles, respectively.

When an article reported the use of two different standards or target genes in the same study, we considered the article to include two independent studies. In accordance with this principle, 24 independent studies were included: 16 compared LAMP with CRS and 8 with culture (Table 1, S1 Table). The characteristics of the LAMP test used in the 14 articles are also shown in Table 1. The most common target genes were IS6110 and MPB64, used in 11 and 5 studies, respectively. Two studies [16, 28] did not define the target gene. The incubation temperatures of all experiments were similar, at approximately 65°C. The specimens included cerebrospinal fluid (CSF), pleural effusion, synovial fluid, pus, fine needle aspiration (FNA) of lymph glands, and others. Eight articles used only one type of specimen (e.g., only CSF) [20–22, 24–28]. The other studies used multiple types of specimens [15–19, 23]. Only one study provided HIV infection status [26]. Only two studies provided median or mean age. The CRS criteria used in the articles included the results of culture.

Table 1

Characteristics of the included studies.

Study	Country	TP(n)	FP(n)	FN(n)	TN(n)	Sensitivity(%)	Specificity(%)	Reference	Gene	Incubation time(min)	Incubation temperature(°C)	Sample condition	LAMP assay	Sample Type
BinFeng Yanga 2011	China	18	0	54	24	25%	100%	CRS	IS1081	60	65	Unknown	Loopamp MTBC	PE
BinFeng Yangb 2011	China	35	4	37	20	49%	83%	CRS	IS1081	90	65	Unknown	Loopamp MTBC	PE
Nagdev, K. J. 2011	India	15	2	2	8	88%	80%	CRS	IS6110	60	63	Frozen	Loopamp MTBC	CSF
Shihui Zhang 2012	China	31	6	2	19	94%	76%	Culture	Unknown	60	65	Fresh	Unclear	Urine
Kumar, P. 2014	India	22	23	0	32	100%	58%	Culture	esat6	35	65	Unknown	In-house	Extrapulmonary samples
Yang Liu 2015	China	44	2	14	30	76%	94%	CRS	hspX	60	63	Fresh	Loopamp MTBC	PE
Joon, D.a 2015	India	28	23	2	262	93%	92%	Culture	sdaA	60	65	Frozen	In-house	Extrapulmonary samples
Joon, D.b 2015	India	49	2	4	260	92%	99%	CRS	sdaA	60	65	Frozen	In-house	Extrapulmonary samples
Balne, P. K. 2015	India	25	0	8	20	76%	100%	CRS	MPB64	60	65	Frozen	In-house	Vitreous and aqueous humor
Sharma, M.a 2016	India	96	0	24	50	80%	100%	CRS	IS6110	45	63	Frozen	In-house	FNA
Sharma, M.b 2016	India	103	0	17	50	86%	100%	CRS	MPB64	45	63	Frozen	In-house	FNA
Sethi, S.a 2016	India	5	57	2	236	71%	81%	Culture	IS6110	60	64	Fresh	In-house	Endometrial biopsy samples
Sethi, S.b 2016	India	45	17	23	215	66%	93%	CRS	IS6110	60	64	Fresh	In-house	Endometrial biopsy samples
Modi, M.a 2016	India	46	78	4	122	92%	61%	Culture	IS6110	45	63	Frozen	In-house	CSF
Modi, M.b 2016	India	48	82	2	118	96%	59%	Culture	MPB64	45	63	Frozen	In-house	CSF
Modi, M.c 2016	India	124	0	26	100	83%	100%	CRS	IS6110	45	63	Frozen	In-house	CSF
Modi, M.d 2016	India	130	0	20	100	87%	100%	CRS	MPB64	45	63	Frozen	In-house	CSF
Wenwen Suna 2017	China	74	2	98	26	43%	93%	CRS	IS6110	40	67	Frozen	Loopamp MTBC	CSF
Wenwen Sunb 2017	China	20	56	2	122	91%	69%	Culture	IS6110	40	67	Frozen	Loopamp MTBC	CSF
Sharma, K.a 2017	India	75	0	15	50	83%	100%	CRS	IS6110	45	63	Frozen	In-house	Synovial fluid and pus
Sharma, K.b 2017	India	79	0	11	50	88%	100%	CRS	MPB64	45	63	Frozen	In-house	Synovial fluid and pus
Joon, D.a 2017	India	43	0	10	262	81%	100%	CRS	IS6110	90	63	Frozen	In-house	Extrapulmonary samples
Joon, D.b 2017	India	26	17	4	268	87%	94%	Culture	IS6110	90	63	Frozen	In-house	Extrapulmonary samples
Ghosh, P. K. 2017	India	22	1	1	21	96%	95%	CRS	Unknown	60	65	Frozen	In-house	Extrapulmonary samples

CRS: Composite reference standard. PE: Pleural effusion. FNA: Fine needle aspiration. CSF: Cerebrospinal fluid. Extrapulmonary samples included PE, CSF, and others.

Study quality

The overall methodological quality of the included studies using a CRS and culture is summarized in Fig 2. Only four studies used a case–control design [15, 21, 26, 27].

Fig 2

Methodological quality graphs (risk of bias and applicability concerns) as percentages across the included studies.

a: composite reference standard. b: culture reference standard.

Diagnostic accuracy of the LAMP assay for EPTB detection

When compared to a CRS using 2001 samples in 16 studies, the combined sensitivity and specificity of the LAMP assay for EPTB were 77% (95% CI 68–85) and 99% (95% CI 96–100), respectively (Fig 3A). The I2 statistical values were 95% for sensitivity and 85% for specificity, suggesting significant heterogeneity in diagnostic validity between the studies. When compared to a culture reference standard (8 studies, 1515 samples), the combined sensitivity of LAMP was 93% (95% CI 88–96) with I2 = 49% and the specificity was 77% (95% CI 64–86) with I2 = 96% for 1515 specimens in 8 studies (Fig 3B). The heterogeneity of the sensitivity was acceptable; however, the heterogeneity of the specificity was significant. The AUC of SROC was 0.96 (95% CI 0.94–0.97) and 0.94 (95% CI 0.92–0.96) versus vs CRS and culture, respectively, suggesting very good overall diagnostic validity.

Fig 3

Forest plot of LAMP sensitivity and specificity for tuberculosis detection in EPTB.

a: composite reference standard. b: culture reference standard. The squares represent the sensitivity and specificity of a study, and the black line their confidence intervals. The diamonds represent the pooled sensitivity and specificity and their confidence intervals. LAMP: loop-mediated isothermal amplification; EPTB: extrapulmonary tuberculosis.

We explored the heterogeneity between the studies using hierarchical analysis on predefined subgroups of target genes, sample types, incubation time, condition of samples and types of LAMP used in the assay.

The pooled sensitivity and specificity of the MPB64 LAMP assay (613 samples) vs. CRS were 86% (95% CI 86–86) with I2 = 3.33% and 100% (95% 100–100) with I2 = 0, respectively (Fig 4A). There was no heterogeneity in diagnostic validity between studies of MPB64 LAMP. The AUC of SROC was 0.97 (95% CI 0.95–0.98) for MPB64 LAMP vs CRS, suggesting very high efficiency. One study used MPB64 as the target gene in the LAMP assay compared with culture, but further analysis could not be carried out.

Fig 4

Forest plot of LAMP sensitivity and specificity for tuberculosis detection using different target genes against a composite reference standard.

a: MPB64. b: IS6110. The squares represent the sensitivity and specificity of one study, and the black line their confidence intervals. The diamonds represent the pooled sensitivity and specificity and their confidence intervals. LAMP: loop-mediated isothermal amplification.

When using IS6110 as the target gene, the pooled sensitivity and specificity of IS6110 LAMP compared with CRS were 75% (95% CI 64–84) and 99% (95% CI 90–100), respectively (Fig 4B). I2 statistical values were 95% and 88% for sensitivity and specificity, respectively, of IS6110 LAMP. The P values of meta-regression for sensitivity and specificity of the IS6110 LAMP assay against a non-IS6110 LAMP assay in comparison to CRS were 0.16 and 0.25, respectively, suggesting that this target gene was not a source of heterogeneity in the LAMP assay. Therefore, combining different studies to assess the diagnostic performance of the LAMP assay did not significantly skew the results. Compared with culture, the pooled sensitivity of IS6110 LAMP was 89% (95% CI 81–94), and specificity was 79% (95% CI 62–90). The I2 statistical values of IS6110 LAMP were 42% and 97% for sensitivity and specificity, respectively. Heterogeneity of sensitivity among the studies was moderate. The pooled sensitivity of MPB64 LAMP was significantly higher than that of IS6110 LAMP (P<0.05); however, the difference between the specificities was not statistically significant (P>0.05), and the AUC of MPB64 LAMP was higher than that of IS6110 LAMP when assessed against CRS. Data of other target genes were too limited to analyze.

Four studies assessed LAMP in CSF samples in comparison to a CRS. Pooled sensitivity was 76% (95% CI 56–89, I2 = 97%), and pooled specificity was 99% (95% CI 77–100, I2 = 89%) (Fig 5). The P-values of meta-regression for sensitivity and specificity were 0.33 and 0.29, respectively. The AUC of SROC was 0.99 (95% CI 0.77–1.00) for CSF samples vs CRS. Sensitivity and specificity of LAMP for pleural effusion against CRS ranged from 25% to 75.8% and 83.3% to 100%, respectively. For fine needle aspiration of lymph nodes, synovial fluid, and pus, sensitivity of this assay was 80%, 85.3%, and 83.3%, respectively. Sensitivity was 87.7% for IS6110 and MPB64 vs CRS, and specificity was consistent at 100%. However, data were too limited to perform meta-analysis.

Fig 5

Forest plot of LAMP sensitivity and specificity using cerebrospinal fluid samples against a composite reference standard.

The squares represent the sensitivity and specificity of one study, and the black line their confidence intervals. The diamonds represent the pooled sensitivity and specificity and their confidence intervals. LAMP: loop-mediated isothermal amplification.

When LAMP was compared to CRS, meta-regression showed that the type of LAMP was the source of heterogeneity (P < 0.05) rather than incubation time and sample condition (meta-regression P > 0.05). The pooled sensitivity and specificity of Loopamp MTBC (Eiken Chemical, Co., Tokyo, Japan) and in-house assays were 54% (95% CI 40–69) 93% (95% CI 84–100) and 84% (95% CI 79–90), 100% (95% CI 99–100), respectively. The differences between sensitivity and specificity were statistically significant (P < 0.01 and P = 0.03, respectively). However, heterogeneity in the subgroup was still relatively significant, and the results should be treated with caution.

Discussion

Timely and accurate diagnosis of tuberculosis is very important for effective management of the disease and prevention of infection in the community, particularly in areas with high burdens tuberculosis. Conventional diagnostic methods, such as smears and culture, are time-consuming and not very sensitive.

LAMP is an innovative point-of-care diagnostic technique with increased specificity, speed, and low cost [32]. It can provide results within 1 or 1.5 hours. Several studies have evaluated the diagnostic validity of the test for pulmonary TB [33, 34]. A systematic review and meta-analysis reported by Nagai et al. showed summary estimates of sensitivity at 89.6% (95% CI 85.6–92.6%) and specificity at 94.0% (95% CI 91.0–96.1%) and a diagnostic odds ratio (DOR) of 145 (95% CI 93–226) [35]. However, there has been no reported systematic review and meta-analysis evaluating the diagnostic accuracy of LAMP for EPTB. Ours is the first study for this purpose.

In this meta-analysis, we reviewed the diagnostic efficiency of the LAMP assay for EPTB compared with that of a CRS or culture reference. Based on AUC, the diagnostic performance of the LAMP assay was very good for EPTB, regardless of the reference standard used. However, this test was less effective than PCR assays such as Xpert MTB/RIF [36, 37]. We found that LAMP had very high pooled specificity (99% 95% CI 96–100) but more moderate pooled sensitivity (77% 95% CI 68–85) for the diagnosis of EPTB vs. CRS. As expected, when culture was used as the reference standard, the pooled sensitivity for the diagnosis of EPTB was improved to 93% (95% CI 88–96), and pooled specificity decreased to 77% (95% CI 64–86). However, there was obvious heterogeneity among the studies, and the results should be interpreted carefully.

For the detection of the MTB genome, several factors play important roles in standardizing a sensitive and specific LAMP assay. The target gene is an important factor, and a variety of target genes can be used in NAATs. In LAMP, the commonly used target genes are IS6110, MPB64, and IS1081, among others. Through this meta-analysis, we found that diagnostic efficacy was different when using different target genes. The IS6110 gene has been the favored target gene in studies using the LAMP assay, as multiple copies are present in the MTB genome [38, 39]. However, in this study, it was not the most efficient target gene for diagnosis of EPTB in areas with high burdens of tuberculosis. We observed that the pooled sensitivity and specificity of MPB64 LAMP were significantly higher than those of non-MPB64 LAMP (P = 0.03 and P = 0.00, respectively) when compared to CRS.

Heterogeneity between the studies using MPB64 LAMP was not significant. The pooled sensitivity and specificity of IS6110 LAMP compared with those of non-IS6110 LAMP were not significantly different vs. CRS. However, heterogeneity between IS6110 LAMP studies was very significant. The pooled sensitivity and AUC of MPB64 LAMP were higher than those of IS6110 LAMP against CRS. This result was consistent with those of studies using other NAATs [40, 41]. Included studies using culture as the reference standard were limited, and the difference in pooled sensitivity and specificity for the two target genes could not be analyzed.

We considered that LAMP accuracy for tuberculosis detection in EPTB specimens might vary widely according to specimen type, as it did in another systematic review and meta-analysis of EPTB diagnosis using the Xpert MTB/RIF assay [42]. However, our meta-analysis could not reach a conclusion, partially due to the limited number of studies using the same sample types. Only four studies used CSF to analyze the diagnostic accuracy of LAMP, and there were not enough separate studies using other sample types to carry out meta-analysis. Additionally, these results must be treated with caution, as the heterogeneity between the studies was very significant; this may lead to bias in the results. Further studies using different types of specimens are needed to assess the diagnostic accuracy of LAMP for individual samples.

We observed that incubation time and sample condition in LAMP assays did not affect test results. Different assay types might affect results, e.g., an in-house LAMP assay might be better than the Loopamp MTBC assay. As the heterogeneity between the studies was very significant, further studies using different types of the LAMP assay are needed to assess its sensitivity and specificity.

PCR tests are considered the most effective means of diagnosis [43]. However, these assays, such as Xpert MTB/RIF, are very costly, which is an obstacle to their application in low-income areas. LAMP is gradually being accepted as an alternative test in resource-limited areas due to its relatively small financial burden [35]. In the current meta-analysis, all studies were conducted in low-income countries where medical resources are limited. We observed that the effectiveness of LAMP in EPTB diagnosis was similar to that of Xpert MTB/RIF, which was consistent with a previous study [44]. However, compared with Xpert MTB/RIF, LAMP has shortcomings, such as its inability to determine rifampicin resistance. For low-income areas with low prevalence of drug-resistant tuberculosis, LAMP might be a useful alternative to Xpert MTB/RIF.

Several limitations existed in our review. First, the meta-analysis was limited by the number of studies using different target genes and sample types, particularly those comparing LAMP against culture. Only two target genes and one sample type (CSF) could be analyzed through meta-analysis; the diagnostic validity of the LAMP assay for other target gene and sample types could not be assessed. Some included studies used multiple sample types, which may have led to some bias in the results. Second, the quality of some studies in this analysis was relatively poor. The heterogeneity between the studies was remarkable, and the meta-analysis results should be interpreted with caution.

Conclusions

In this meta-analysis, we observed that the pooled sensitivity and specificity of LAMP for the detection of EPTB were 77% and 93%, respectively, when compared with a CRS, and 99% and 77%, respectively, when compared with culture. Depending on the assessment of AUC, LAMP showed good diagnostic efficacy. We also found that the diagnostic efficacy of LAMP tests varied according to different target genes; the diagnostic efficacy of MPB64 LAMP was better than that of IS6110 LAMP. The diagnostic accuracy of LAMP for different samples could not be effectively assessed, as the number of studies using different sample types was limited. Additionally, an in-house LAMP assay might be superior to the Loopamp MTBC assay. Because of its low cost, LAMP could be useful in the diagnosis of EPTB, particularly in areas where financial resources are limited and drug-resistant MTB is not prevalent.

Search strategy.

(DOCX)

Click here for additional data file.

Preferred Reporting Items Systematic Reviews Meta Analyses (PRISMA) 2009 flow diagram.

(DOC)

Click here for additional data file.

PRISMA 2009 checklist.

(DOC)

Click here for additional data file.

(DOCX)

Click here for additional data file.