Evaluation of RT-LAMP Assay for Rapid Detection of SARS-CoV-2.

OBJECTIVE: To evaluate the accuracy of the reverse transcription loop-mediated isothermal amplification (RT-LAMP) assay for rapid detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in community or primary-care settings.
METHOD: We systematically searched the Web of Science, Embase, PubMed, and Cochrane Library databases. We conducted quality evaluation using ReviewManager software (version 5.0). We then used MetaDisc software (version 1.4) and Stata software (version 12.0) to build forest plots, along with a Deeks funnel plot and a bivariate boxplot for analysis. RESULT: Overall, the sensitivity, specificity, and diagnostic odds ratio were 0.79, 0.97, and 328.18, respectively. The sensitivity for the subgroup with RNA extraction appeared to be higher, at 0.88 (0.86-0.90), compared to the subgroup without RNA extraction, at 0.50 (0.45-0.55), with no significant difference in specificity.
CONCLUSION: RT-LAMP assay exhibited high specificity regarding current SARS-CoV-2 infection. However, its overall sensitivity was relatively moderate. Extracting RNA was found to be beneficial in improving sensitivity.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the causative agent of coronavirus disease 2019 (COVID-19), poses the most severe threat to global public health in the past 100 years.[1] Currently, the reference standard for diagnosing COVID-19 is reverse transcription–quantitative polymerase chain reaction (RT-qPCR).[2] It enables the detection of viral RNA at very low levels, and is highly sensitive and especially reliable. However, although PCR-based tests can diagnose COVID-19 within a short period,[3] they still have a variety of limitations, such as the complex experimental conditions that are required, the demand for skilled technical personnel, and the high cost.[4]

Loop-mediated isothermal amplification (LAMP) is a technique that can perform highly specific, efficient, and rapid amplification of DNA under isothermal conditions. It has been used to detect viruses, bacteria, and fungi.[5] Over the years, reverse transcription loop-mediated isothermal amplification (RT-LAMP) has been developed to detect infectious diseases caused by RNA viruses.[6] The convenience, sensitivity, and low cost of this technology make RT-LAMP a promising candidate for rapid screening for COVID-19.[7]

Many factors contribute to the molecular detection of SARS-CoV-2, including the temporal variations of viral loads, the selection and handling of specimens, adequacy of RNA purification, and selection of the extraction-free assays or RNA extraction kit,[8,9] all of which can lead to false-positive and false-negative results. False-negative results in patients with COVID-19 are particularly harmful because they can result in delayed treatment and increased risk of transmission in severely ill patients.[10] Therefore, it is necessary to evaluate the diagnostic accuracy of the RT-LAMP assay for detecting SARS-COV-2 and the factors that affect the diagnostic accuracy, which is the purpose of this meta-analysis.

The primary outcomes were the overall sensitivity and specificity of the RT-LAMP assay. The secondary outcomes were specific sensitivity and specificity in the subgroup based on study or participant characteristics, including the need for RNA extraction, specimen type, and viral load.

Methods

Protocol and Registration

A protocol was predetermined in PROSPERO, for which the registration number was CRD42020212489 (https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42020212489). This research was conducted following Preferred Reporting Items for a Systematic Review and Meta-Analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA).[11]

Search Strategy

The investigators systematically searched the Embase, PubMed, Cochrane Library, and Web of Science databases, with the keywords COVID-19 (or SARS-CoV-2) and RT-LAMP assay (or LAMP or loop-mediated isothermal amplification) for studies before September 21, 2020, with no restrictions on geography or language.

Selection Criteria

The investigators systematically sifted through all the articles based on preestablished screening criteria. The inclusion criteria were as follows: specimens from the hospital or routine COVID-19 screening station, any test based on the RT-LAMP technology for detecting SARS-CoV-2 with results available within 2 hours of specimen collection, viral culture or RT-qPCR or RT-PCR (not quantitative) or next-generation sequencing (NGS) as the reference standard. The exclusion criteria were as follows: review articles, editorials, case reports, comments, and letters; small-scale studies with fewer than 10 specimens or participants; nonhuman specimens; and preprints from the medRxiv and bioRxiv servers.

Data Extraction

The 2 review authors extracted the data independently and resolved the differences through dialogue. A third review author was consulted when necessary. We collected information on the country; study design; specimen type; index test; RNA extraction or lack thereof; viral load (cycle threshold); and reference testing and true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN).

Assessment of Methodological Quality

The 2 researchers independently reviewed the methodological quality of the included articles, based on the quality assessment of the diagnostic accuracy studies (QUADAS-2), recommended by the Cochrane Collaboration. Any disagreements were resolved through discussion, and a third review author could be consulted if necessary. RevMan systematic review software helped us in collecting and putting together relevant data, and in building appropriate figures.

Statistical Analysis and Data Synthesis

Using Meta-DiSc software, version 1.4,[12] we calculated the sensitivity (SEN), specificity (SPE), positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR) with corresponding 95% CIs. Also, the area under the summary receiver operating characteristic (SROC) curve was used to evaluate the overall accuracy. Whereas studies presented only sensitivity estimates, we fitted univariate-random effects logistic regression models using Stata software, version 12.0 (StataCorp). A bivariate boxplot was constructed to perform heterogeneity testing. A Deeks funnel plot was constructed to evaluate potential publication bias visually.

Subgroup Analysis

Subgroup analysis was adopted to explore the potential sources of heterogeneity. Subgroup analysis was performed according to whether RNA extraction was needed (group A), specimen type (group B), and viral load (group C). Further, to synthesize the effects of RNA extraction and viral load, we performed a subgroup analysis of RNA extraction in high- and low-viral-load groups, respectively.

Results

Search Results

According to the selection criteria, a total of 258 publications were retrieved. After eliminating duplicates, 135 articles were left. After screening the abstracts, 73 articles were left. After reviewing the complete text, 51 articles were excluded. The exclusion reasons were shown in Figure 1. Finally, 22 qualified articles were included.

Figure 1.

Flow diagram of study identification and inclusion.

Characteristics of the Included Studies

We extracted 30 fourfold tables from the 22 included articles. Taking cycle threshold (CT) 30 as the cutoff value of the high and low viral loads group, there were 11 high and 10 low viral load fourfold tables. The characteristics of the studies included in the articles are shown in Table 1.[13-34]

Table 1.

Characteristics of the Studies Included

Author	Year	Study Design	Country	Specimen Type	Index Test	RNA Extraction	CT	Reference Standard	Result
									TP	FP	FN	TN
Zhu et al[34]	2020	Prospective	China	OP	RT-LAMP-LFB	Yes	NA	RT-qPCR	33	0	0	96
Yan et al[33]	2020	Prospective	China	Other	RT-LAMP	Yes	NA	RT-qPCR	58	0	0	72
Dao Thi et al[17]	2020	Prospective	Germany	P	RT-LAMP	Yes	0–40	RT-qPCR	83	2	34	649
Dao Thi et al[17]	2020	Prospective	Germany	P	Hot swab RT-LAMP	No	0–40	RT-qPCR	60	1	68	214
Dao Thi et al[17]	2020	Prospective	Germany	P	Direct swab RT-LAMP	No	0–40	RT-qPCR	26	9	56	144
Rohaim et al[32]	2020	Prospective	England	NP	AI-LAMP	Yes	NA	RT-qPCR	61	65	9	64
Rödel et al[31]	2020	Prospective	Germany	Other	Variplex RT-LAMP	No	NA	RT-qPCR	29	0	9	35
Nagura-Ikeda et al[30]	2020	Prospective	Japan	Other	RT-LAMP	Yes	NA	RT-qPCR	73	0	30	0
Mohon et al[29]	2020	Prospective	Canada	NP	RT-LAMP	Yes	NA	RT-PCR	65	0	1	58
Lu et al[28]	2020	Prospective	China	P	RT-LAMP	Yes	NA	RT-qPCR	34	2	2	18
Lee et al[27]	2020	Prospective	Australia	NP	N1-STOP-LAMP	Yes	NA	RT-qPCR	93	0	14	50
Lau et al[26]	2020	Prospective	Malaysia	NP	RT-LAMP	Yes	NA	RT-qPCR	47	0	0	42
Lamb et al[25]	2020	Prospective	United States	Other	RT-LAMP	Yes	NA	RT-qPCR	19	2	1	18
Lamb et al[25]	2020	Prospective	United States	NP	RT-LAMP	No	0–40	RT-qPCR	4	0	6	10
Klein et al[24]	2020	Prospective	Germany	P	Colorimetric RT-LAMP	Yes	0–40	RT-qPCR	33	0	19	29
Klein et al[24]	2020	Prospective	Germany	P	Fluorescent RT-LAMP	Yes	0–40	RT-qPCR	41	0	11	29
Kitagawa et al[23]	2020	Prospective	Japan	NP	RT-LAMP	Yes	25.31–36.08	RT-qPCR	30	2	0	44
Jiang et al[22]	2020	Prospective	China	Other	RT-LAMP	Yes	0–40	RT-qPCR	32	0	3	133
Jiang et al[22]	2020	Prospective	China	Other	RT-LAMP	Yes	0–40	RT-qPCR	10	1	2	79
Huang et al[21]	2020	Prospective	China	P	RT-LAMP	No	0–36	RT-qPCR	8	0	0	8
Hu et al[20]	2020	Prospective	China	NP	RT-LAMP	Yes	NA	NGS	31	3	4	291
Hu et al[20]	2020	Prospective	China	Other	RT-LAMP	Yes	NA	NGS	41	1	5	105
Ganguli et al[19]	2020	Prospective	United States	NP	RT-LAMP	No	20.00–30.00	RT-PCR	10	0	0	10
Eckel et al[18]	2020	Prospective	Germany	P	Variplex LAMP	No	NA	RT-PCR	8	7	39	55
Chow et al[16]	2020	Prospective	China	NP	RT-LAMP	Yes	NA	RT-qPCR	95	0	1	143
Chow et al[16]	2021	Prospective	China	Other	RT-LAMP	Yes	NA	RT-qPCR	65	0	2	143
Chow et al[16]	2022	Prospective	China	P	RT-LAMP	Yes	NA	RT-qPCR	59	0	1	143
Ben-Assa et al[15]	2020	Prospective	Israel	P	RT-LAMP	No	0–35	RT-qPCR	42	1	10	30
Baek et al[14]	2020	Prospective	South Korea	NP	RT-LAMP	Yes	21.11–32.76	RT-qPCR	14	2	0	138
Ali et al[13]	2020	Prospective	South Korea	NP	RT-LAMP-coupled CRISPR-Cas12	Yes	0–40	RT-qPCR	17	0	3	4

AI-LAMP, artificial intelligence–assisted rapid detection of color changes associated with the loop-mediated isothermal amplification reaction; CT, cycle threshold; FN, false negative; FP, false positive; LAMP, loop-mediated isothermal amplification; NA, nonapplicable; NGS, next-generation sequencing; NP, nasopharyngeal swab; N1-STOP-LAMP, N1 gene single-tube Optigene loop-mediated isothermal amplification assay; OP, oropharynx swab; P, pharyngeal swab (includes oropharynx swab and pharyngeal swab); RT-LAMP, reverse transcription loop-mediated isothermal amplification; RT-LAMP-LFB, reverse transcription loop-mediated isothermal amplification lateral flow biosensor; RT-PCR, reverse transcription polymerase chain reaction; RT-qPCR, reverse transcription quantitative polymerase chain reaction; TN, true negative; TP, true positive.

Methodological Quality

The overall quality of the 22 included studies is shown in Figure 2. In the patient selection domain, 7 studies (31.8%) were at high risk of bias, considering some studies were case control studies. A total of 9 studies (40.9%) were rated as “high risk” in the index test domain because many studies were usually unclear on whether the index tests were interpreted without knowledge of the results of the reference standard. In total, 3 studies (13.6%) were rated as “high risk” in the reference standard domain, and no study was judged to be a high bias risk in the flow and timing domain.

Figure 2.

Quality levels of the studies. A, Overall quality assessment of the included studies. B, Quality assessment of the individual studies.

Threshold Effect Analysis

As can be observed in the SROC curve (Figure 3A), there was no “shoulder-arm” distribution. Also, the Spearman correlation coefficient was −0.346 (<0.6), and the P value was .06 (>.05), thus not statistically significant. Therefore, we concluded that there was no threshold effect in the included studies.

Figure 3.

Forest plots for reverse transcription loop-mediated isothermal amplification assay for coronavirus disease 2019 of the summary receiver operating characteristic (SROC) curve (A), diagnostic odds ratio (B), sensitivity (C), specificity (D), positive likelihood ratio (E), and negative likelihood ratio (F). AUC, area under the curve.

Overall Accuracy of RT-LAMP Assay

To determine the diagnostic accuracy of the RT-LAMP assay, we used a random effects model in our research. As shown in Figures 3A and 3B, area under the curve (AUC) = 0.9863, the Q index = 0.9499 (SE = 0.0161), and the DOR was 328.18 (95% CI, 113.22–951.26). As shown in Figures 3C and 3D, the SEN was 79% (95% CI, 77%–81%), and the SPE was 97% (95% CI, 96%–97%). As shown in Figures 3E and 3F, the PLR was 36.13 (95% CI, 11.40–114.51), and the NLR was 0.13 (95% CI, 0.08–0.22), which indicates that RT-LAMP had relatively moderate sensitivity and high specificity in detecting the presence of SARS-CoV-2.

Heterogeneity Analysis of the Nonthreshold Effect

Quantitative indicators of heterogeneity were judged by the inconsistency index (I2), which was automatically generated by the Meta-DiSc software. The I2 was interpreted as follows: 0%–40%, might not be important; 30%–60%, moderate heterogeneity; 50%–90%, substantial heterogeneity; 75%–100%, considerable heterogeneity.[35] As shown in Figure 3, high heterogeneity was detected across studies: DOR (I2 = 86.6%), SEN (I2 = 93.9%), SPE (I2 = 92.8%), PLR (I2 = 95.5%), and NLR (I2 = 96.1%).

Subgroup analysis was performed according to whether RNA extraction was needed (group A), specimen type (group B), and viral load (group C). The results of subgroup meta-analyses were summarized in Table 2. According to the bivariate boxplot (Figure 4A), there were 4 sets of data outside the circle.

Table 2.

Results of Subgroup Analysis

Subgroup Analysis	No. of Studies	Sensitivity (95% CI)	I2	Specificity (95% CI)	I2
Group A
With RNA extraction	22	0.88 (0.86–0.90)	87.3%	0.97 (0.96–0.97)	94.5%
Without RNA extraction	8	0.50 (0.45–0.55)	92.4%	0.97 (0.95–0.98)	69.6%
Group B
Pharyngeal swab	22	0.76 (0.74–0.79)	94.9%	0.96 (0.95–0.97)	94.3%
Other	8	0.86 (0.82–0.90)	85.5%	0.99 (0.98–1.00)	45.5%
Group C
High viral load
Total	11	0.97 (0.95–0.99)	76.1%	NA	NA
With RNA extraction	6	1.00 (0.99–1.00)	0	NA	NA
Without RNA extraction	5	0.85 (0.73–0.97)	88.9%	NA	NA
Low viral load
Total	10	0.37 (0.19–0.55)	93.4%	NA	NA
With RNA extraction	6	0.51 (0.22–0.81)	93.8%	NA	NA
Without RNA extraction	4	0.09 (0.04–0.14)	0%	NA	NA

NA, not available.

Figure 4.

Testing for reverse transcription loop-mediated isothermal amplification assay. A, Bivariate boxplot to perform heterogeneity testing. B, Deeks funnel plot asymmetry test to assess publication bias.

Group A (regardless of whether RNA extraction was needed): With RNA extraction, the sensitivity was 88% (95% CI, 86%–90%; I2 = 87.3%), and the specificity was 97% (95% CI, 96%–97%; I2 = 94.5%). Without RNA extraction, the sensitivity was 50% (95% CI, 45%–55%; I2 = 92.4%), and the specificity was 97% (95% CI, 95%–98%; I2 = 69.6%).

Group B (by specimen type in the subgroup with RNA extraction): in the specimen with the pharyngeal swab, the sensitivity was 76% (95% CI, 74%–79%; I2 = 94.9%), and the specificity was 96% (95% CI, 95%–97%; I2 = 94.3%). In other specimen types, sensitivity was 86% (95% CI, 82%–90%; I2 = 85.5%), and the specificity was 99% (95% CI, 98%–100%; I2 = 45.5%).

Group C (by viral load): For the high viral load subgroup, the total sensitivity was 97% (95% CI, 95%–99%; I2 = 76.1%), and the sensitivity with RNA extraction was 100% (95% CI, 99%–100%; I2 = 0.0%); however, the sensitivity without RNA extraction was 85% (95% CI, 73%–97%; I2 = 88.9%). For the low viral load subgroup, the total sensitivity was 37% (95% CI, 19%–55%; I2 = 3.4%), and the sensitivity with RNA extraction was 51% (95% CI, 22%–81%; I2 = 93.8%); however, the sensitivity without RNA extraction was 9% (95% CI, 4%–14%; I2 = 0.0%).

Publication Bias

We used a Deeks funnel plot to assess the presence of any potential publication bias visually. Despite the fact that some points were not distributed symmetrically, as shown in Figure 4B, the P value of the Deeks test was .57, which indicated that there was no significant bias in this study. If the test results were P < .01, the publication bias test results were significant.

Discussion

The focus of our research was to evaluate the value of the RT-LAMP assay in the diagnosis of COVID-19. Before subgroup analysis, the SEN, SPE, PLR, NLR, and DOR were 0.79, 0.97, 36.13, 0.13, and 328.18, respectively. The SROC AUC was 0.9863 (close to 1.000), which indicated the moderate sensitivity and high specificity of this technique in identifying COVID-19. To some extent, our study supported the conclusions of Mustafa Hellou et al,[36] who reported that RT-LAMP or isothermal assays had a sensitivity of 84.2% (95% CI, 75.5%–90.5%) and specificity of 97.7% (95% CI, 92.8%–99.3%) in detecting SARS-CoV-2.

Also, we performed a subgroup analysis to investigate the sources of heterogeneity of the included articles. When subgroup analysis was performed directly based on RNA extraction, specimen type, and viral load, the heterogeneity of sensitivity was slightly reduced but still high. We discovered that the heterogeneity of sensitivity was significantly reduced to 0 when RNA was extracted with high viral load and not extracted with low viral load. The pooled sensitivity for the subgroup with RNA extraction appeared to be higher, at 88% (95% CI, 86%–90%), compared with the subgroup without RNA extraction, at 50% (95% CI, 45%–55%), with little change in the specificity. These findings indicated that extracted RNA might affect the sensitivity of the test. Pan et al[37] also reported that extraction-free techniques are easier to perform and are more amenable for point-of-care testing (POCT), but the release of RNA by preheating the specimen may reduce the sensitivity of detection of specimens containing low viral load. The average sensitivity of the non-RNA extraction group was 50% and varied from 45%–55%. The reasons for the significant difference between the studies are unclear and may be due to characteristics of the studies themselves. More data are needed to conclude that not extracting RNA will certainly affect the detection accuracy.

In addition, we performed a subgroup analysis on the specimen collection types. We found no significant differences in sensitivity or specificity between the subgroup of pharyngeal swabs and the subgroup of the other specimen collection types, indicating that the specimen collection type is not the cause of heterogeneity. We then conducted subgroup analysis by viral load. The CT threshold for high viral loads was 30 or less.[38] On average, the sensitivity of low viral loads was very poor, at 37% (95% CI, 19%–55%), compared with that of the high viral load group, at 97% (95% CI, 95%–99%), with little change in the specificity. We observed significant differences in sensitivity according to viral load and suspected that differences in the distribution of specimens with high and low viral load between studies may have affected overall accuracy estimates. However, the findings of 2 studies remained high sensitivity, with regard to low viral loads. By analyzing the research design and test method of each of those manuscripts, we speculated that the findings might be related to the small positive specimen size[22] and the combined test method.[13] More data are needed to determine whether test performance can be repeated at low viral loads.

Along with the aforementioned 3 sources, some other sources of heterogeneity are still being considered, such as patient symptom severity, time from symptom onset to index test, the target gene of the LAMP assay, and whether other detection methods and reference standards are being used. Only a few studies have reported patient clinical background information. Nagura-Ikeda et al[30] divided the specimens according to the time from symptom appearance to detection into early phase, late phase, and no specific time (asymptomatic), and the respective sensitivities in these categories were 85.2%, 44.4%, and 60.0%.

Viral load varies depending on the time of infection and severity of symptoms, contributing to false-negative results in asymptomatic patients.[39] Due to lack of relevant reporting, the LAMP assay detection effect may be different in different prevalence settings. As to the target gene for the LAMP assay, some studies only use the N, RdRP, or M gene as the target gene; others combined 1 of those genes with 2 or more genes, among which the N gene was used as the target gene in most studies. Corman et al[40] suggest that the N gene is one of the best targets for high-sensitivity detection of SARS-CoV-2. Because there were few studies on other genes as the target gene, we did not conduct subgroup analysis in this regard.

For the studies that combined another detection method with the primary one studied, one using AI-LAMP (artificial intelligence–assisted rapid detection of color changes associated with the LAMP reaction) has detected the SARS-CoV-2 RNA in specimens that had tested negative via qRT-PCR,[32] the other using coupled CRISPR-Cas12 maintained high accuracy in detecting low viral load.[13] Except for the studies by Hu et al,[20] all studies used RT-qPCR or RT-PCR only as the reference standard for diagnosing SARS-CoV-2 infection.

This study has several limitations. The weaknesses of the review primarily reflect the shortcomings in the primary studies and their reporting. Many studies omitted descriptions of the participants and the key aspects of the study design and execution. We grouped the viral loads only for studies that provided the CT threshold. We acknowledged that using CT values as a surrogate for viral load is inappropriate, when we discussed viral load in the setting of symptomatic and asymptomatic infection. Further prospective and comparative evaluation of RT-LAMP tests, especially in clinically relevant settings, is urgently needed.

Conclusion

In this article, RT-LAMP assay exhibited high specificity with regards to SARS-CoV-2 infection. However, its overall sensitivity was relatively moderate. Moreover, extracting RNA was found to be beneficial in improving sensitivity. It showed high sensitivity in high viral load specimens, and it may be used for rapid detection and separation of the greatest number of patients with positive results. However, more clinical data are needed to support these conclusions.