Diagnostic accuracy of LAMP versus PCR over the course of SARS-CoV-2 infection.

OBJECTIVE: Reverse transcription loop-mediated isothermal amplification (RT-LAMP) has been validated for diagnosis of several viral infections. However, its diagnostic accuracy in detecting SARS-CoV-2 in real-life clinical settings remains unclear. The aim of this study was to determine the diagnostic sensitivity and specificity of RT-LAMP compared to reverse transcription-quantitative polymerase chain reaction (RT-qPCR) over the disease course of COVID-19.
METHODS: A total of 124 nasopharyngeal swab samples obtained from 24 COVID-19 patients were tested by RT-LAMP and RT-qPCR. Sensitivities and specificities of RT-LAMP compared with RT-qPCR were analyzed as a function of time from onset.
RESULTS: Up to the 9th day after onset, the RT-LAMP had positivity of 92.8%, and the sensitivity and specificity compared with RT-qPCR was 100%. After the 10th day of onset, however, the positivity of RT-LAMP decreased to less than 25%, and concordance of positivity between the two methods was below 60%. The limit of detection of RT-LAMP was 6.7 copies/reaction.
CONCLUSIONS: Until the 9th day after the onset of symptoms, RT-LAMP had the same diagnostic accuracy as RT-qPCR. These finding suggest that RT-LAMP can be used as a diagnostic tool for COVID-19 as an alternative to RT-qPCR in the acute symptomatic phase of COVID-19.

Introduction

The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was first reported in Wuhan City, Hubei Province, China, at the end of 2019 and has since caused a major pandemic (Lu et al., 2020, Wu et al., 2020). To date, several drugs, including remdesivir, favipiravir, and dexamethasone, have been reported to be effective in treating coronavirus disease 2019 (COVID-19), but the COVID-19 pandemic has been accelerating around the world (Beigel et al., 2020, Sterne et al., 2020). With over 70 million confirmed cases and 1,500,000 deaths, the world faces an unprecedented economic, social, and health impact. Rapid, sensitive, accurate, and versatile diagnostic methods are essential tools in curtailing the spread of the virus.

Currently, positive nucleic acid amplification tests (NAATs) for SARS-CoV-2 using a nasopharyngeal swab, lower respiratory tract specimen, or saliva are primarily used to diagnose COVID-19 (Altawalah et al., 2020, Wang et al., 2020, Bwire et al., 2021). The gold standard for molecular diagnosis of COVID-19 is the detection of SARS-CoV-2 RNA by real-time quantitative reverse transcription-polymerase chain reaction (RT-qPCR) (Corman et al., 2020). This assay is reported to have relatively high sensitivity, however it has several disadvantages, such as requiring capital investment, highly trained technicians, and upwards of several hours required to process the tests (Shen et al., 2020). Therefore, it can be challenging to perform RT-PCR for the diagnosis of COVID-19 in hospitals and clinics.

Recently, the reverse transcription loop-mediated isothermal amplification (RT-LAMP) method has been developed and applied as a point of care test (POCTs) for the diagnosis of several viral infections (Silva et al., 2019, Lopez-Jimena et al., 2018). In contrast to RT-PCR, this method has the advantage of amplifying genes at a constant temperature of approximately 65 °C and is relatively easy to operate (Huang et al., 2020). In addition, this method can produce results in less than 30 min. To date, several in vitro studies demonstrated that it could detect low copies of SARS-CoV-2 RNA (Ganguli et al., 2020, Augustine et al., 2020, Dao Thi et al., 2020). Therefore, RT-LAMP holds promise as a POCT for detecting SARS-CoV-2. However, it is still unclear whether the method has the same diagnostic accuracy as RT-PCR in detecting SARS-CoV-2 in real-life clinical settings. Hence, we aimed to determine the diagnostic sensitivity and specificity of RT-LAMP compared to RT-PCR over the disease course of COVID-19.

Methods

Clinical specimens

A total of 124 nasopharyngeal swab samples obtained from 24 COVID-19 patients who were admitted to a university hospital in Japan from March 1 to April 30, 2020, were analyzed. Severity classifications were made according to the National Institutes of Health COVID-19 Treatment Guidelines (https://www.covid19treatmentguidelines.nih.gov). Swab samples were collected using a flocked sterile plastic swab applicator and placed in 3 mL of BD universal viral transport medium (Becton, Dickinson and Company, Franklin Lakes, NJ, USA). RNA was extracted from the swab samples immediately. This study was approved by the institutional review board of Fujita Health University (No. HM19-493).

RNA extraction

According to the manufacturer’s instructions, total viral RNA extraction was performed with the QIAamp Viral RNA Mini Kit (Qiagen, Hilden, Germany) using an automated nucleic acid extraction device, QIAcube. The extracted RNA was eluted in 60 μL of buffer AVE and stored at −80 °C.

RT-LAMP reaction

According to the manufacturer’s instructions, RT-LAMP was performed using the Loopamp SARS-CoV-2 Detection kit (Eiken Chemical Co., Ltd., Tokyo, Japan). The reaction volume of 25 μL contained 10 μL of purified RNA and 15 μL of reaction mix containing 172.3 ng/μL of SARS-CoV-2 specific primer sets. The mixture was incubated for 35 min at 62.5 °C, and the process was monitored using a Loopamp Real-time Turbidimeter (LA-200; Eiken Chemical). For visual evaluation of fluorescence, the reaction tube was illuminated with ultraviolet light using an ultraviolet illumination system (WSE-5300; ATTO, Tokyo, Japan) and observed by the naked eye.

Quantitative reverse transcription PCR assay (RT-qPCR)

To evaluate the genomic amount of SARS-CoV-2 RNA, one-step RT-qPCR was performed on QuantStudio 1 Real-Time PCR System (QS-1) (Thermo Fisher Scientific, Waltham, MA, USA) using TaqMan Fast Virus 1-Step Master Mix (Thermo Fisher). Primers and probes were as follows: NIID_2019-nCOV_N_F2, 5′-AAATTTTGGGGACCAGGAAC-3′; NIID_2019-nCOV_N_R2, 5′- TGGCAGCTGTGTAGGTCAAC-3′; NIID_2019-nCOV_N_P2, 5′- FAM ATGTCGCGCATTGGCATGGA BHQ-3′ (Shirato et al., 2020). Reverse transcription and amplification were performed according to the manufacturer’s instructions. PCR conditions were as follows: reverse transcription at 50 °C for five min; enzyme activation at 95 °C for 20 s; and 45 cycles of denaturation at 95 °C for 15 s and primer annealing/extension/fluorescence emission at 60 °C for 60 s. The real-time RT-PCR reaction mixture (20 μL total volume) contained 5.0 μL of 4× Fast Virus Master Mix, 1.0 μL of primer-probe pre-mix, 5.0 μL of template RNA, and nuclease-free water. Genomic numbers of SARS-CoV-2 RNA were calculated by a standard curve. Ten-fold serial dilutions ranging from 1 × 10° to 1 × 104 copies/μL of SARS-CoV-2 positive control RNA (NIHON GENE RESEARCH LABORATORIES Inc) were tested in duplicate in RT-qPCR to construct the standard curve.

Clinical sensitivity and specificity of RT-LAMP compared to RT-qPCR

The clinical samples used in this study were classified into four groups: Group 1 with samples collected by the 9th day from symptom onset, Group 2 with those collected from the 10th to 19th days after onset, Group 3 with ones collected between the 20th and 29th days after onset, and Group 4 with those collected 30 days or more after onset. The samples were tested by both the RT-LAMP method and RT-qPCR in singlicate per sample to detect SARS-CoV-2.

Relationship between genomic copy number of SARS-CoV-2 RNA and RT-LAMP positivity

124 clinical samples obtained from 24 COVID-19 patients were tested by RT-qPCR for SARS-CoV-2 RNA quantitation. Additionally, RT-LAMP was performed using the positive samples tested by RT-qPCR. These assays were performed in singlicate per sample. To determine the positivity rate of RT-LAMP for a different viral load of SARS-CoV-2 and a limit of detection (LOD) of RT-LAMP with clinical samples, the relationship between genomic copy number of SARS-CoV-2 RNA and RT-LAMP positivity was evaluated.

Data analysis

Two-by-two tables were established, and analytical performance characteristics with 95% confidence intervals were calculated for sensitivity and specificity of RT-LAMP compared to RT-qPCR. Cohen’s kappa (κ) was calculated as a measure of agreement with RT-qPCR. Concordance between both tests was assessed using the κ coefficient. κ values below 0.40 indicate weak correlation, values of 0.41–0.60 indicate good agreement, and values above 0.60 indicate strong agreement (Landis and Koch, 1977). Fisher’s exact test was used to calculate P values to compare positivity rates of RT-LAMP and RT-qPCR in each group. Genomic copy numbers and Ct values in each group were compared using the Kruskal–Wallis test. In addition, the genomic copy numbers for positive and negative RT-LAMP test results were compared using the Mann–Whitney U test. A P value <0.05 was considered statistically significant. All statistical analyses were performed with EZR (Saitama Medical Center, Jichi Medical University, Saitama, Japan), a graphical user interface of R designed for use in biostatistics (The R Foundation for Statistical Computing Vienna, Austria).

Results

Patient characteristics

Patient characteristics in this study are shown in Table 1 . Of 24 COVID-19 patients included in this study, the median age was 37 years (IQR, 26.5–47.5), with 15 males and nine females. The most common comorbidity was diabetes, followed by hypertension. Nine, 14, and one of the patients had mild, moderate, and severe diseases, respectively. The median interval between symptom onset and the day of first PCR testing was ten days (IQR, 8–12). The median length of hospital stay was 21.5 days (IQR, 18.3–25.0). Twelve patients received favipiravir. Twenty-two patients were discharged home after their symptoms improved, and negative RT-qPCR test results were obtained from at least two nasopharyngeal specimens collected ≥24 h apart. The remaining two patients were transferred to an isolation facility pending documentation of two negative PCR test results.

Table 1

Patient characteristics in this study.

Patient characteristics	N (%) or median (IQR)
Median Age, years (interquartile range)	37 (26.5–47.5)
Median interval between symptom onset to the day of the first PCR test, days (interquartile range)	10 (8–12)
Sex
Female	9 (37.5%)
Male	15 (62.5%)
Comorbidities
Diabetes	4 (16.7%)
Hypertension	3 (12.5%)
Cancer	1 (4.2%)
Cardiovascular disease	2 (8.3%)
Cerebrovascular disease	2 (8.3%)
Severity
Mild	9 (25%)
Moderate	14 (67%)
Severe	1 (8%)
Antiviral drug
Favipiravir	12 (50%)
None	12 (50%)
Length of hospital stay
Among those discharged (N = 22)	21.5 (18.3–25.0)
Transfer to a different hospital (N = 2)	33 and 17

RNA quantification by RT-qPCR

Genomic copy numbers of SARS-CoV-2 RNA measured by RT-qPCR in each group are shown in Figure 1 . The median RNA copy number of all positive samples was 5.7 copies/reaction (IQR, 2.3–91.4). The RNA copy numbers of the positive samples in Group 1 were statistically higher than those in the other Groups, with a median of 243.7 copies/reaction (IQR, 68.7–946.9). The median RNA copy numbers ​​of the samples in Groups 2 to 4 were 3.1 copies/reaction (IQR, 1.2–20.6), 9.2 copies/reaction (IQR, 3.8–27.0), and 2.4 copies/reaction (IQR, 1.6–5.8), respectively. In addition, the Ct values determined by RT-qPCR in each group are shown in Supplementary Figure S1.

Figure 1

Genomic copy numbers of SARS-CoV-2 RNA determined by RT-qPCR in Groups 1–4. The box indicates the 25th and 75th percentiles. The thin line within the box marks the median. * Statistical significance (P < 0.05).

Comparison of RT-LAMP with RT-qPCR

The results of 124 samples tested by RT-LAMP and RT-qPCR are shown in Table 2 . In all RT-LAMP assays, the reaction tube illuminated with ultraviolet light using an ultraviolet illumination system was evaluated by the naked eye (Figure 2 ). Positivity rates of RT-qPCR and RT-LAMP were 50.0% (62/124) and 29.0% (36/124), respectively. The sensitivity of RT-LAMP was 56.6% (with 95% confidence intervals [CIs], 43.3–69.0%), and its specificity was 98.4% (95% CI, 91.3–100.0%). Figure 3 illustrates the positivity rates by group. The positivity rate of Group 1 was 92.8% in both the RT-LAMP and RT-qPCR, which was significantly higher than those in Groups 2–4. On the other hand, the positivity rate of RT-LAMP in Groups 2–4 was significantly lower than that of RT-qPCR in Groups 2–4, but there were no significant differences in Groups 3 and 4. The results of the sensitivity and specificity of RT-LAMP compared with RT-qPCR in each group are shown in Table 3 . In Group 1, the sensitivity of RT-LAMP compared with RT-qPCR was 100% (95% CI, 66.1–100%), whereas those in Groups 2–4 were below 55%. The specificity was 100% in Groups 1, 3, 4. Only 1 sample in Group 2 yielded a false-positive result with RT-LAMP.

Table 2

Overall comparison between RT-LAMP and RT-qPCR for detection of SARS-CoV-2.

LAMP result	No. of samples with RT-qPCR result			Sensitivity (95% CI)	Specificity (95% CI)	Kappa
	Positive	Negative	Total
Positive	35	1	36	56.6 (43.3–69.0)	98.4 (91.3–100.0)	0.55
Negative	27	61	88
Total	62	62	124

Figure 2

Ultraviolet light detection at the end of the assay. Light gray indicates a positive reaction (+), and dark gray indicates a negative reaction (−).

Figure 3

Positivity rates of the RT-LAMP and RT-qPCR in each group of samples collected from patients with a confirmed diagnosis of COVID-19. Black bar, RT-qPCR; gray bar, LAMP methods. * Statistical significance (P < 0.05).

Table 3

Comparison between RT-LAMP and RT-qPCR for detection of SARS-CoV-2 in each group. A: Group 1, B: Group 2, C: Group 3, D: Group 4.

A	LAMP result	No. of samples with RT-qPCR result			Sensitivity (95% CI)	Specificity (95% CI)	Kappa
		Positive	Negative	Total
	Positive	13	0	13	100.0 (94.2–100.0)	100.0 (24.7–100.0)	1.00
	Negative	0	1	1
	Total	13	1	14

Positivity rate of RT-LAMP for different genomic copy number of SARS-CoV-2 RNA

Of the 124 samples, 62 were RT-qPCR for positive, which contained 0.4 to 94,749.3 SARS-CoV-2 RNA copies/reaction (Figure 4 ). The number of samples containing <1.0 × 101 SARS-CoV-2 RNA copies/reaction, 1.0 × 101 to <1.0 × 102 SARS-CoV-2 RNA copies/reaction, 1.0 × 102 to <1.0 × 103 SARS-CoV-2 RNA copies/reaction, 1.0 × 103 to <1.0 × 104 SARS-CoV-2 RNA copies/reaction and ≥ 1.0 × 104 SARS-CoV-2 RNA copies/reaction were 32, twelve, ten, four and four, respectively. Positivity rates of RT-LAMP for each genomic copy number of SARS-CoV-2 RNA are shown in Figure 5 . 18/18 (100%) samples with SARS-CoV-2 RNA copy numbers greater than 1.0 × 102 copies/reaction were detected by RT-LAMP, whereas the positivity rate of RT-LAMP for samples containing under 1.0 × 101 SARS-CoV-2 RNA copies/reaction was much lower at 21.9%. The positivity rate of RT-LAMP for samples containing 1.0 × 101 to 1.0 × 102 SARS-CoV-2 RNA copies/reaction was 91.7%.

Figure 4

SARS-CoV-2 viral load of positive samples tested with RT-qPCR.

Figure 5

Positivity rate of RT-LAMP for each genomic copy number of SARS-CoV-2 RNA determined by RT-qPCR. * Statistical significance (P < 0.05).

Limit of detection for RT-LAMP

The difference in genomic copy numbers of SARS-CoV-2 RNA determined by RT-qPCR between positive and negative RT-LAMP test results is shown in Figure 6 . The genomic copy numbers for positive RT-LAMP test results were statistically higher than those for negative RT-LAMP test results, with a median of 111.3 copies/reaction (IQR, 19.6–620.4) and 2.1 copies/reaction (IQR, 1.1–3.1), respectively. The limit of detection for RT-LAMP determined using the 62 RT-qPCR-positive samples was 6.7 copies/reaction.

Figure 6

SARS-CoV-2 viral load of positive samples tested with RT-qPCR for positive and negative RT-LAMP test results. * Statistical significance (P < 0.05).

Discussion

In the present study, we analyzed the sensitivity and specificity of RT-LAMP compared with RT-qPCR stratified by time from disease onset of COVID-19. Until the 9th day after the onset of symptoms, the sensitivity and specificity of RT-LAMP were 100%. This suggests that RT-LAMP has the same diagnostic accuracy as RT-qPCR in the acute phase of infection.

To date, several studies have reported high sensitivity of RT-LAMP in comparison with RT-qPCR in detecting SARS-CoV-2 from clinical respiratory samples (Jiang et al., 2020, Kashir and Yaqinuddin, 2020, Lee et al., 2020). Franklin et al. reported that, of 223 respiratory samples positive for SARS-CoV-2 by RT-qPCR, 212 and 219 were positive by COVID-19 RT-LAMP with a reaction time of 60 and 90 min (sensitivities of 95.07% and 98.21%), respectively (Chow et al., 2020). However, it was unclear how the diagnostic performance of LAMP would change over time from symptom onset in real-life clinical settings since most previous studies did not consider the infection phase of patients from whom the samples were collected. This study demonstrated that the LAMP method had a high positivity rate of 92.8%, similar to the RT-qPCR method up to the 9th day of symptom onset among patients with PCR-confirmed COVID-19. However, after the 10th day of onset, the sensitivity of RT-LAMP decreased considerably, resulting in a positive result agreement between the two methods of 60% or less. In the present study, genomic copy numbers of SARS-CoV-2 RNA up to the 9th day of onset were predominantly higher than those after the 10th day of onset. In addition, genomic copy numbers of SARS-CoV-2 RNA for negative RT-LAMP test results were statistically lower than those for positive RT-LAMP test results. Therefore, low genomic copy numbers of SARS-CoV-2 after the 10th day of onset likely contributed to the lower performance of RT-LAMP in the late phase of onset. RT-LAMP can be used as a diagnostic tool for COVID-19 as an alternative to RT-qPCR in the acute symptomatic phase of COVID-19, but it is not suitable for patients presenting late in the course of illness or to document clearance of SARS-CoV-2 among those who had a positive test previously.

In the present study, we investigated how the diagnostic accuracy of RT-LAMP compared with RT-qPCR changed at intervals of ten days. The reasons for choosing the 10-day interval are as follows. First, the median interval between symptom onset and the day of first PCR testing of the 24 COVID-19 patients whose samples were collected in this study was ten days. Second, a systematic review by Mallet et al. reported that the positivity rate of RT-qPCR decreased significantly after ten days from onset (Mallett et al., 2020), but it is unclear how the positive rate of RT-LAMP changes after ten days from onset. Therefore, we first compared the diagnostic sensitivity and specificity of LAMP and RT-qPCR during the first ten days after onset and then investigated the diagnostic accuracy of both methods at 10-day intervals after ten days from onset.

The limit of detection of RT-LAMP using the Loopamp SARS-CoV-2 Detection kit was 6.7 copies/reaction, and the positivity rate of RT-LAMP was 29.0%. Previous studies reported detection limits of RT-LAMP ranging from 1.0 × 101 to 1.0 × 102 copies/reaction, indicating that this RT-LAMP test kit has high sensitivity (Park et al., 2020, Yan et al., 2020). However, when the RNA copy numbers were 1.0 × 101 copies/reaction or less, the positivity rate of RT-LAMP was as low as 21.9%. Therefore, as previously reported, samples with a low number of RNA copies can yield false-negative results by RT-LAMP.

In this study, we have not evaluated the analytical specificity of the RT-LAMP method. This is because the analytical sensitivity of the RT-LAMP method using the Loopamp SARS-CoV-2 Detection kit used in this study has already been validated in our previous publication (Higashimoto et al., 2020). Briefly, no LAMP product was detected in reactions performed with RNA from 22 viral genomes, including SARS coronavirus, Middle East Respiratory Syndrome (MERS) coronavirus, other human coronaviruses, influenza viruses, and respiratory syncytial viruses associated with respiratory infections. In addition, the analytical sensitivity and specificity of the RT-qPCR method used in this study were also sufficiently validated by Shirato et al. (2020). Therefore, we also did not perform these analyses.

This study has several limitations. First, the results of this study are based on the data from a single-center, and the number of samples is relatively small. In addition, this study includes only one severe case. Studies that include multiple centers and large numbers of patients with a broader spectrum of disease would be required to fully understand the clinical utility of RT-LAMP for SARS-CoV-2. Second, we did not examin the quantity and quality of RNA extracted from the sample. Third, we have conducted the RT-LAMP and RT-qPCR assays in singlicate per sample. Since it is necessary to test a large number of samples at one time in real-life clinical settings, the analysis of the quantity and quality of RNA has not typically been performed, and RT-qPCR and RT-LAMP have usually been conducted in singlicate per sample. In the present study, we have aimed to evaluate the diagnostic accuracy of RT-LAMP compared with RT-qPCR in actual clinical settings. Therefore, we have not examined the quantity and quality of RNA and have performed these assays in singlicate per sample as in actual clinical practice. Therefore, the accuracy of the results may be slightly inferior compared with a more controlled, experimental environment.

In summary, RT-LAMP has a sensitivity comparable to RT-qPCR in detecting COVID-19 in the acute phase of the illness and can be considered an alternative diagnostic tool to RT-qPCR in hospitals and clinics where the latter is not feasible on site.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflict of interest

All authors declare no conflict of interest with regard to this work.

Ethical approval

This study was approved by the institutional review board of Fujita Health University (No. HM19-493).