The screening value of RT-LAMP and RT-PCR in the diagnosis of COVID-19: systematic review and meta-analysis.

The purpose of this systematic review is to evaluate the test accuracy of reverse-transcription loop-mediated isothermal amplification (RT-LAMP) and reverse transcription-PCR (RT-PCR) for the diagnosis of coronavirus disease 2019 (COVID-19). We comprehensively searched PUBMED, Web of Science, the Cochrane Library, the Chinese National Knowledge Infrastructure, and the Chinese Biomedical Literature Service System until September 1, 2021. We included clinical studies assessing the sensitivity and specificity of RT-PCR and RT-LAMP using respiratory samples. Thirty-three studies were included with 9360 suspected cases of SARS-CoV-2 infection. The RT-PCR or other comprehensive diagnostic method was defined as the reference method. The results showed that the overall pooled sensitivity of RT-PCR and RT-LAMP was 0.96 (95 % CI, 0.93-0.98) and 0.92 (95 % CI, 0.85-0.96), respectively. RT-PCR and RT-LAMP had a 0.06 (95 % CI, 0.04-0.08) and 0.12 (95 % CI, 0.06-0.16) false-negative rates (FNR), respectively. Moreover, subgroup analysis showed mixed sampling and multiple target gene diagnosis methods had better diagnostic value than single-site sampling and a single target gene. The sensitivity and FNR were also significantly affected by the reference method. Comparing RT-LAMP with established suboptimal RT-PCR may exaggerate the performance of RT-LAMP. RT-PCR and RT-LAMP showed high values in the diagnosis of COVID-19, but there was still a FNR of about 6%-12%.

Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is the causative agent of the coronavirus disease 2019 (CoVID19) pandemic. The control and management of this pandemic have faced unprecedented challenges (Wiersinga et al., 2020). Isolating infected individuals, contact tracing, extensive diagnostic testing, and vaccinations are crucial to limit this infection (Bosetti et al., 2021). Accurate diagnosis is the basis to ensure the effective implementation of the above strategies.

Reverse transcription-PCR (RT-PCR) and reverse-transcription loop-mediated isothermal amplification (RT-LAMP) are two widespread testing methods, and each has distinct advantages. Evidence to date indicates that RT–PCR is considered the most sensitive method for the detection and quantification of SARS−COV-2 nucleic acid (Walsh et al., 2020). RT-LAMP is a reliable and rapid screening test, which can be used in the field or under non-laboratory conditions(Ganguli, 2020). However, there are analytical and interpretation issues with the molecular detection results of SARS−COV-2 infection. The false negative results of molecular tests would need detailed analysis (Bohn et al., 2020; Xiao et al., 2020).

Most clinical studies assessed agreement between different nucleic-acid amplification tests (NAATs) to evaluate the diagnostic accuracy. Generally, the existing PCR is the reference standard, and the new method is the index test. Although PCR is the current widely accepted reference test for detecting SARS−COV-2, it does not have excellent sensitivity and specificity (Ducray et al., 2020; Ridgway et al., 2020; Zhifeng et al., 2020). Therefore, a meta-analysis based on clinical research should consider the influence of references on the analysis and interpretation of results.

At present, some systematic analyses have evaluated the diagnostic accuracy of nucleic acid detection methods in COVID-19 diagnosis, but the sensitivity observed in various studies is inconsistent. A recent meta-analysis showed that the sensitivity of RT-PCR for detection of SARS−COV-2 in nasopharyngeal swabs (NPS) and sputum samples was 73.3 % and 97.2 %, respectively (Böger et al., 2021). Mustafa et al. found that the pooled sensitivity of NAATs evaluating SARS−COV-2 was 90.4 %. Furthermore, they argue that since there is no gold standard for diagnosing coronavirus infection, any existing NAATs can be used as a reference standard (Mustafa Hellou et al., 2021). A systematic review showed that the sensitivity of RT-LAMP was 94 % in purified RNA from COVID-19 patient samples and 78 % in crude samples. And RT-LAMP was prone to false-negative results in low viral load samples (Subsoontorn et al., 2020). Although these studies reported the performance of molecular detection in SARS−COV-2 infection, few of them consider the impact of inconsistent reference standards on the results. The reference method is essential for evaluating the performance of the detection method. Comparing the new suboptimal analysis with the established but suboptimal analysis may lead to untrue conclusions about the performance of the new way (Bohn et al., 2020).

Moreover, in these meta-analyses mentioned above, few studies have explored the significance of false-negative rate (FNR) in diagnosing SARS−COV-2 infection in detail. False-positive can result in unnecessary case isolation and further testing. However, FNR may present delays with prompt isolation and treatment, leading to the spread of infection (Xiao et al., 2020). The FNR might be related to the current outbreaks growing and epidemic rebound of SARS−COV-2. Taking the epidemic situation in Shijiazhuang, China, as an example, as of January 23, 2021, three rounds of mass testing have been carried out, with a total of more than 30 million tests in 17 days. Patients with a positive result in the three tests were 354, 247, and 30, respectively. Eighty percent of them are from quarantine sites (Li, 2021). Besides the low viral load and extended incubation period, the FNR has become another important factor of concern (Kucirka et al., 2020; Walsh et al., 2020). Therefore, to put nucleic acid testing in perspective, clearly understanding the diagnostic accuracy of different detection methods is essential for diagnosis, prevention, and control of this pandemic.

This study evaluated the diagnostic accuracy of RT-PCR and RT-LAMP and the potential effects of reference methods on diagnostic accuracy. Detailed subgroup analyses, FNR, and false positive rates (FPR) were included in this meta-analysis.

Method

Search strategy

We searched PubMed, the Cochrane Library, Web of Science, China National Knowledge Infrastructure (CNKI), and the Chinese Bio-Medical Literature Service System (Sino Med) without limits of language (the literature time limit is updated September 1; 2021). The search strategy included the following search string: “(COVID-19 OR novel coronavirus disease OR SARS−COV-2) AND (laboratory molecular test OR nucleic acid amplification tests OR RT-PCR OR RT-LAMP) AND (diagnostic)”. We performed manual searches in the references lists.

Inclusion and exclusion criteria

We included clinical studies assessing the diagnostic accuracy of RT-PCR and RT-LAMP using respiratory samples. Sensitivity (Se) and specificity (Sp) were quantitatively reported. We included both case-control and prospective cohort studies published until September 1, 2021. Animal or in-vitro studies and case reports were excluded.

Included: (i) Diseases: COVID-19 infection or suspected cases; (ii) Type of study: Diagnostic accuracy test or comparsion between different nucleic acid assay; prospective study or retrospective study; (iii) Index test methods: RT-LAMP and RT-PCR; (iv) Reference standard: RT-PCR (currently recognized as the gold standard for detection) or the other methods (such as clinical diagnosis, sequencing, a combined result of different RT-PCR assays.

Excluded: (i) Diagnostic tests for other diseases; (ii) The accuracy study of chest computed tomography (CT), serum antibody detection, and other non-nucleic acid detection methods in COVID-19 diagnosis; (iii) Case reports, in vitro cell experiments, and animal experiments; (iv) incomplete data or the inability to extract relevant data; (v) the studies that are missing in patient selection and reference method reporting were excluded.

Data collection and quality assessment

The Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool was used to assess the quality, the potential for bias, and the applicability of each selected study. Two review authors (DS and YB) independently screened studies and resolved disagreements with a third review author (YH). The QUADAS-2 tool evaluates four domains of bias: patient selection, index test, reference test, and flow and timing of testing. Studies considered to have a high risk of bias were not included in the meta-analysis. Two review authors (RP and YY) independently extracted study characteristics and 2 × 2 diagnostic-accuracy table data, which were checked by two authors (LM and SL).

Statistical analyses

Spearman correlation coefficient between the logarithm of sensitivity (Se) and the logarithm of (1-Sp) was calculated to detect the threshold effect. Generally, a strong positive correlation indicates the existence of a threshold effect (Leeflang, 2014). Deek's Funnel graph method was used to detect publication bias, and the result was determined by funnel plot and P value of the slope coefficient.

We performed an analysis of diagnostic trials based on the extracted data of true positive (TP), false positive (FP), false negative (FN), true negative (TN). The bivariate random-effects model of Midas command of Stata 12.0 software was used. The diagnostic performance of RT-PCR and RT-LAMP was shown in terms of summary sensitivity (Se), specificity (Sp), likelihood ratio (LR), and summary receiver operating curve (SROC), with a 95 % confidence interval (CI). The results were visually represented using forest plots.

The FNR and FPR were calculated according to the formula:FNR =

We obtained summary estimates of FNR and FPR by using random-effects meta-analysis. SROC curves were built to describe the relationship between test sensitivity and specificity. An area under the SROC close to 1 indicated an excellent diagnostic performance of the test. All analyses were performed using STATA 12.0.

This meta-analysis evaluated diagnostic test accuracy indexes, including authenticity evaluation indexes (Se, Sp, FNR, FPR) and comprehensive evaluation indexes (LR and SROC curves). Results were presented as summary values (95 % CI). The heterogeneity of the studies was established by I2, with I2 > 50 % considered moderate heterogeneity and I2 > 75 % defined as high heterogeneity. Our study conducted subgroup analyses to explore the possible sources of heterogeneity. Meanwhile, we assumed the prevalence of 10 %, 20 %, and 30 % of the population, respectively, to calculate the positive predictive value (PPV) and negative predictive value (NPV) under different prevalence rates.

We hypothesized that the population prevalence (P) of COVID-19 was 10 %, 20 %, and 30 %, respectively. The PPV and the NPV were calculated as follow:

PPV = Se * P/ Se* P+ (1-P)* (1-Sp); NPV = Sp* (1-P)/ Sp* (1-P) + (1-Se)* P

Result

Search result

A total of 2584 non-duplicated studies were obtained, and 2274 studies were excluded at the title and abstract level. Finally, 145 full-text articles were evaluated. Through further screening and evaluation of bias risk and applicability of research, 112 full texts have been excluded. Reasons for exclusion: the included study population was unqualified (P). The intervention measures were other detection methods (I). RT-PCR was a reference test, but the index test was another non-nucleic acid detection method (C). Outcome indicators are unavailable, or data cannot be extracted (O). Risk assessment shows a high risk, such as not mentioning whether or not to use the blind method. Other reasons: The number of studies updated in the later stage was insufficient for analysis, so they were excluded. When more similar files are updated later, further analysis can be carried out. Thirty-three studies were included, with 13 related studies on the diagnostic accuracy of RT-LAMP and 16 on RT-PCR (Fig. 1 ).

Fig. 1

PRISMA flow diagram detailing process of study selection for the meta-analysis. Key: CNKI, China National Knowledge Infrastructure. Sino Med, Chinese Biomedical Literature Service System.

Risk of bias and applicability concerns

We assessed the risk of bias according to the QUADAS-2 tool. Implementation of coded or blinded studies was described in 11 studies (Bisoffi et al., 2020; Bordi et al., 2020; Boutin et al., 2020; Chow et al., 2020; Dao Thi, 2020; Gibani et al., 2020; Harrington et al., 2020; Hu et al., 2020; Smith et al., 2020; Yan et al., 2020; Zhen et al., 2020b). Patient inclusion was not continuous and randomized in 9 case-control studies (Baek et al., 2020; Chow et al., 2020; Fukumoto et al., 2020; Li et al., 2020; Matzkies et al., 2020; Mitchell and George, 2020; Shen et al., 2020; Smithgall et al., 2020; Williams et al., 2020). That may overestimate the test's diagnostic accuracy and bring some risks for the interpretation of the results (Fig. 2 ). The results of the publication bias test showed that the Deek's funnel plot was symmetric, and P value of slope coefficient is equal to 0.076 (P > 0.05), indicating no publication bias (Fig. S1).

Fig. 2

Risk of bias and applicability concerns graph of included studies.

Characteristics of included studies

Characteristics of included studies are shown in detail in Table 1 . Four studies assessed more than one index test to compare consistency between different tests. In order to fully present the data of these studies, we included these duplicates in our meta-analysis (Bulterys et al., 2020; Hu et al., 2020; Smithgall et al., 2020; Zhen et al., 2020b). RT-PCR was defined as a reference method in most of the included studies. Mixed respiratory tract samples or deep sputum samples from the trachea or throat were reported in 7/33 studies. Clinical diagnosis, sequencing, and multiple RT-PCR combinations were used as the reference criteria in four studies (Bisoffi et al., 2020; Fukumoto et al., 2020; Shen et al., 2020; Zhen et al., 2020a).

Table 1

Characteristics of included studies in the meta-analysis.

author	country	index test	index test gene target	reference text	reference test gene target	sample type
							TP	FP	FN	TN
(Baek et al., 2020)	Korea	RT-LAMP	N	RT-qPCR	–	NPS	14	2	0	138
(Basu et al., 2020)	USA	RT-LAMP	RdRp	real time RT-PCR	E, N2	NPS	17	1	14	69
(Bisoffi et al., 2020)	Italy	RT-PCR	S, RdRp,	Other a	–	NPS	78	0	7	261
	Italy	real-time RT-PCR	N	Other a	–	NPS	64	1	20	260
	Italy	real-time RT-PCR	E, RdRp	Other a	–	NPS	52	1	33	260
(Bordi et al., 2020)	Italy	real-time RT-PCR	ORF1ab, S	real time RT-PCR	E, RdRp	NPS	99	8	0	171
(Boutin et al., 2020)	Canada	real-time RT-PCR	E	RT-PCR	ORF1, E	NPS/OPS	279	2	22	74
(Bulterys et al., 2020)	USA	iAMP	ORF1ab, N	real time RT-PCR	E	NPS	24	0	5	50
	USA	real-time RT-PCR	E, S, N	real time RT-PCR	E	NPS	26	0	4	50
	USA	real-time RT-PCR	N	real time RT-PCR	E	NPS	27	0	3	50
(Chen et al., 2020)	Hong Kong, China	RT-PCR	ORF1ab, N, E	RT-PCR	RdRp,E	NPS	89	0	2	123
(Chow et al., 2020)	China, Hong Kong	RT-LAMP	orf3a, E	RT-qPCR	N1	total respiratory samples	219	0	4	143
(Cradic et al., 2020)	US	RT-LAMP	RdRp	real time RT-PCR	ORF1ab, S	NPS	30	0	3	151
(Dao Thi, 2020)	Germany	RT-LAMP	ORF1ab, N	RT-qPCR	E	NPS	79	6	2	681
(Fukumoto et al., 2020)	Japan	RT-PCR	–	RT-qPCR	–	total	52	3	1	15
(Gibani et al., 2020)	UK	real-time RT-PCR	E, N, RdRp	RT-PCR	ORF2ab, S, N	NPS	67	0	4	315
(Harrington et al., 2020)	USA	RT-LAMP	RdRp	real time RT-PCR	RdRp, N	NPS	139	2	47	336
(Hogan, 2020)	USA	RT-PCR + lateral flow	N	real time RT-PCR	E	NPS	34	0	16	50
(Hu et al., 2020)	China	RT-LAMP	S	RT-qPCR	ORF1ab, N	NPS	72	4	9	396
	China	RT-qPCR	ORF2ab, N	RT-qPCR + NGS	ORF1ab, N	NPS	66	0	15	400
(Jiang et al., 2020)	China	RT-LAMP	ORF1ab, N	RT-qPCR	ORF1ab, N	NPS	32	0	3	133
	China	RT-LAMP	ORF1ab, N	RT-qPCR	ORF1ab, N	NPS	11	1	1	79
(Kitagawa et al., 2020)	Japan	RT-LAMP	–	RT-qPCR	N	NPS	30	2	0	44
(Li et al., 2020)	China	RT-MCDA	ORF1ab, N	RT-qPCR	–	total respiratory samples	20	2	0	43
(Michael J. Loeffelholz et al., 2020)	USA	real-time RT-PCR	E, N2	real time RT-PCR	ORF1ab, E, N, S, RdRp	NPS, tracheal aspirate	219	11	1	250
(Lu et al., 2020)	China	RT-LAMP	N	RT-qPCR	ORF1ab,N,E	NPS	34	2	2	20
(Matzkies et al., 2020)	Austria	RT-PCR	S	RT qPCR	ORF1ab, E	NPS/OPS	47	0	21	27
(Mitchell and George, 2020)	US	RT-LAMP	RdRp	real time RT-PCR	N	NPS	33	0	13	15
(Rödel et al., 2020)	Germany	RT-LAMP	M	RT-PCR	E	NPS	72	0	24	41
(Shen et al., 2020)	China	RT-qPCR	ORF1ab, N	Other a	–	throat swab	38	2	2	14
	China	RT-qPCR	ORF1ab, N	Other a	–	throat swab	36	2	4	14
	China	RT-qPCR	ORF1ab	Other a	–	throat swab	33	3	7	13
(Smith et al., 2020)	USA	RT-PCR	ORF1ab	TMA, RT-PCR	ORF1ab,ORF8	NPS	74	0	1	74
(Smithgall et al., 2020)	USA	RT-LAMP	RdRp	RT-PCR	ORF1ab, E	NPS	74	0	1	75
	USA	RT-PCR	N2, E	RT-PCR	ORF1ab, E	NPS	65	0	23	25
(Saleem et al., 2020)	USA	real-time RT-PCR	E, N	real time RT-PCR	ORF1ab	NPS	87	2	1	23
(Wang et al., 2020)	China	RT-qPCR	ORF1ab, N	RT-qPCR	ORF1ab, N	throat swabs	53	0	1	50
(Williams et al., 2020)	Australia	real-time RT-PCR	–	RT-PCR	RdRp,E	NPS	25	14	0	142
(Wolters et al., 2020)	Netherlands	real-time RT-PCR	N2, E	RT-PCR	E,RdRp	NPS	53	0	1	621
(Yan et al., 2020)	China	RT-LAMP	ORF1ab, S	RT-qPCR	ORF1ab, S	total respiratory samples	30	0	0	30
(Zhen et al., 2020a)	USA	real-time RT-PCR	N1, N2	Other a	–	NPS	58	0	0	72
	USA	RT-PCR	S, ORF1ab	Other a	–	NPS	51	1	0	52
	USA	RT-PCR	N	Other a	–	NPS	51	0	0	53
	USA	RT-PCR	ORF1ab	Other a	–	NPS	49	0	2	53
(Zhen et al., 2020b)	USA	RT-LAMP	RdRp	RT-PCR	ORF1ab	NPS	51	2	0	51
	USA	real-time RT-PCR	N, S, ORF1ab	RT-PCR	N, S, ORF1ab	NPS	50	0	7	50

NPS, nasopharyngeal swab; NGS, next-generation sequencing; OPS, oropharyngeal swab; RT-LAMP, reverse-transcription loop-mediated isothermal amplification; RT-PCR, reverse transcription-PCR; RT-qPCR, reverse transcription quantitative real time PCR; iAMP, the Atila isothermal amplification assay; RT-MCDA, transcription multiple cross displacement amplification,which is a new rapid isothermal amplification technique. a Other reference: clinical diagnosis, sequencing, combination of the results of multiple detection methods.

Analysis of diagnostic threshold effect

Threshold effect is one of the main causes of heterogeneity in diagnostic tests. The threshold effect between studies can be determined by calculating the Spearman correlation coefficient between the Se and Sp of included studies. When Se is positively correlated with 1-Sp, it indicates a threshold effect.

The Spearman correlation coefficient of RT-LAMP is equal to -0.218 (P = 0.435). The Spearman correlation coefficient of RT-PCR is equal to -0.365 (P > 0.5). The above results indicate no threshold effect.

The meta-analysis of diagnostic accuracy of RT-LAMP and RT-PCR

Meta-analysis results showed that the pooled sensitivity and specificity of RT-LAMP in the diagnosis of COVID-19 were 0.92 (95 % CI, 0.85−0.96) and 0.99 (95 % CI, 0.99−0.99), respectively. The pooled sensitivity and specificity of RT-PCR for the diagnosis of COVID-19 were 0.96 (95 % CI, 0.93−0.98) and 1.00 (95 % CI, 0.98–1.00), respectively (Fig. 3 ). However, results of both RT-PCR and RT-LAMP showed high heterogeneity (I2 > 50 %), especially RT-PCR.

Fig. 3

Forest plot of sensitivity and specificity in COVID-19 diagnosis (A) sensitivity of RT-LAMP; (B) specificity of RT-LAMP and (C) sensitivity of RT-PCR (D) specificity of RT-PCR.

Besides, the meta-analysis showed that the FNR and the FPR of RT-LAMP were 0.12 (95 %CI, 0.06−0.16) and 0.01 (95 % CI, 0.00−0.01), respectively. The FNR and FPR of RT-PCR were 0.06 (95 % CI, 0.04−0.08) and 0.01 (95 % CI, 0.00−0.01), respectively (Fig. 4 ).

Fig. 4

Forest plot of false negative rate (FNR) and the false positive rate (FPR) in COVID-19 diagnosis (A) FNR of RT-LAMP (B) FPR of RT-LAMP and (C) FNR of RT-PCR (D) FPR of RT-PCR.

The positive likelihood ratio (PLR) and negative likelihood ratio (NLR) of RT-LAMP were 112.28 (70.51–178.78) and 0.08 (0.04−0.15), RT-PCR were 193.11(57.52–648.39) and 0.04(0.02−0.07), respectively (Fig. S2). The areas under the SROC of RT-LAMP and RT-PCR were 0.99(0.98–1.00) and 1.00(0.99–1.00), respectively (Fig. S3).

The positive predictive values (PPV) and negative predictive values (NPV) of RT-LAMP and RT-PCR are shown in Table 2 .

Table 2

The predictive value of RT-LAMP and RT-PCR for different prevalence estimates.

Prevalence (P)a	RT-LAMP		RT-PCR
	PPV	NPV	PPV	NPV
P = 10 %	91 %	99 %	100 %	100 %
P = 20 %	96 %	98 %	100 %	99 %
P = 30 %	98 %	97 %	100 %	98 %

NPV, negative predictive value; PPV, positive predictive value; RT-LAMP, reverse-transcription loop-mediated isothermal amplification; RT-PCR, reverse transcription-PCR.

Prevalence is the estimated prevalence.

Subgroup analysis

Subgroup analyses of RT-LAMP diagnostic values are shown in Fig. 5 . Compared with nasopharyngeal swabs(NPS)or oropharyngeal swabs (OPS), the mixed sample of the respiratory tract showed better sensitivity (0.98) and lower FNR (0.02). Compared with a single target gene, multiple target genes has higher sensitivity (0.95) and a lower FNR (0.03). Interestingly, when the reference method was RT-PCR with multiple target genes, the sensitivity and FNR of RT-LAMP were 0.84 and 0.15, respectively; when the reference method was RT-PCR with a single target gene, the sensitivity and FNR of RT-LAMP were 0.90 and 0.11, respectively.

Fig. 5

Subgroup analysis of RT-LAMP in COVID-19 diagnosis. Key: CI, confidence interval; FNR, false negative rate; FPR, false positive rate; I2, I square; LR, likelihood ratio; NPS, nasopharyngeal swab; OPS, oropharyngeal swab. * Number of study, one subgroup contains at least 3 studies.

Subgroup analyses of RT-PCR diagnostic values are shown in Fig. 6 . When the samples were NPS or OPS, the FNR of RT-PCR was 0.06, but the heterogeneity of the results was high (I2 > 75 %). The RT-PCR with multiple target genes had higher sensitivity (0.95) and a lower FNR (0.05) compared with a single target gene. Of note, the sensitivity of RT-PCR was 0.96 (95 % CI, 0.90−0.98) when multi-target gene RT-PCR was used as the reference method. However, when other methods (clinical diagnosis, sequencing, combination of the results of multiple detection methods) were used as the reference, the sensitivity of RT-PCR was only 0.84, and the FNR was as high as 0.15.

Fig. 6

Subgroup analysis of RT-PCR in COVID-19 diagnosis. Key: CI, confidence interval; FNR, false negative rate; FPR, false positive rate; I2, I square; LR, likelihood ratio. NPS, nasopharyngeal swab; OPS, oropharyngeal swab. * Number of study, subgroup contains at least 3 studies.

Discussion

A recent systematic review showed that nucleic acid testing has a high sensitivity (86 %) and specificity (96 %) in the diagnosis of acute respiratory syndrome due to coronavirus (Floriano et al., 2020). However, data on COVID-19 were not mentioned in the study of Floriano et al. In other COVID-19 related meta-analysis and systematic analyses, there is no further detailed classification of reference standards, and the heterogeneity of research results is high (Böger et al., 2021; Castro et al., 2020).

In this systematic review, the pooled sensitivity of RT-LAMP and RT-PCR was 92 % and 96 %, respecitively. The pooled specificity of RT-LAMP and RT-PCR was 99 % and 100 %, respecitively. The positive LR and negative LR results of the two methods suggest a high diagnostic value. Moreover, the area under the SROC was above 0.9. However, the I2 in most of these results was more than 50 %.

Our study found that compared with NPS/OPS, mixed samples from multiple parts of the respiratory tract have better sensitivity and lower FNR, which is consistent with the results of other studies (Böger et al., 2021; Mustafa Hellou et al., 2021). The guideline for preventing and treating COVID-19 in China indicates that molecular diagnosis detection should use at least two gene targets to minimize the risk of false-negative (Jin et al., 2020). Our results also showed that RT-PCR and RT-LAMP with more than one target gene showed better diagnostic value than a single target gene. When RT-PCR was used as a reference method to evaluate the accuracy of RT-LAMP, the sensitivity of RT-LAMP with multi-gene targets was 15 % higher than that of RT-LAMP with a single gene target. Similarly, the sensitivity of RT-PCR with multiple-gene targets increased by 5%.

We can speculate that the detection method with multi-gene targets has high sensitivity and a low FNR. When RT-PCR with multi-gene targets is used as a reference method, the higher the consistency between the index test results and the reference, the higher the diagnostic value. However, from our results, the above rules are only applicable to comparing different RT-PCR methods. When other methods that are likely to correctly classify infected persons, such as clinical diagnosis and sequencing, are used as reference standards, the sensitivity of RT-PCR decreases to 87 % and FNR increases to 15 %. Therefore, RT-PCR is not a perfect gold standard. When RT-PCR (single gene target) was used as the reference method to evaluate RT-LAMP, RT-LAMP showed high sensitivity due to selecting the reference method with a low diagnostic value. Therefore, comparing RT-LAMP with established suboptimal RT-PCR may exaggerate the diagnostic value of RT-LAMP.

Due to the high heterogeneity of the above results, the conclusions need to be treated with caution. Therefore, a better reference method and research design to evaluate the diagnostic accuracy of nucleic acid detection are required. There are many influencing factors in diagnostic trials, which will result in high heterogeneity in the meta-analysis of diagnostic trials. Several factors can result in False-negative results, such as sampling, sample transportation, storage, RNA extraction, the incubation period prior to the symptom (Loeffelholz and Tang, 2020; Touma, 2020).

The advantage of this study is to analyze the diagnostic accuracy of RT-PCR and RT-LAMP more deeply based on previous evidence. Our study investigated the potential effects of different reference methods on diagnostic accuracy and performed subgroup analysis. Besides, compared to similar studies, we chose a more comprehensive evaluation index to evaluate the diagnostic accuracies of the two methods. To the best of our knowledge, this study is the first to use FNR and FPR as evaluation indexes of diagnostic accuracy-test meta-analysis. This may help to extend understanding and further explain problems in clinical diagnosis and epidemiology.

Our systematic review and meta-analysis has several limitations. Firstly, some results showed high heterogeneity. Although, the heterogeneity of analysis results can be reduced through subgroup analysis, some may still have high heterogeneity (I2 > 50 %). This may be related to the inevitable clinical heterogeneity among studies. Secondly, we unified the reference method as RT-PCR, and stratified analysis according to different subgroups. However, different reagent manufacturers and experimental platforms will lead to inevitable heterogeneity. Besides, without a definitive reference standard, it may produce uncertain factors in the design and analysis of such consistent research (Petitti, 2001). These may explain the considerable heterogeneity of most meta-analyses on the same subject.

Conclusion

RT-PCR and RT-LAMP have high diagnostic values in the diagnosis of COVID-19. However, meta-analysis results of the sensitivity and FNR showed high heterogeneity. The accuracy of diagnostic tests is closely related to the sample type and reference criteria. RT-PCR has high sensitivity and a low FNR (pooled FNR of about 6%). The overall pooled specificity of RT-LAMP and RT-PCR was as high as 99 %. However, the sensitivity of RT-LAMP was poor, and about 12 % of true positive patients were considered negative. RT-PCR is the most commonly used reference method at present. The premise is that PCR has excellent sensitivity and specificity, but this is not the case. Therefore, choosing accurate and reliable reference methods is essential when evaluating the diagnostic accuracy of nucleic acid detection methods.

Author contribution statement

Conceptualization: NC; RP; methodology: DS; XR; YL; software: WZ; YY; validation: YH; SL; LM; formal analysis: RP; YB; WZ; investigation and data curation: RP; DS; YB; YY; writing-original draft: RP; SL; writing-review and editing: RP; XR; YH; LM; supervision: NC; SL.

Funding statement

This work is supported by the Belt and Road Special Project of Lanzhou University (grant numbers 2018ldbrzd008).