Comparative performances of seven quantitative Reverse-Transcription Polymerase Chain Reaction assays (RT-qPCR) for detecting SARS-CoV-2 infection in samples from individuals suspected of COVID-19 in São Paulo, Brazil.

Introduction: Brazil is the second largest country with COVID-19 positive cases worldwide. Due to the potent spread of the virus and the scarcity of kits and supplies, the Brazilian Ministry of Health has granted authorization for the use of kits available during this emergency, without an accurate evaluation of their performance. This study compared the performance and cost-effectiveness of seven molecular assays/kits available in São Paulo, Brazil, for SARS-CoV-2 diagnosis. Materials and methods: A total of 205 nasopharyngeal/oropharyngeal samples from suspected cases of COVID-19, were tested using the following assays: (i) GeneFinder COVID-19 plus RealAmp kit; (ii) 2019-nCoV RNA PCR-Fluorescence Probing, Da An Gene Co.; (iii) in-house RT-qPCR SARS-CoV-2 IAL; (iv) 2019-nCoV kit, IDT; (v) molecular SARS-CoV-2 (E) kit, Bio-Manguinhos; (vi) Allplex 2019-nCoV modified Assay, Seegene Inc, and (vii) Biomol one-step COVID-19 kit, IBMP. The criteria for determining a SARS-CoV-2 true positive result included the cycle threshold cut-off values, the characteristics of exponential/linear curves, the gene target diversity, and a positive result in at least two assays.
Results: The overall sensitivity of the assays listed were GeneFinder 83.6%, Da An Gene 100.0%, IAL 90.4%, IDT 94.6%, Bio-Manguinhos 87.7%, Allplex 97.3%, and IBMP 87.7%. The minor sensitive gene target was RdRP. Although all assays had a Cohen's Kappa index ≥0.893, the best tests used multiplex assays identifying N-gene and/or E-gene targets.
Conclusion: All assays tested accurate for diagnosis, but considering cost-effectiveness (cost, time consumption, number of samples tested, and performance), the in-house IAL assay was ideal for COVID-19 diagnosis in São Paulo, Brazil.

Introduction

On 23 March 2021, the World Health Organization (WHO) reported more than 123.4 million confirmed cases of COVID-19 and more than 2.7 million deaths worldwide. Brazil alone, accounts for more than 12.1 million cases and 298 thousand deaths, making it the second country with the highest number of confirmed cases and deaths, according to worldwide rankings [1,2]. The Brazilian pandemic has raised serious concerns since the end of 2020, when the second wave of COVID-19 emerged in the country [2]. In Brazil, the state of São Paulo accounts for the highest number of COVID-19 cases, with 2,332,043 cases confirmed and 68,623 deaths [2], and São Paulo city reported 691,379 cases and 20,574 deaths [3].

The Instituto Adolfo Lutz (IAL) is a public health laboratory located in São Paulo city and a reference laboratory for the diagnosis and surveillance of respiratory viruses. In February 2020, the IAL started to diagnose COVID-19, searching for SARS-CoV-2 in samples from suspected patients. However, given the exponential spread of the virus and the scarcity of kits and supplies available in the market, the Ministry of Health of Brazil has granted authorization for the use of various commercially available kits, without an accurate evaluation of their performance.

To select the most appropriate kits/assays for SARS-CoV-2 molecular diagnosis in São Paulo, Brazil, that would help to reduce costs and provide quick, accurate results, especially during this pandemic, when health care expenditure is rising and resources are limited, we carried out a comparative study on the performance and cost-effectiveness of seven RT-qPCR kits/assays, including one kit modified for the laboratory conditions (Allplex kit) and one standardised in-house assay (duplex RT-qPCR), using SARS-CoV-2 samples obtained from IAL in São Paulo, Brazil.

Materials and methods

Study population

Two hundred and five nasopharyngeal/oropharyngeal samples collected between June and August 2020 and sent to IAL for COVID-19 diagnosis were obtained from a biorepository. These samples were randomly selected for RT-qPCR performance analyses, and the use of samples containing volumes larger than 2000 µL, was the only inclusion criteria. Three new RNA extractions (200 µL each) were conducted using the Biogene Extração de DNA/RNA viral kit (Bioclin Quibasa, MG, Brazil) according to the manufacturer's instructions. The pool of RNA extractions that allowed testing of all RT-qPCR assays was aliquoted, maintained at -70 °C, and used within four days after the extraction.

Laboratory methods

The kits/assays employed in the RT-qPCR COVID-19 comparative performance analyses are described in Table 1 .

Table 1

Diagnostic kits/assays tested for the molecular diagnosis of SARS-CoV-2 infection in São Paulo, Brazil.

Name of the Kit	Developer/Manufacturer (country)	Assay/kit study name	Gene target (labelled probe)	Methods	Ct values considered for positive results
GeneFinder COVID-19 plus RealAmp kit	Osang Healthcare Co. (Korea)	GeneFinder	RdRP (FAM), N (VIC), E (Texas Red), IC (Cy5)	Multiplex RT-qPCR	Ct up to 40
2019-nCoV RNA PCR-Fluorescent Probing	Da An Gene Co. (China)	Da An Gene	N (FAM), ORF 1ab (Yellow), IC (Cy5)	Multiplex RT-qPCR	Ct up to 40
In-house RT-qPCR SARS-CoV-2, IAL	IAL (Brazil)	In-house IAL	E (FAM), RP (HEX) RdRP (FAM)	Duplex RT-qPCR Single RT-qPCR	Ct up to 38
2019-nCoV kit	IDT (USA)	IDT	N1 (FAM), N2 (FAM), RP (FAM)	Single RT-qPCR	Ct up to 39
Molecular SARS-CoV-2 (E) Bio-Manguinhos	Bio-Manguinhos (Brazil)	Bio-Manguinhos	E (FAM), RP (FAM)	Single RT-qPCR	Ct up to 40
Allplex 2019-nCoV Assay (modified)	Seegene (Korea)	Allplex	E (FAM), N (Quasar 670), RdRP (Cal Red 610), IC (HEX)	Multiplex RT-qPCR	Ct up to 40
Biomol OneStep/COVID-19 IBMP	IBMP (Brazil)	IBMP	N (HEX/VIC), ORF 1ab (FAM), IC (ROX)	Multiplex RT-qPCR	Ct up to 40

Ct, cycle threshold ; RdRP, RNA-dependent RNA polymerase; ORF1ab, open-reading frame ORF1ab region ; N, nucleocapsid ; E, envelope gene; RP, human ribonuclease p; IC, internal control; FAM, fluorescein amidite; VIC, 2′‑chloro-7′-phenyl-1,4-dichloro-6-carboxyfluorescein; Cy5, Cyanine 5; HEX, hexachloro fluorescein; ROX, carboxy-X-rhodamine.

These kits differ in terms of composition, gene targets, labelled probes, methods, and interpretation of results, and were used according to the respective manufacturer's instructions, except for the Allplex kit, Seegene, which was modified. For the Allplex kit, 5 µL of RNA were used instead of the recommended 8 µL, and no internal control (IC) was added to the RNA extraction reaction, once it was standardized for use with the Seegene NIMBUS/STARlet equipment. These modifications were aimed at comparing the performance across all assays using the same RNA extraction product (quantity and quality) and the equipment available in our laboratory. Thus, we considered the IC of all other assays to assure the quality of the RNA to be amplified by the Allplex kit.

The results were interpreted according to the manufacturer's instructions. As the Ct limit of positivity (cut-off) value varies according to each developer/manufacturer's criteria across all the assays/kits (Ct varying from 38 to 40), we also analyzed the characteristics of the exponential/linear curves (slope and linear regression analysis).

The RT-qPCR reactions were conducted with 5 μL of RNA and in duplicate, except for the one with the Da An Gene kit, which was conducted individually due to its limited availability. In samples showing Ct values around the cut-off (half Ct above the cut-off value), the reactions were repeated in duplicate (depending on the availability of RNA samples and assays/kits), resulting in four results of several samples for analysis. According to the IDT kit manufacturer samples tested positive for only one N-gene target were considered inconclusive. The highest Ct value obtained in duplicate was considered for data interpretation.

The in-house SARS-CoV-2 IAL assay comprised two steps. The first employed a duplex RT-qPCR assay using oligonucleotides for the E-gene target and the human RNase P (RP-gene) as a control [4]. Thermocycler conditions used were as described by Corman and coworkers (Charité/Berlin Protocol) [5]. The probe for target E was labelled with fluorescein amidite (FAM) and the internal control RP was labelled with hexachloro fluorescein (HEX). Samples that were positive for target E were further tested for SARS-CoV-2 (second step) by RT-qPCR analysis using the RdRP oligonucleotides. Ct values ≤38 were considered positive.

All quantitative PCR assays were performed using the Bio-Rad CFX96 equipment (Bio-Rad, Hercules, CA, USA).

Statistical analysis

GraphPad Prism software version 5.03 (GraphPad, San Diego, CA, USA) was used for RT-qPCR analysis using Kruskal-Wallis method of variance (ANOVA), complemented with the Dunn's multiple comparison test, and the Mann-Whitney U test for the comparison of two assays. Any p-values ≤0.05 were considered statistically significant. In addition to the Ct values of the median, minimum and maximum, values at 25 and 75% of the mean and standard deviation (SD), and the coefficient of variation (CV) were calculated.

The Epi Info software, version 3.5.4 (Atlanta, GA, USA) was used for the comparative analyses of the final results of samples across the kits. The sensitivity, specificity, and Cohen's Kappa values were determined. To identify a SARS-CoV-2 true positive sample, the characteristics of the exponential/linear curves, and the genetic diversity among SARS-CoV-2 strains in Brazil were carefully analyzed, as they may affect the performance of molecular assays [6], [7], [8]. Our criteria required a positive result in at least two of the seven assays independently, based on its interaction with one or more gene targets.

Differences in sensitivity and specificity of the assays were statistically evaluated using the Chi-square test or Fisher's exact test, as appropriate. Statistical significance was set at p ≤ 0.05.

For a health economic evaluation, cost-effectiveness was calculated based on factors such as a correlation between both cost and outcome (in USD) of each assay/kit acquired in May 2020, number of samples analyzed using each assay/kit, and consequently the number of results released by plate, time consumption, ease of use, and assay sensitivity and specificity [9].

Ethical review

The study was approved by the IAL Technical and Scientific Council (CTC#21M-2020) and the Ethics Committee for Research under the Ministry of Health protocol number CAAE – 33282820.8.0000.0059. The data was analyzed while strictly adhering to patient confidentiality guidelines.

Results

The RT-qPCR SARS-CoV-2 assay performances of different kits were compared and summarized in Table 2 . The Da An Gene was highly sensitive, whereas GeneFinder was the least sensitive assay.

Table 2

Results of the kits tested for the molecular diagnosis of SARS-CoV-2.

Assay/Kit	Positive	Negative	Inconclusive	Total
GeneFinder (Korea)	61	144	0	205
Da An Gene Co. (China)	76	129	0	205
In-house IAL (Brazil)	67a/49b	138a/156b	0	205
IDT (USA)	71	128	6	205
Bio-Manguinhos (Brazil)	64	141	0	205
Allplex (Korea)	71	134	0	205
IBMP (Brazil)	64	141	0	205

a, considering the first step; b, considering the second step.

Subsequent analyses were conducted to understand the discrepancy in results among assays and establish the criteria to consider the sample as a true positive.

When the Ct values of the positive samples were analyzed, the median was similar and independent of the gene target across all the SARS-CoV-2 molecular assays tested (Fig. 1 ). Statistically significant differences were detected when comparing the Ct values of RdRP-gene target in GeneFinder and in-house IAL assay, in-house IAL assay and Allplex, and for E-gene target when comparing Bio-Manguinhos and Allplex (Fig. 1).

Fig. 1

Cycle threshold (Ct) values obtained from positive samples for different SARS-CoV-2 genes using seven RT-qPCR assays obtained from different developers/manufacturers. The number of samples positive for each gene corresponds to the number inside the box. Statistical significance is depicted by values * p < 0.05 and ** p ≤ 0.01 using the Kruskal Wallis test complemented with Dunn's Multiple Comparison Test.

The number of positive samples in each assay and their Ct values for each target are listed in Table 3 . Overall, the median Ct values varied from 27 to 31 and the mean Ct values from 27.34 to 30.17. Interestingly, the lower coefficient of variation was detected for the RdRP-gene target (Table 3, in bold), although this target showed minor positive results, while the N-gene target showed more positive results.

Table 3

Analysis of the Cycle threshold (Ct) values obtained from each gene target using seven RT-qPCR assays employed for SARS-CoV-2 molecular diagnosis in São Paulo, Brazil.

	GeneFinder			Da An Gene		In-house IAL		IDT		Bio-Man guinhos	Allplex			IBMP
	RdRP	N	E	N	ORF1ab	E	RdRP	N1	N2	E	E	RdRP	N	ORF1ab	N
Number of positive samples	47	61	56	76	73	67	49	71	71	64	69	65	70	64	63
Minimum	21,00	18,00	17,00	17,00	18,00	17,00	20,00	16,00	16,00	19,00	15,00	19,00	15,00	17,00	20,00
25% Percentile	27,00	26,00	24,00	24,00	26,00	24,00	24,00	23,00	23,00	26,00	22,50	25,50	25,75	25,00	26,00
Median	31,00	29,00	27,00	28,00	30,00	28,00	28,00	27,00	28,00	31,00	27,00	30,00	29,50	28,50	30,00
75% Percentile	33,00	33,50	31,00	34,00	35,50	32,00	31,00	33,00	33,00	35,00	32,00	34,50	35,00	33,00	33,00
Maximum	36,00	40,00	37,00	40,00	40,00	38,00	33,00	39,00	39,00	40,00	37,00	39,00	40,00	40,00	38,00
Mean	29,87	29,52	27,34	28,50	30,16	27,97	27,61	27,75	27,86	30,17	26,41	29,91	29,60	28,64	29,43
Std. Deviation	3848	5458	4933	6551	6392	5792	3931	6124	6220	5924	5939	5648	6289	5610	4613
Std. Error	0,5613	0,6988	0,6592	0,7515	0,7482	0,7076	0,5615	0,7268	0,7382	0,7405	0,7150	0,7005	0,7516	0,7012	0,5812
Lower 95% CI of mean	28,74	28,13	26,02	27,00	28,67	26,56	26,48	26,30	26,39	28,69	24,98	28,51	28,10	27,24	28,27
Upper 95% CI of mean	31,00	30,92	28,66	30,00	31,66	29,38	28,74	29,20	29,33	31,65	27,83	31,31	31,10	30,04	30,59
Coefficient of variation	12.88%	18.49%	18.04%	22.99%	21.19%	20.71%	14.24%	22.07%	22.33%	19.64%	22.49%	18.88%	21.25%	19.59%	15.68%

RdRP, RNA-dependent RNA polymerase; N, nucleocapsid; E, envelope gene; ORF1ab, open-reading frame ORF1ab region; Std., standard; CI, confidence interval.

Meticulous analysis of results using the seven assays/kits revealed that they were negative in 125 samples and positive in 49 samples. In 31 samples, inconclusive results were detected, which are presented in Table 4 , confirming that RdRP was the minor sensitive target.

Table 4

Cycle threshold (Ct) values of samples tested for SARS-CoV-2.

	GeneFinder				Da An Gene			In-house IAL 1st step		In-house IAL 2nd step
	RdRp	N	E	Result	N	ORF1ab	Result	E	Result	RdRP	Result
1	0/0/0/0	35/35/35/34	0/35/0/0	Positive	33	35	Positive	34/33	Positive	0/0/0/0	Negative
2	0/0/0/0	36/36/39/36	0/0/0/0	Positive	34	36	Positive	37/37	Positive	0/0/0/0	Negative
3	0/0/0/0	0/40/41/0	0/0/0/0	Negative	37	39	Positive	0/0/36/36	Positive	0/0/0/0	Negative
4	0/0/0/0	34/34/35/35	0/0/37/35	Positive	32	35	Positive	0/0/33/33	Positive	0/0/0/0	Negative
5	0/0	0/39	0/0	Negative	39	40	Positive	0/0	Negative	0/0	Negative
6	0/0	34/34	36/34	Positive	33	36	Positive	33/33	Positive	0/0/0/0	Negative
7	0/0	0/0	0/0	Negative	40/39/38	41/41/39	Positive	39/0/0/38	Negative	0/0	Negative
8	0/0/0/0	0/38/0/0	0/0/0/0	Negative	38/36	42/40	Positive	37/37	Positive	0/0/0/0	Negative
9	0/0	0/39	0/0	Negative	39/40/40	0/41/0	Positive	0/39/38/0	Negative	0/0	Negative
10	0/0/0/0	37/36/38/36	37/0/0/0	Positive	34	28	Positive	36/35	Positive	0/0/0/0	Negative
11	0/0/0/0	37/36/38/36	0/0/0/0	Positive	35	38	Positive	36/35	Positive	0/0/0/0	Negative
12	0/0	35/34	34/35	Positive	34	38	Positive	34/34	Positive	0/0/0/0	Negative
13	0/0/0/0	37/36/38/0	0/0/0/0	Negative	34	39	Positive	36/35	Positive	0/0/0/0	Negative
14	0/0/0/0	37/40/36/38	41/0/0/0	Positive	36	39	Positive	36/36	Positive	0/0/0/0	Negative
15	0/0	34/34	34/33	Positive	32	35	Positive	34/34	Positive	0/0/0/0	Negative
16	0/0	0/0	0/0	Negative	39/0	41/0	Negative	0/39/40/40	Negative	0/39/40/40	Negative
17	34/34	33/33	33/32	Positive	31	33	Positive	32/31	Positive	0/0/0/0	Negative
18	0/0	35/35	33/33	Positive	34	36	Positive	34/34	Positive	0/0/0/0	Negative
19	0/0/0/0	38/37/38/0	0/0/0/0	Negative	37	40	Positive	37/37	Positive	0/0/0/0	Negative
20	0/0/0/0	37/38/0/37	36/0/0/0	Negative	38	39	Positive	38/38	Positive	0/0/0/0	Negative
21	0/0	0/0	0/0	Negative	39/0	41/0	Negative	39/0/39/0	Negative	0/0	Negative
22	0/0	40/0	0/0	Negative	39	40	Positive	39/0/0/40	Negative	0/0	Negative
23	0/0	0/0	0/0	Negative	39	39	Positive	0/39/40/0	Negative	0/0	Negative
24	0/0/0/0	38/0/40/0	0/0/0/0	Negative	36	38	Positive	39/38/38/39	Negative	0/0	Negative
25	35/36	32/33	32/32	Positive	31	32	Positive	31/31	Positive	0/0/0/0	Negative
26	0/0	0/0	0/0	Negative	0	0	Negative	38/38	Positive	0/0/0/0	Negative
27	0/0	0/0	0/0	Negative	0/40/40	40/40/0	Positive	0/0	Negative	0/0	Negative
28	0/0	39/0	0/0	Negative	37	39	Positive	0/38	Negative	0/0	Negative
29	0/0	0/40	0/0	Negative	37	39	Positive	0/38/39/40	Negative	0/0	Negative
30	0/0	0/0	0/0	Negative	0	41	Negative	0/0	Negative	0/0	Negative
31	0/0	39/0	0/0	Negative	38/39/39/39	41/41/41/0	Positive	38/0/40/40	Negative	0/0	Negative

The in-house IAL assay showed 18 of the 31 RdRP negative samples as positive in the first step and/or in other assays/kits (samples code 1 to 4, 6, 8, 10 to 15, 17 to 20, 25, and 26, Table 4). Of these 18 samples, 10 were positive in 7 assays, 5 in 6 assays, 2 in 5 assays, and one was positive only in the first step of this in-house assay. These results corroborate the minor sensitivity of RdRP-gene target when employed in the second step of the in-house IAL assay. The results obtained by the GeneFinder kit, show RdRP negative results and positive results for other targets in samples with codes 1, 2, 4, 6, 10, 11, 12, 14, 15, and 18 (Table 4). Therefore for the in-house IAL assay, the results of the first step are considered as final, in accordance with the revised recommendation of the Pan American Health Organization (PAHO/WHO) using the Charité protocol [10].

The inconclusive RT-qPCR results were detected in six samples using the IDT assay (samples code 5, 7, 9, 21, 29, and 30, Table 4), which employs only the N-gene target.

Taking into consideration the deficiencies in detecting some targets and the differences in analytical sensitivity of the RT-qPCR assays [11,12], to calculate the sensitivity and specificity of assays, we established the following criteria to identify true positive and negative samples.

First, we re-analyzed the number of negative samples listed in Table 4 that were positive in other assays/kits, including the inconclusive results of the IDT kit. The result of these analyses are shown in Fig. 2 .

Fig. 2

Samples negative for SARS-CoV-2 in each molecular assay that resulted in positive and/or inconclusive outcomes in other assays. The number inside the spheres denotes the number of samples categorised as: Pos 6, positive in 6 assays; Pos 5, positive in 5 assays; Pos 4, positive in 4 assays; Pos 3, positive in 3 assays; Pos 2, positive in 2 assays; Pos 1, positive in one assay; Pos 2 + Inc, positive in 2 assays plus one inconclusive result; Pos 1 + Inc, positive in one assay plus one inconclusive; Inc, one inconclusive.

We identified characteristics of the exponential/linear curves in samples that resulted in Ct values close to the cut-off value, and in samples presenting inconclusive results. Using both analyses, we identified the samples that were negative in all assays (n = 125) as true negative, Pos 1 (sample 16, IDT; sample 26, in-house IAL; sample 27, Da An Gene); Pos 1 + Inc. (samples 5 and 7, Da An Gene, and IDT) and Inc. (samples 21 and 30, IDT) (Table 4) and, the sample that was positive in at least two assays was considered as a true positive (n = 73).

Thus, the final results of SARS-CoV-2 obtained from 205 samples tested by each assay/kit, along with its sensitivity, specificity, and Cohen's kappa indices, are presented in Table 5 .

Table 5

Sensitivity and specificity of kit/assays used to test SARS-CoV-2 samples.

Assay/Kit	True Positive	Sensitivity	Specificity	Kappa index (95% CI)
GeneFinder (Korea)	61	83.56%	100.00%	0.893 (0.834–0.952)
Da An Gene Co. (China)	73	100.0%	97.78%	0.975 (0.948–1.003)
In-house IAL (Brazil)	66	90.41%	99.25%	0.931 (0.884–0.978)
IDT (USA)	70	94.59%	99.25%	0.958 (0.922–0.994)
Bio-Manguinhos (Brazil)	64	87.67%	100.00%	0.921 (0.871–0.972)
Allplex modified (Korea)	71	97.26%	100.00%	0.983 (0.960–1.006)
IBMP (Brazil)	64	87.67%	100.00%	0.921 (0.871–0.972)

CI, confidence interval.

The sensitivity of the assays varied from 83.6% to 100.0%. Differences in assay sensitivity were detected between the following, GeneFinder and Da An Gene (p < 0.001), GeneFinder and IDT (p = 0.034), GeneFinder and Allplex (p = 0.005), Da An Gene and in-house IAL (p = 0.020), Da An Gene and Bio-Manguinhos (p = 0.006), and Allplex and IBMP (p = 0.028) assays/kits. No significant differences in specificity were observed. All assays performed well as per the Cohen's Kappa index analysis (all above 0.893). Considering both sensitivity and specificity, the best assay kits were the Allplex, followed by the Da An Gene.

Data summarizing the cost-effectiveness of the assays/kits are shown in Table 6 . According to the purchase value of kits/assays in May 2020, and other parameters (presented in Table 6), the most economic kit was the in-house IAL first-round assay.

Table 6

Cost-effectiveness of seven SARS-CoV-2 molecular diagnostic assays/kits available in São Paulo, Brazil.

Parameters	GeneFinder	Da An Gene	In-house IAL 1st step	IDT	Bio-Manguinhos	Allplex	IBMP
Number of targets	4	3	2	3	2	4	3
Number of reactions	1	1	1	3	2	1	1
Amount of RNA (µL) for final result	5	5	5	15	10	5	5
Cost of the assay in American dollar ($)	1112.93	1550.40	1820.00a	2461.85a	1681.70	701.36	1191.49
Number of tests in each assay/kit	100	96	500	500	96	100	96
Cost of one test in American dollar ($)	11.13	16.15	3.64	4.92	17.52	7.01	12.41
Number of samples tested per plateb	96	96	96	32	48	96	96
Limitations	16.4% False-negative results	2.2% False-positive results	9.6% False-negative results	5.4% False- negative including inconclusive results	12.3% False- negative results	2.7% False-negative results	12.3% False-negative results

aIncluding SuperScript III Platinum One-Step qRT-PCR System, 500 rxs (Invitrogen, Thermo Fisher Scientific); b including positive and negative controls.

Discussion

Brazil has a population of 212,821,952 inhabitants, of which 46,289,333 belong to the state of São Paulo [13], and 12,325,232 inhabit São Paulo city [14]. The state and municipality are responsible for the major number of cases and deaths due to COVID-19 in Brazil [2,3]. Hence, a large number of samples are sent daily to the IAL for analysis, emphasizing the need for optimum testing within minimal time and complete accuracy of results. At the beginning of the pandemic, only the IDT and Bio-Manguinhos assays/kits (both using single RT-qPCR reactions) were available, therefore, we decided to standardize one duplex RT-qPCR using the E- and RP-gene targets. We tested RdRP levels to confirm SARS-CoV-2; however, the second strategy was inefficient. The possible success and performance of the in-house IAL first step assay and the poor performance of the second step assay have also been confirmed by recently published studies [[6], [7], [8],15].

Briefly, Ramirez et al. [6] highlighted the role of E-gene as a screening target for the diagnosis of SARS-CoV-2. Using in silico analysis of 373 genomes from South America (95 from Brazil), they identified mutations in the primer-probe binding sites to RdRP, N, and E genes. The results revealed abundant genomic diversity in the RdRP and N genes, in contrast to the E gene. This highlights a probable effect on the false-negative results when employing the RdRP gene as the target, and supports the use of the E gene for SARS-CoV-2 screening in South America, thereby corroborating the results obtained from our study conducted in São Paulo, Brazil.

Vogels et al. [7] compared the analytical efficiencies and sensitivities of the primer-probe sets of four most common SARS-CoV-2 RT-qPCR assays developed by the China Center for Disease Control (China CDC), United States CDC (US CDC), Charité Institute of Virology, Universitätsmedizin Berlin (Charité), and Hong Kong University (HKU). They concluded that the E-gene Sarbeco (Charité) was highly sensitive, and the RdRP-SARS gene (Charité) was the least sensitive. The authors attributed the deficiency of the RdRP-gene target to a mismatch in the reverse primer, when compared to the majority of RdRP-genes of SARS-CoV-2 circulating in the USA. Nevertheless, they detected similar analytical sensitivities of the four assays, as observed in the present study. The minor sensitivity of the RdRP primer-probe was further confirmed by Nalla et al. [15] which compared the N-, RdRP-, and E-gene primer-probe sets described by Corman et al. [5], and N1, N2, and N3 primer-probe sets developed by the CDC-USA to diagnose SARS-CoV-2 clinical samples, and detected the E and N2 targets as the most sensitive [15]. In addition, based on in silico alignments, Pillonel et al. showed the use of an incorrect degenerate base (S instead of R) in the design of the reverse RdRP primer described by Corman et al. [8]. These studies reinforce the need to confirm the low sensitivity of such RdRP primers/probe sets for SARS-CoV-2 molecular diagnosis.

In the present study, we were unable to sequence the SARS-CoV-2 false-negative samples. However, a recent study at IAL showed the complete genomic sequencing of 29 SARS-CoV-2 samples that resulted in RdRP-negative outcomes using the Charité-Berlin protocol and SARS-CoV-2 positivity using the Allplex kit. These sequences were deposited in the Global Initiative on Sharing Avian Influenza Data - GISAID (accession numbers EPI_ISL_693215 to EPI_ISL_693243), confirming that the RdRP-gene target employed in the Charité-Berlin protocol is inaccurate for SARS-CoV-2 detection. Studies identifying mutations in regions harboring primer-probe binding sites are currently in progress.

On 19 March 2020, the WHO recommended considering a COVID-19 positive result when at least two specific gene targets were positive on nucleic acid amplification tests (NAAT), or when one target was positive and the COVID-19 virus was identified by sequencing (partial or whole genomic sequence) [16]. On 30 March 2020, PAHO guidelines recommended the use of E and RdRP genes (Charité-Berlin protocol) for diagnosis, prioritizing the E gene for single-target testing [17]. On 8 July 2020, these guidelines were revised and an algorithm that employs only the E-gene for confirming COVID-19 virus infection was recommended, as the only Sarbecovirus that currently circulates in humans is the SARS-CoV-2 [10]. These guidelines and prior studies altogether support the use of in-house IAL first-step RT-qPCR assays for COVID-19 molecular diagnosis.

In conclusion, although the GeneFinder and Allplex detected the major number of gene targets (four) in a single multiplex assay, the GeneFinder resulted in low sensitivity and increased costs. In contrast, the Da An Gene kit presented the highest sensitivity, lacked specificity, and is also expensive. The Bio-Manguinhos and IBMP assays/kits, of Brazilian origin had the same performance, in addition to being expensive however; the IBMP has an advantage as a multiplex assay.

When compared to the in-house IAL and IDT assays, although both were inexpensive, the IDT kit takes longer as it employs three single RT-qPCR reactions in contrast to the in-house IAL assay that employs one reaction (32 tests versus 96 tests per plate, respectively). Interestingly, although the IAL assay showed low sensitivity when compared to other assays, (except for the GeneFinder assay) a Ct value cut-off of 40 would improve assay sensitivity to 94.8%. In retrospect, the stringent criteria for a true-positive adopted by us (Ct up to 38, the minor of all assays) needs to be revised considering the characteristics of the exponential/linear curves, and the new kit launched in the national market by Bio-Manguinhos-FIOCRUZ-RJ, Brazil (Kit Molecular SARS-CoV-2 Bio-Manguinhos E/RP) which employs the E/RP RT-qPCR duplex assay, and a cut-off value of ≤40 for the E-gene target.

Concerning the Allplex kit, despite the problems pointed by several laboratories in Brazil (unpublished data), and also detected by us regarding its internal control (Ct values more than 35, and 20% of false-negative results when employed in the Loccus Extracta RNA equipment and using 10 µL of IC input), we have to consider that this assay was standardized for the conditions and the equipment of the Seegene manufacturer, and more recently for four kits/equipments of RNA extractions: two automatized (SEEPREP 32, Seegene, and NucliSENS easyMAG, BioMérieux), and two manual RNA extractions kits (Ribospin vRD kit, GeneAII, and QIAamp DSP Virus Spin Kit, Qiagen). None of these kits/equipments were available in our laboratory. However, this kit performed well against other genetic targets, showing the best performance among all assay kits tested (Cohen's Kappa index= 0.983). Therefore, this kit could be used without an IC. This assay is cost-effective and is currently in use at IAL and other public health laboratories in Brazil, using an RNA control, in a single or multiplex RT-qPCR assay (unpublished data). Recently, this kit was used to test the community spread of COVID-19 among laboratory staff and other employees of IAL, identifying differences in Ct values based on the clinical status of SARS-CoV-2 infection: symptomatic, presymptomatic, and asymptomatic [18].

Another aspect to consider is the limit of detection (LoD) of the RT-qPCR assays using RNA SARS-CoV-2 standard samples and clinical samples [11,12], which could explain our findings. We did not exclude the lack of SARS-CoV-2 positivity when the viral loads of the samples were under the LoD of the assays.

Besides the limitations of assays reported in the literature [[6], [7], [8],[10], [11], [12]] and in this study, the results showed a good performance of all assays/kits (all Cohen's Kappa index above 0.893), enabling their use in routine diagnosis. Moreover, the results obtained proved that RdRP is not the best gene target by which patients suspected with COVID-19, can be identified in São Paulo, Brazil.

In conclusion, multiplex RT-qPCR assays are the choice for COVID-19 diagnosis and can cater to the current demand for testing in São Paulo, Brazil. The in-house IAL duplex assay proved to be the best method, with respect to cost, time, sensitivity, reagent consumption, ease of performance, immediate results, and accuracy and can be effectively used in the current SARS-CoV-2 pandemic plaguing São Paulo, Brazil.

Declaration of Competing Interest

The authors declare no conflicts of interest.