Multiple-centre clinical evaluation of an ultrafast single-tube assay for SARS-CoV-2 RNA.

OBJECTIVE: To evaluate the performance of an ultrafast single-tube nucleic acid isothermal amplification detection assay for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) RNA using clinical samples from multiple centres.
METHODS: A reverse transcription recombinase-aided amplification (RT-RAA) assay for SARS-CoV-2 was conducted within 15 minutes at 39°C with portable instruments after addition of extracted RNA. The clinical performance of RT-RAA assay was evaluated using 947 clinical samples from five institutions in four regions of China; approved commercial fluorescence quantitative real-time PCR (qRT-PCR) kits were used for parallel detection. The sensitivity and specificity of RT-RAA were compared and analysed.
RESULTS: The RT-RAA test results of 926 samples were consistent with those of qRT-PCR (330 were positive, 596 negative); 21 results were inconsistent. The sensitivity and specificity of RT-RAA was 97.63% (330/338, 95% confidence interval (CI) 95.21 to 98.90) and 97.87% (596/609, 95% CI 96.28 to 98.81) respectively. The positive and negative predictive values were 96.21% (330/343, 95% CI 93.45 to 97.88) and 98.68% (596/604, 95% CI 97.30 to 99.38) respectively. The total coincidence rate was 97.78% (926/947, 95% CI 96.80 to 98.70), and the kappa was 0.952 (p < 0.05).
CONCLUSIONS: With comparable sensitivity and specificity to the commercial qRT-PCR kits, RT-RAA assay for SARS-CoV-2 exhibited the distinctive advantages of simplicity and rapidity in terms of operation and turnaround time.

Introduction

In December <span class="Disease">2019 a pneumonia outbreak associated with a <span class="Species">novel coronavirus occurred in Wuhan, Hubei province, China [1]. This outbreak was named coronavirus disease 2019 (COVID-19) [2]; later the associated novel coronavirus was named severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) [3].

At present there is no effective treatment for <span class="Disease">COVID-19. The key to prevention and control lies in early diagnosis, early treatment and early isolation of <span class="Species">patients. Therefore, rapid and accurate diagnosis of SARS-CoV-2 infection is crucial to break the chain of transmission. Virus isolation and culture is the reference standard of SARS-CoV-2 detection, but it is time consuming and is limited to experimental conditions. Nucleic acid test is not only a vital tool for the diagnosis and differential diagnosis of SARS-CoV-2 but also a basis for judgeing the curative effect. Fluorescence quantitative real-time PCR (qRT-PCR) is the most widely used detection method of SARS-CoV-2 RNA. However, faster and simpler molecular detection assays are greatly in need in resource-limited areas, where PCR cyclers and trained personnel are lacking. For example, in community fever clinics, a large number of patients with fever need to be identified or ruled out in a timely manner to prevent potential cross-infection in the crowded clinics.

Compared to qRT-PCR, recombinase-aided amplification (RAA)-based assay is faster and simpler. The results can be obtained under a constant temperature with portable instruments within 30 minutes. Briefly, the principle of RAA is to use a recombinase binding tightly to the primer to form a complex of the enzyme and primer. When the primer searches for a sequence completely complementary to the template DNA, strand replacement occurs between the primers and the template with the single-stranded DNA binding. In the presence of DNA polymerase, the new DNA fragment could be amplified rapidly in vitro. A specific fluorescent probe is added to the amplification system to realize RAA real-time monitoring. RAA has been successfully applied in the detection of adenovirus [4], pertussis [5], respiratory syncytial virus [6], coxsackievirus, enterovirus [7,8] and hepatitis B virus [9] in our previous reports.

Recently we developed a single-tube reverse transcription recombinase–aided amplification (RT-RAA) assay for <span class="Species">SARS-CoV-2. This RT-RAA kit was first launched on 29 January 2020, and it passed the external quality assessments of the National Institute for <span class="Disease">Viral Disease Control and Prevention (IVDC), Chinese Center for Disease Control and Prevention (CCDC) and Beijing Center for Disease Control (CDC).

Here we report a <span class="Gene">multicentre clinical evaluation of this RT-RAA kit using a large number of clinical samples at five institutions in four regions, performed in China.

Methods

Specimens

The specificity evaluation panel was provided by IVDC and CCDC. This panel consisted of inactivated culture of <span class="Species">SARS-CoV-2, <span class="Species">human coronavirus (CoV-229E, OC43, HKU1, NL63), human rhinovirus (HRV), human bocavirus (HBoV), parainfluenza virus (PIV), respiratory syncytial virus (RSV), human metapneumovirus (HMPV), influenza A virus (Flu A), Chlamydia pneumoniae (CP), Streptococcus pneumoniae (SP) and Klebsiella pneumoniae (KPN). We obtained clinical samples from consecutive patients being treated in five institutions from four regions in China (Hubei, Jiangsu, Zhejiang and Shanxi provinces) for multicentre clinical evaluation. These samples were divided into four groups. Group A comprised Jiangsu CDC; group B, Zhejiang CDC; group C, Datong CDC, Shanxi province and the Fifth People's Hospital of Datong City, Shanxi province; and group D, Hubei CDC. The specimens that could be processed within 24 hours were stored at 4°C; those that could not were stored at −70°C or colder. All aspects of the study were performed in accordance with national ethics regulations and were approved by the relevant institutional review boards.

Primer and probe design, and analytical sensitivity and specificity of RT-RAA kit

We downloaded all <span class="Species">SARS-CoV-2 genome sequences available from the Global Initiative on Sharing All Influenza Data (GISAID). We then selected the conserved region of the <span class="Gene">ORF1ab gene of SARS-CoV-2 as the target. The corresponding primers and probes were designed using Oligo 7 (https://www.oligo.net/) according to the principles required by RAA. A total of 46 pairs of primers and two probe candidates were synthesized by Sangon Biotech (Shanghai, China). A preliminary evaluation of the RT-RAA specificity was conducted in silico. We used the recombinant plasmid containing the target gene to screen the best set of primers and probes. The recombinant plasmid was diluted to 105, 104, 103, 102, 101, 2 and 1 copies per test; nonribozyme water was used as a negative control. We then used the specificity evaluation panel to further explore the specificity of the RT-RAA kit for SARS-CoV-2. Finally two primers and one probe with the highest amplification efficiency were chosen.

Nucleic acid extraction

According to the instructions recommended by the manufacturer, total RNA was extracted from 200 μL of sample preservation solution using an automatic extraction kit (Tian Long, Soochow, China). The nucleic acid was eluted in 50 μL of nuclease-free <span class="Chemical">water and stored at −80°C until use.

Protocol of RT-RAA kit for SARS-CoV-2

We freeze-dried the selected primers and probe to the reaction unit tube, then made a single-tube <span class="Species">SARS-CoV-2 nucleic acid isothermal amplification rapid detection (RT-RAA) kit. A reaction system of 50 μL was set up. The system included one reaction unit tube containing lyophilized primer/probe and enzymes, 42.5 μL of reaction buffer, 2.5 μL of 280 mM magnesium acetate, 5 μL of extracted nucleic acid or 5 μL negative/positive control. A 42.5 μL buffer was added to each reaction unit tube with a pipette, and 2.5 μL magnesium acetate solution was added to the inside of the reaction tube <span class="Species">cover. After that the sample or negative and positive control was added. After capping the tube, the reaction tube was symmetrically placed in RAA-B6108 for mixing and centrifugation for 7 minutes. The reaction tube was then removed and transferred to the nucleic acid amplification fluorescence detector RAA-F1620. The reaction temperature was set at 39°C for 10 minutes, and the time of each sample to reach the threshold was measured in real time by the fluorescence signal detector.

Reference qRT-PCR kits for SARS-CoV-2 detection

The 3rd edition of the technical guide for laboratory testing of <span class="Disease">COVID-19 recommended that the confirmed case of <span class="Disease">COVID-19 infection should be identified by qRT-PCR kits. So far about 19 qRT-PCR kits have been officially approved by the National Medical Products Administration (NMPA) and used in the detection of COVID-19 throughout China.

The reference qRT-PCR kits in this study were the <span class="Species">2019-nCoV Detection Kit (BioGerm, Shanghai, China) for groups A and B, the <span class="Species">2019-nCoV Detection Kit (Sansure, Hunan, China) for group C and the 2019-nCoV Detection Kit (BGI, Shenzhen, China) for group D. All test kits were approved by NMPA and received European Union CE certification. The parameters of the kits are shown in Table 1 . The qRT-PCR tests were performed in parallel with RT-RAA tests at the cooperative sites that provided the samples.

Table 1

Parameters of commercial quantitative real-time PCR kit used as reference methods

Group	Manufacturer	Gene	RNA (μL)	Temperature and cycles	Result (Ct)		Lowest detection limit (copies/mL)	Internal control
					+	—
A, B	BioGerm Biotechnology Co Ltd	ORF1ab, N	5	60°C, 40 cycles	≤38	>38	1000	Yes
C	Sansure Biotechnology Co Ltd	ORF1ab, N	10/20	60°C, 45 cycles	≤40	>40	500	Yes
D	BGI Co Ltd	ORF1ab	10	60°C, 40 cycles	≤40	>40	100	Yes

Data from manufacturers' instructions. C, cycle threshold.

Groups are as follows: A, Jiangsu Center for Disease Control (CDC); B, Zhejiang CDC; C, Datong CDC, Shanxi Province and Fifth People's Hospital of Datong City, Shanxi Province; and D, Hubei CDC.

Groups are as follows: A, Jiangsu Center for Disease Control (CDC); B, Zhejiang CDC; C, Datong CDC, Shanxi Province and Fifth <span class="Species">People's Hospn>ital of Datong City, Shanxi Province; and D, Hubei CDC.

Operation standard

All the experimental operations and biosafety protections in this study strictly abided by the <span class="Species">SARS-CoV-2 laboratory biosafety guidelines (2nd ed.) [10] and the technical guide for laboratory testing of <span class="Disease">COVID-19 (4th ed.) [11] issued by the general office of the state health commission.

Statistical analysis

SPSS Statistics 21 software (IBM, Armonk, NY, USA) was used to perform all of the statistical analyses. The results of qRT-PCR and RT-RAA assays were analysed using kappa and McNemar tests, and a value of p < 0.05 was considered statistically significant. Kappa stands for the measure of agreement between the two tests; a value of >0.9 is excellent. Linear regression analysis was used to analyse the relationship between the time threshold (TT; minutes) detected by RT-RAA and the cycle threshold (C ) of the qRT-PCR method.

Results

Sensitivity and specificity of RT-RAA kit

A panel of diluted recombinant plasmids (105, 104, 103, 102, 101, 2 and 1 copies per test) were tested to ascertain the endpoint dilution. As shown in Fig. 1 , the sensitivity of RT-RAA kit was 2 copies per reaction. As shown in Fig. 2 , the RT-RAA kit showed 100% specificity for <span class="Species">SARS-CoV-2; No <span class="Disease">cross-reaction with four common coronaviruses and other viral and bacterial pathogens was observed.

Fig. 1

Sensitivity of RT-RAA assays for SARS-CoV-2. RT-RAA, reverse transcription recombinase–aided amplification; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

Fig. 2

Specificity of RT-RAA assays for SARS-CoV-2 for human coronavirus (CoV-229E, OC43, HKU1, NL63), human rhinovirus (HRV), human bocavirus (HBoV), parainfluenza virus (PIV), respiratory syncytial virus (RSV), human metapneumovirus (HMPV), Chlamydia pneumoniae (CP), influenza A virus (Flu A), Streptococcus pneumoniae (SP) and Klebsiella pneumoniae (KPN). RT-RAA, reverse transcription recombinase–aided amplification; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

Sensitivity of RT-RAA assays for <span class="Species">SARS-CoV-2. RT-RAA, reverse transcription recombinase–aided amplification; <span class="Species">SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

Specificity of RT-RAA assays for Species">SARS-CoV-2 for <Species">span class="Species">human coronavirus (CoV-229E, OC43, HKU1, NL63), human rhinovirus (HRV), human bocavirus (HBoV), parainfluenza virus (PIV), respiratory syncytial virus (RSV), human metapneumovirus (HMPV), Chlamydia pneumoniae (CP), influenza A virus (Flu A), Streptococcus pneumoniae (SP) and Klebsiella pneumoniae (KPN). RT-RAA, reverse transcription recombinase–aided amplification; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

Characteristics of samples

A total of 947 clinical specimens were obtained from <span class="Species">patients with suspn>ected <span class="Disease">pneumonia and SARS-CoV-2 infection from 18 January 2020 to 1 April 2020, of which 59.66% (565/947) came from male subjects. The age range was 7 to 93 years old, with an average age of 44 ± 17.1 (standard deviation) years. The types of specimens included pharynx swab (834, 88.07%), sputum (82, 8.66%), nasopharyngeal swab (16, 1.69%), nasal swab (8, 0.84%), bronchoalveolar lavage fluid (4, 0.42%), stool (2, 0.21%) and whole blood (1, 0.11%). The C values of qRT-PCR positive samples ranged from 8.59 to 39.07.

Comparison of RT-RAA and qRT-PCR

In total, 947 samples were detected by RT-RAA and qRT-PCR (Table 2 , Supplementary Table S1). Among the 947 samples, RT-RAA results of 926 samples were consistent with qRT-PCR results (330 were positive, 596 negative), and 21 samples (1 sputum and 20 pharynx swabs) were inconsistent. The total coincidence rate between RT-RAA and qRT-PCR was 100% in both group A and group B. The C value of positive samples was in the range of 17.00 to 37.00 in group A and 20.50 to 36.50 in group B. In group C, six samples had inconsistent results. Among them, two samples with C value of 38.00 were positive by qRT-PCR but missed by RT-RAA, and the remaining four samples were negative by qRT-PCR but positive by RT-RAA. In group D, 15 samples had discordant results, including six samples positive by qRT-PCR but missed by RT-RAA; the C values of these samples were 38.33, 38.22, 39.07, 36.89, 37.56 and 38.16 respectively. The remaining nine samples were negative by qRT-PCR but positive by RT-RAA. Positive rates of RT-RAA and qRT-PCR in different sample types are shown in Table 3 . The lower <span class="Species">respiratory tract samples (bronchoalveolar lavage fluid) had the highest positive rate, followed by nasopharyngeal swab, nasal swab and pharynx swab. Compared to qRT-PCR, the sensitivity of RT-RAA was 97.63% and the specificity was 97.87%. The positive predictive value was 96.21% (330/343) (95% CI 93.45 to 97.88), the negative predictive value was 98.68% (596/604) (95% CI 97.30 to 99.38) and the total coincidence rate was 97.78% (926/947) (95% CI 96.80 to 98.70); kappa was 0.952 (p < 0.05).

Table 2

Results of 947 samples detected by RT-RAA and qRT-PCR

RT- RAA	qRT-PCR result		Total	Sensitivity (%) (95% CI)	Specificity (%) (95% CI)	PPV (%)	NPV (%)	Kappa
	+	—
+	330	13	343	97.63 (95.21–98.90)	97.87 (96.28–98.81)	96.21	98.68	0.952
—	8	596	604
Total	338	609	947

CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value; qRT-PCR, quantitative real-time PCR; RT-RAA, reverse transcription recombinase–aided amplification.

Table 3

Positive rates of RT-RAA and qRT-PCR in different samples

Sample	Positive RT-RAA	Positive qRT-PCR
Pharynx swab	37.89 (309/834)	38.37 (313/834)
Sputum	7.32 (6/82)	8.54 (7/82)
Nasopharyngeal swab	87.5 (14/16)	87.5 (14/16)
Nasal swab	62.5 (5/8)	62.5 (5/8)
Bronchoalveolar lavage fluid	100 (4/4)	100 (4/4)
Stool	0 (0/2)	0 (0/2)
Whole blood	0 (0/1)	0 (0/1)

Data are presented as percentage (n/N).

qRT-PCR, quantitative real-time PCR; RT-RAA, reverse transcription recombinase–aided amplification.

Time distribution of RT-RAA–positive samples

We showed the time distribution of partial RT-RAA–positive samples (90 samples from groups A, B and C, Fig. 3 ) to explore the quantitative possibility of RT-RAA. The correlation between the TT detected by RT-RAA and the C values of qRT-PCR method was weak, with a coefficient R 2 value of 0.303. The regression equation was y = (0.08821 × x) − 2.005 (gradient: 0.08821, 95% CI 0.06465 to 0.1634; intercept: −2.005, 95% CI −2.841 to −1.170; p < 0.0001).

Fig. 3

Linear regression analysis of RT-RAA TT (y-axis) and qRT-PCR cycle threshold values (C) (x-axis). Data determined by SPSS 21.0 software. qRT-PCR, fluorescence quantitative real-time PCR; RT-RAA, reverse transcription recombinase–aided amplification; TT, threshold time.

As shown in Fig. 3, the time of most samples to reach the threshold was within 3 minutes, to which must be added the prereaction of 7 minutes; therefore, for most samples 10 minutes were needed to determine positivity. Because most of the samples with low virus load (C  ≥ 35) had higher TT values, we prolonged and set up the detection time to 8 minutes to ensure no occurrence of false-negative results, making the duration of total process 15 minutes or less.

Discussion

The manifestations of <span class="Disease">SARS-CoV-2 infection are highly nonspecific, including <span class="Disease">respiratory symptoms, fever, cough, dyspnea and viral pneumonia [12]. A large number of qRT-PCR methods have been developed for specifically detecting SARS-CoV-2 [[13], [14], [15]], but they are not well suited for use in the community setting. Point-of-care testing is undoubtedly the most powerful detection in this regard; however, the main obstacle of point-of-care testing is the relatively low throughput, so it is not ideal for rapid screening [16]. Isothermal nucleic acid amplifications, such as NASBA, LAMP [17,18] and CRISPR's Sherlock technology [19,20], are good alternatives to qRT-PCR. These methods showed comparable sensitivity and specificity to qRT-PCR in the detection of SARS-CoV-2 as well as relatively easier operation and shorter turnaround time; however, these reports only included small numbers of clinical samples and data, so the feasibility of these methods needs to be further verified. In this study we conducted a multicentre clinical evaluation of RT-RAA kit for SARS-CoV-2, using about 1000. We also included respiratory tract samples (throat swabs, sputum, nasopharyngeal swabs, nasal swabs, bronchoalveolar lavage fluid) and nonrespiratory samples (stool, whole blood), which makes our results more reliable and adaptable.

The most attractive feature of the RAA kit is the ultrafast speed of detection, making it ideal for rapid preliminary screening in the clinical setting, particularly for applications in resource-poor settings. The detection results could be achieved in 8 to 15 minutes after addition of extracted nucleic acid. Actually, the time for the vast majority to test positive (corresponding C values ranged 17.00–37.00) was within 10 minutes. Because RT-RAA is able to test 16 samples per run, in practice, a single person is able to finish 16 samples in 40 minutes by working with automatic DNA extraction. In addition, the RAA device is portable and easy to use. Moreover, the RT-RAA kits containing the lyophilized reaction pellets are convenient for transport because lyophilized reagents are stable at room temperature. Like qRT-PCR kits, the single-tube design of the RT-RAA reduces the possibility of cross-contamination. Finally, it is cheaper than the qRT-PCR kits.

The proposed RT-RAA kit had a sensitivity of 2 copies per reaction using DNA template, which might not reflect the true sensitivity using RNA template. A total of 21 inconsistent samples either had high C values (positive only by qRT-PCR) or high TT value (positive only by RT-RAA), highlighting the importance of cautious consideration of these positive findings. This discrepancy between the two methods was not reconciled by repeated experiments because of the heavy work burden during the outbreak at our institutions. The failure (potential false-negative and false-positive result) of the RT-RAA kit might be due to virus load lower than the detection limit, operation <span class="Disease">error or different sensitivity and specificity of the different qRT-PCR kits used. Yet compared to the commercial qRT-PCR kits, the total coincidence rate was 97.78% (926/947, 95% CI 96.80 to 98.70) and the kappa value 0.952 (p < 0.05), thus indicating that the overall clinical performance of our RT-RAA kit was comparable to approved qRT-PCR kits.

Nevertheless, this study has a few limitations. The internal control was not included in the kit, so there was no monitoring of occurrence of falsely negative results. Additionally, only a single gene (<span class="Gene">ORF1ab) was targeted; specificity or sensitivity might thus be compromised. The nucleic acid extraction step was not integrated in the RAA device, which hinders its portability in field detection. Further improvements to this RT-RAA kit are underway in this regard.

In conclusion, our results strongly demonstrate that our RT-RAA kit for testing for <span class="Species">SARS-CoV-2 shows comparable sensitivity and specificity to commercial qRT-PCR kits while having the distinct advantages of simplicity and rapidity over qRT-PCR kits in terms of operation and turnaround time. This RT-RAA kit is therefore a promising tool to be potentially used in resource-limited settings and/or in remote areas such as community <span class="Disease">fever clinics and mobile laboratories.

Transparency declaration

This work was supported by grants from the China Mega-Projects for <span class="Disease">Infectious Disease (2018ZX10711001, 2017ZX10104001 and 2018ZX10713-002), IVDC (<span class="CellLine">2019HYDQNJJ03) and COVID-19 Prevention Research Program of Datong (2020-1). All authors report no conflicts of interest relevant to this article.