Clinical Performance and Analytical Sensitivity of Three SARS-CoV-2 Nucleic Acid Diagnostic Tests.

Hundreds of RT-qPCR kits are available in the market for SARS-CoV-2 diagnosis, some of them with emergency use authorization (EUA) by the Food Drug Administration (FDA) or their country of origin agency, but also many of them without any independent clinical performance evaluation. We performed a clinical evaluation for two Chinese SARS-CoV-2 RT-PCR kits available in South America, COVID-19 Nucleic Acid Test Kit (eDiagnosis Biomedicine, Wuhan, China) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech, Changsha, China), for RT-qPCR SARS-CoV-2 diagnosis using the FDA EUA 2019-nCoV CDC kit (IDT, Coralville, IA) as gold standard. We found an excellent clinical performance and analytical sensitivity for both kits with sensitivity values of 100% and 95.3% and estimated limit of detection of 500 copies/mL and 1,000 copies/mL, for eDiagnosis and Sansure Biotech kits, respectively. COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) are both made in China and hold EUA by the Chinese CDC. Also, Sansure Biotech kit has EUA by the FDA. In conclusion, our results endorse the use of these two commercially available kits imported to Ecuador for SARS-CoV-2 diagnosis, as they had the similar clinical performance as the gold standard from the CDC.

INTRODUCTION

The COVID-19 pandemic has challenged public health systems worldwide, from patient care to epidemic management, but also to even guarantee the quality of SARS-CoV-2–related diagnostic products.[1] Hundreds of in vitro SARS-CoV-2 RT-qPCR diagnostic kits are available on the market, some of them have received emergency use authorization (EUA) from the U.S. Food & Drug Administration (FDA) or other international agencies like Chinese CDC (C-CDC), whereas many others only report clinical validations made by manufacturers. The U.S. CDC (CDC) designed the FDA EUA 2019-nCoV CDC kit (IDT), based on N1 and N2 gene targets to detect SARS-CoV-2 and RNase P as an RNA extraction quality control that is considered a gold standard worldwide for SARS-CoV-2 RT-PCR diagnosis.[2-5]

Among the commercial kits available in South America for SARS-CoV-2 diagnosis, several are made in China. For instance, COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) are both multiplex SARS-CoV-2 RT-qPCR kit for N and ORF1ab viral gene targets and RNase P as an RNA extraction quality control. Although both of them are authorized for clinical use in China, only the Sansure Biotech kit holds the FDA EUA.[6] We also have previously reported that other made in China SARS-CoV-2 RT-PCR kits lacking FDA EUA but holding C-CDC EUA had a great clinical performance and analytical sensitivity.[7]

We herein present a comparison of the analytical and clinical performance of COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) for SARS-CoV-2 RT-qPCR diagnosis from nasopharyngeal samples, using the CDC protocol as a gold standard.

MATERIALS AND METHODS

Study design.

Ninety seven clinical specimens (nasopharyngeal swabs collected on 0.5 mL TE pH 8 buffer) were included on this study. Also, negative controls (TE pH 8 buffer) were included as control for carryover contamination, one for each set of RNA extractions.

RNA extraction and RT-qPCR for SARS-CoV-2 diagnosis using 2019-nCoV CDC kit.

All the samples included on the study were tested following an adapted version of the CDC protocol: (1) using AccuPrep Viral RNA extraction kit (Bioneer, Daejeon, South Korea) as an alternate RNA extraction method, and (2) using CFX96 BioRad instrument.[8-12]

RT-qPCR for SARS-CoV-2 diagnosis using COVID-19 Nucleic Acid Test Kit (eDiagnosis).

Same RNA extractions from all the samples included in the study were tested using COVID-19 Nucleic Acid Test Kit (eDiagnosis), following the manufacturer’s manual. To avoid RNA degradation, samples were processed for 2019-nCoV CDC kit and COVID-19 Nucleic Acid Test Kit (eDiagnosis) within the same day, or frozen at −80 C to be processed the next day.

RT-qPCR for SARS-CoV-2 diagnosis using 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech).

Seventy four from the total 97 RNA extractions included in the study were tested using 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech). Although this kit includes a lysis buffer for an alternative RNA isolation without using RNA extraction kits, we did not follow that protocol. Also, although the manufacturer’s manual recommends a final volume reaction of 50 µL (30 µl of PCR Master Mix + 20 µL of RNA extraction), we used a final volume reaction of 15 µL (9 µL of PCR Master Mix + 6 µL of RNA extraction), keeping the proportion of reagents recommended by the manufacturer. To avoid RNA degradation, samples were processed for 2019-nCoV CDC kit and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) within the same day, or frozen at −80 C to be processed the next day.

Analytical sensitivity.

Limit of detection (LoD) was performed using the commercially available 2019-nCoV N positive control (IDT); provided at 200.000 genome equivalents/mL, it was used for calibration curves to obtain the viral loads of the samples. Viral loads can be expressed as copies/µL of RNA extraction or copies/mL of sample; the conversion factor is 200, as 0.2 mL of sample is used for RNA extraction and 40 µL is used as final elution volume of RNA extraction.

Statistics.

95% confident intervals were calculated using Jamovi statistical software.

Ethics statement.

All samples have been submitted for routine patient care and diagnostics. Ethics approval was not sought because the study involves laboratory validation of test methods and the secondary use of anonymous pathological specimens that falls under the category “exempted” by Comité de Etica para Investigación en Seres Humanos from “Universidad de Las Américas.”

RESULTS

Clinical performance and estimation of LoD of COVID-19 Nucleic Acid Tests Kit (eDiagnosis) using 2019-nCoV CDC kit as a gold standard.

Ninety seven samples were tested for SARS-CoV-2 following both COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV CDC kit protocols as described on the methods. For the 2019-nCoV CDC EUA kit, 63 samples tested positive and 34 samples tested negative (Table 1 and Supplemental Tables 1 and 2). Thirty-two of 34 samples tested negative for 2019-nCoV CDC kit were also SARS-CoV-2 negative for COVID-19 Nucleic Acid Test Kit (eDiagnosis), so the specificity obtained in our study was 94.1% (91.0–97.2, CI 95%). The two “false-positive” samples had Ct values > 38 (Supplemental Tables 1 and 2) that are associated to low viral loads. As the LoD of COVID-19 Nucleic Acid Test Kit (eDiagnosis) is 2.5 copies/µL according to the manufacturers, these two “false-positive” samples would be actually true-positive SARS-CoV-2 samples that are below the LoD of the CDC protocol.[8-12] So the real specificity for COVID-19 Nucleic Acid Test Kit (eDiagnosis) is 100%.

Table 1

Clinical performance of COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) using the CDC protocol as a gold standard (% values: sensitivity)

RT-PCR kit	Positive samples	False-negative samples	Total SARS-CoV-2–positive samples
COVID-19 Nucleic Acid Test Kit (eDiagnosis)	43 (100%)	0	43
2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech)	41 (95.3%)	2	43

Only SARS-CoV-2–positive samples included on the study are detailed.

For the 63 SARS-CoV-2–positive samples for 2019-nCoV CDC EUA kit, all were also positive for COVID-19 Nucleic Acid Test Kit (eDiagnosis), resulting in a sensitivity of 100% (Table 1 and Supplemental Tables 1 and 2).

As the LoD is defined as the lowest viral load in which all replicates are detected (100% sensitivity), our data indicate that the LoD for COVID-19 Nucleic Acid Test Kit (eDiagnosis) would be around 2.5 copies/µL of RNA extraction (500 viral RNA copies/mL of sample), as no failure to detect SARS-CoV-2–positive samples above this threshold was found (Supplemental Table 1).

Clinical performance and estimation of LoD of 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) using 2019-nCoV CDC kit as a gold standard.

Seventy-four samples were tested for SARS-CoV-2 using 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) and 2019-nCoV CDC kit protocols as described on the methods. For the 2019-nCoV CDC EUA kit, 43 samples tested positive and 31 samples tested negative (Table 1 and Supplemental Table 1). Twenty-eight of 31 samples tested negative for 2019-nCoV CDC kit were also SARS-CoV-2 negative using 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech), so the specificity obtained in our study was 90.3% (87.3–93.7, CI 95%). The three “false-positive” samples had Ct values > 37 (Supplemental Table 1) that are associated to low viral loads. As the LoD of 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) is 1 copie/µL according to the manufacturers, these three “false-positive” samples would be actually true-positive SARS-CoV-2 samples that are below the LoD of the CDC protocol.[7-One Health. 2020 ">11] So the real specificity for 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) is 100%.

For the 43 SARS-CoV-2–positive samples for 2019-nCoV CDC EUA kit, 41 samples were also positive for 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech), resulting in a sensitivity of 95.3% (93.1–97.5, CI 95%; Table 1 and Supplemental Table 1).

As the LoD is defined as the lowest viral load in which all replicates are detected (100% sensitivity), our data indicate that the LoD for 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) would be above five copies/µL of RNA extraction (1,000 viral RNA copies/mL of sample), as only one failure (40/41; 97.6% sensitivity) to detect SARS-CoV-2–positive samples above this threshold was found (Supplemental Table 1).

DISCUSSION

Although the main limitation of our study is the sample size (97 specimens), our results support that both COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) had a great clinical performance compared with 2019-nCoV CDC EUA, with sensitivity values up to 100% and 95.3%, and specificity values of 100%.

As we have described on the Results, we could estimate the LoD of COVID-19 Nucleic Acid Tests Kit (eDiagnosis) on approximately 500 viral RNA copies/mL of sample which is on agreement with the LoD detailed at the manufacturer’s manual. Moreover, we could estimate the LoD of 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) to be around on 1,000 viral RNA copies/mL of sample, slightly over the LoD of 200 copies/mL detailed at the manufacturer’s manual. However, the LoDs obtained for both kits are quite acceptable for a reliable SARS-CoV-2 diagnosis considering the viral load frequency population distributions.[13,14]

On Table 2, analytical parameters and other characteristics for COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) are summarized. As we have detailed on the Introduction, only the Sansure Biotech kit has both FDA EUA and C-CDC EUA, whereas the eDiagnosis only holds C-CDC EUA. Moreover, we have already published another report showing a great clinical performance from another C-CDC EUA kit, from Da an Gene company.[7] In addition, we have also already published other clinical evaluation reports for Spanish, Canadian, and South Korean RT-PCR kits with really variable results in terms of sensitivity and LoD.[7,8,15-17] Although further studies would be necessary involving many more SARS-CoV-2 RT-PCR diagnostic kits, we already found an interesting trend: only those kits with their country of origin clinical use authorization had an acceptable clinical performance and analytical sensitivity.

Table 2

Comparison of 2019-nCoV CDC EUA (IDT), COVID-19 Nucleic Acid Test Kit (eDiagnosis), and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech)

SARS-CoV-2 RT-PCR kit (company/country)	Viral gene targets	LoD observed (promised by the manufacturer)	EUA
2019-nCoV CDC EUA (IDT)	N1, N2	1,000 viral copies/mL	FDA
COVID-19 Nucleic Acid Test Kit (eDiagnosis)	N, ORF1ab	500 viral copies/mL (500 viral copies/mL)	C-CDC
2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech)	N, ORF1ab	1,000 viral copies/mL (200 viral copies/mL)	FDA C-CDC

C-CDC = Chinese CDC; EUA = Emergency Use Authorization; FDA = Federal Drug Administration; LoD = limit of detection.

Considering the worldwide high demand of reagents for SARS-CoV RT-qPCR diagnosis, independent clinical performance and analytical sensitivity evaluations are necessary to guarantee the quality of the supplies in the market for every country in the world, particularly for developing countries usually lacking of reliable regulatory agencies. We claim for the role of universities at developing countries like Ecuador, not only to assess quality evaluations like the ones described on this work but also to support an extremely needed massive SARS-CoV-2 diagnosis.

Supplemental materials