Comparative Analysis of SARS-CoV-2 Detection Kits.

Background: SARS-CoV-2 is a coronavirus that causes a respiratory disease, COVID-19. For COVID-19 testing, real-time PCR is considered gold standard and therefore many commercial SARS-Cov-2 detection kits are available. Objective: The aim of the study is to determine diagnostic values of 10 different commercially available SARS-CoV-2 detection kits, based on their Ct value.
Methods: For this study thirty clinical nasopharyngeal samples were collected in ALEA Genetic Center. Twenty four of them were positive, while six were negative and used as a negative control. Positive samples were selected based on the day when first symptoms appeared. RNA was extracted using the same extraction method for all samples. For amplification and comparison of detection kits, the same RT- PCR instrument was used.
Results: Accuracy, sensitivity, specificity and Cohen's kappa coefficient were estimated to evaluate diagnostic values of the tested kits. This study showed that all kits showed 100% specificity. Accuracy, sensitivity and kappa coefficient varied among examined assays. Based on clinical features, LabGunTM COVID-19 Assay by LabGenomics proved to be the most sensitive, the most accurate and most specific. Therefore this assay was used as a reference kit.
Conclusion: If things from practice are taken into account, accuracy and reliability of the tested commercial kits can vary compared to those obtained in this study where results were based on ideal functioning of the kits. When choosing the convenient commercial SARS-CoV-2 detection kit using RT-PCR method, many parameters need to be considered.

BACKGROUND

Coronavirus disease 2019 (COVID-19) is an infectious disease which is caused by SARS-CoV-2 virus (1). Coronaviruses belong to the subfamily Coronavirinae that belongs to Coronaviridae family (2). The genome of SARS-CoV-2 is among the largest of RNA viruses, consisting of 29, 881 bp (base pairs), encoding for 9860 amino acids (3). SARS-CoV-2 contains four structural proteins (S, E, N and M) and sixteen non-structural proteins (nsp1-16) (4). Surface of SARS-CoV-2 is covered of large number of glycosylated S proteins that bind to host cell receptor angiotensin-converting enzyme 2 (ACE2), mediating viral cell entry (5). Both symptomatic and asymptomatic patients are the sources of infection and airborne transmission is a dominant route for infection (3, 6-10). Rapid and accurate diagnostic tests are essential for controlling the COVID-19 pandemic. Real-time PCR is is considered the ‘gold standard’ method for SARS-CoV-2 detection (11, 12). The performance of molecular diagnostics is dependent on quality and optimized concentration of primers, probes and reagents needed for SARS-CoV-detection. In this study, real-time PCR analysis of different commercially available kits was performed in order to determine their sensitivity and diagnostic value, based on their Ct value. The results from ten different kits are reported in this paper: UltraGene Combo2Screen SARS-CoV-2 Assay by ABL (Advanced Biological Laboratories) (targeting E and N gene) (13), LabGunTM COVID-19 Assay by LabGenomics (targeting E and RdRp gene) (14), LabGun COVID-19 ExoFast RT-PCR Kit by LabGenomics (targeting RdRp and N gene) (15), PhoenixDx® SARS-CoV-2 Multiplex (targeting RdRp/N gene) (16), DIAGNOVITAL® SARS-CoV-2 Real-Time PCR Kit by RTA Laboratories (targeting RdRp, N and E gene) (17), Bosphore Novel Coronavirus (2019-nCoV) Detection Kit v2 by Anatolia Geneworks (targeting orf1ab and 2019-nCoV screening region) (18), COVID-19 SARS-Cov-2 Real-TM by Saccace Biotechnologies (targeting E gene, N gene and SARS-like coronaviruses) (19), Bio-Speedy SARS-CoV-2 (2019-nCoV) qPCR Detection Kit by Bioeksen (targeting orf1ab) (20), STAT-NAT® COVID-19 MULTI by Sentinel Diagnostics (targeting RdRP and ORF1b-genes) (21) and SAVVYGEN™ SARS-COV-2 by Savyon diagnostics (targeting orf1ab and N gene) (22).

OBJECTIVE

The aim of the study is to determine diagnostic values of 10 different commercially available SARS-CoV-2 detection kits, based on their Ct value.

MATERIALS AND METHODS

Thirty clinical nasopharyngeal samples (24 positive and 6 negative) were collected in ALEA Genetic Center using the same viral transport medium for all SARS-CoV-2 samples. Samples were chosen according to the day when the symptoms first occurred (day 3). Each patient had typical COVID-19 symptoms: cough, fever, muscle pain and anosmia. Each patient was tested on the third day since the symptoms begin. Negative samples were tested as a control and as a validation of the real-time PCR reactions. RNA was extracted using automated magnetic extraction with TianLong® GeneRotex 96 extractor. It uses magnetic bead method and the instrument can process serum, plasma, swab, feces, tissue and other sample types. It has integrated UV light disinfection that can greatly reduce the risk of contamination. RNA extraction was performed according to manufacturer’s instruction. For magnetic extraction, 200 μl of viral transport medium is needed for the final elution buffer of 100 μl. For all kits, Bio-Rad CFX96 real-time PCR was used. Cycling conditions are different for every used kit (Table 1).

Table 1.

PCR cycling conditions for SARS-CoV-2 detection kits

UltraGene Combo2Screen SARS-CoV-2 Assay by ABL (Advanced Biological Laboratories)
Stage	Cycle	Temperature	Time
1 (cDNA synthesis)	1	70	5:00
2 (cDNA synthesis)	1	93	0:03
3 (cDNA synthesis)	1	67.5	2:30
4 (predanturation)	1	93	0:03
5 (denaturation)	1	66	2:30
6 (annealing and extension)	40	95	00:10
		58	00:15
		72	00:15
7 (hold)	1	40	00:01
LabGunTM COVID-19 Assay by LabGenomics
Stage	Cycle	Temperature	Time
1 (cDNA synthesis)	1	50	30:00
2 (pre-denaturation)	1	95	15:00
3 (denaturation)	45	95	00:15
4 (nnealing and extension)		60	1:00
LabGun COVID-19 ExoFast RT-PCR Kit by LabGenomics
Stage	Cycle	Temperature	Time
1 (cDNA synthesis)	1	50	5:00
2 (pre-denaturation)	1	95	1:00
3 (denaturation)	10	95	00:01
		60	00:01
4 (annealing and extension)	32	60	00:01
			00:20
			00:30
			00:30
PhoenixDx® SARS-CoV-2 Multiplex
Stage	Cycle	Temperature	Time
1 (reverse transcription)	1	50	05:00
2 (initial denaturation)	1	95	05:00
3 (amplification)	40	95	00:05
		60	00:30
DIAGNOVITAL® SARS-CoV-2 Real-Time PCR Kit by RTA Laboratories
Stage	Cycle	Temperature	Time
1 (reverse transcription)	1	45	20:00
2 (initial denaturation)	1	95	10:00
3 (amplification)	45	95	00:15
		58	00:45
Bosphore Novel Coronavirus (2019-nCoV) Detection Kit v2 by Anatolia Geneworks
Stage	Cycle	Temperature	Time
1 (reverse transcription)	1	50	17:00
2 (initial denaturation)	1	95	06:00
3 (denaturation)	35	97	00:30
4 (annealing)		60	00:30
5 (hold)	1	32	02:00
COVID-19 SARS-Cov-2 Real-TM by Saccace Biotechnologies
Stage	Cycle	Temperature	Time
1 (cDNA synthesis)	1	35	20:00
2 (pre-denaturation)	1	94	05:00
3 (denaturation)	5	94	00:25
		64	00:10
4 (annealing and extension)	45	94	00:25
		64	00:10
Bio-Speedy SARS-CoV-2 (2019-nCoV) qPCR Detection Kit by Bioeksen
Stage	Cycle	Temperature	Time
1 (reverse transcription)	1	52	05:00
2 (initial denaturation)	1	95	00:10
3 (denaturation)	40	95	00:01
4 (annealing and extension)		55	00:30
STAT-NAT® COVID-19 MULTI by Sentinel Diagnostics
Stage	Cycle	Temperature	Time
1 (reverse transcription)	1	50	10:00
2 (initial denaturation)	1	95	2:00
3 (denaturation)4 (annealing)	10	95	00:15
		58	00:30
3 (denaturation)4 (annealing)	35	95	00:15
		58	00:30
SAVVYGEN™ SARS-COV-2 by Savyon diagnostics
Stage	Cycle	Temperature	Time
1 (reverse transcription)	1	45	15:00
2 (initial denaturation)	1	95	2:00
3 (denaturation)	45	95	00:10
4 (annealing)		60	00:50

The performance of kits was evaluated in comparison with LabGunTM COVID-19 Assay by LabGenomics. The samples were divided into three groups and six samples were used as a negative control. First group contains samples with Ct value < 24, second group contains samples with Ct value between 25-33 and third group contains samples with Ct value >34. Accuracy, sensitivity, specificity and Cohen’s kappa coefficient were calculated for each group and for every kit separately (Table 2).

Table 2.

Accuracy, sensitivity and specificity of used SARS-CoV-2 detection kits

Name of the Kit	Accuracy %	Sensitivity %	Specificity %	Kappa coefficient
LabGun COVID-19 ExoFast RT-PCR Kit by LabGenomics	80	75	100	0,54
PhoenixDx® SARS-CoV-2 Multiplex	87	83	100	0,67
DIAGNOVITAL® SARS-CoV-2 Real-Time PCR Kit by RTA Laboratories	83	79	100	0,6
Bosphore Novel Coronavirus (2019-nCoV) Detection Kit v2 by Anatolia Geneworks	76	70	100	0,5
Bio-Speedy SARS-CoV-2 (2019-nCoV) qPCR Detection Kit by Bioeksen	76	70	100	0,5
UltraGene Combo2Screen SARS-CoV-2 Assay by ABL (Advanced Biological Laboratories)	83	79	100	0,6
COVID-19 SARS-Cov-2 Real-TM by Saccace Biotechnologies	80	75	100	0,54
Savvygen™ SARS-CoV-2	56	46	100	0,26
STAT-NAT® COVID-19 MULTI	87	83	100	0,67

RESULTS

As already mentioned, 24 samples were chosen according to patients symptoms and the day when the symptoms started. The samples were tested using 10 different kits. All samples tested positive only on LabGunTM COVID-19 Assay by LabGenomics. The same patients were tested after 3 days to confirm the diagnosis of COVID-19. In this paper LabGunTM COVID-19 Assay by LabGenomics was used as a reference because of its high specificity in regard to other tested reagents for SARS-CoV-2 detection.

Cohen’s kappa coefficient is a correlation statistic test, where kappa can range from -1 to +1. Values ≤ 0 indicate no agreement, while values between 0.81 and 1 indicate almost perfect agreement. Kappa coefficient test was calculated between LabGunTM COVID-19 Assay by LabGenomics and the nine other evaluated kits. Savvygen™ SARS-CoV-2 has the lowest value of 0.26 indicating lowest fair agreement, Bosphore Novel Coronavirus (2019-nCoV) Detection Kit v2 by Anatolia Geneworks, Bio-Speedy SARS-CoV-2 (2019-nCoV) qPCR Detection Kit by Bioeksen, COVID-19 SARS-Cov-2 Real-TM by Saccace Biotechnologies, LabGun COVID-19 ExoFast RT-PCR Kit by LabGenomics, DIAGNOVITAL® SARS-CoV-2 Real-Time PCR Kit by RTA Laboratories and UltraGene Combo2Screen SARS-CoV-2 Assay by ABL (Advanced Biological Laboratories) are classified as moderate agreement and PhoenixDx® SARS-CoV-2 Multiplex and STAT-NAT® COVID-19 MULTI show highest kappa value of 0.67 which represents as substantial agreement.

Figure 1 shows used kits and Ct values for every sample. On X axis different kits are shown, while Ct values are shown on Y axis. Green dots represent true negative control samples and as seen, all tested kits showed 100% specificity. Red dots show false negative results of tested kits compared to LabGunTM COVID-19 Assay by LabGenomics; Savvygen™ SARS-CoV-2 shows the lowest sensitivity, while STAT-NAT® COVID-19 MULTI shows highest sensitivity. Black dots show the distribution of Ct values of true positive samples. Based on the Ct values of the positive samples on the LabGunTM COVID-19 Assay by LabGenomics, three groups of six samples were formed: group I includes samples with Ct values <24, group II includes samples with Ct values between 25-33 and group III includes samples with Ct values >34. All samples with Ct value over 40 are considered negative.

Figure 1.

Ct values summary of tested kits for SARS-CoV-2 detection

Considering the fact that first two groups contain samples with Ct value below 34, all used kits showed 100% sensitivity compared to LabGunTM COVID-19 Assay by LabGenomics, excluding Savvygen™ SARS-CoV-2 which showed 0% sensitivity in group I (Ct <24). Observing the results of the group III, it is shown that PhoenixDx® SARS-CoV-2 Multiplex and STAT-NAT® COVID-19 MULTI have highest sensitivity regarding Ct values over 34, while Bosphore Novel Coronavirus (2019-nCoV) Detection Kit v2 by Anatolia Geneworks and Bio-Speedy SARS-CoV-2 (2019-nCoV) qPCR Detection Kit by Bioeksen showed lowest sensitivity (Figure 2).

Figure 2.

Obtained sensitivity of SARS-CoV-2 detection kits in group III

DISCUSSION

Statistical parameters show performance of used kits, but it should be noted that results of this study are based on flawless functioning of the kits. If things from practice are taken into account, such as probe instability, this can affect the period in which the results are reliable using mentioned kits. For example, a kit that was at the top of the scales in statistical parameters (PhoenixDx® SARS-CoV-2 Multiplex) showed the lowest shelf life in terms of probe instability leading to false positive results, while other kits showed more stable results.

Main point of this study is to facilitate the choice in the detection of SARS-CoV-2 using RT-PCR method. During the pandemic in Bosnia and Herzegovina, we encountered several epidemic waves, so in those periods it was necessary to process as many samples as possible in a very short time. Therefore, the efficiency of kits is also reflected in cycling conditions, as well as in the ease of preparation of the mastermix. Although a total of 10 different kits on 30 samples were used in this study, the statistical results obtained show slight deviations from laboratory practice. Consequently, more samples should be processed in the future in order to bring the results of the study as close as possible to laboratory practice.

CONCLUSION

If things from practice are taken into account, accuracy and reliability of the tested commercial kits can vary compared to those obtained in this study where results were based on ideal functioning of the kits. When choosing the convenient commercial SARS-CoV-2 detection kit using RT-PCR method, many parameters need to be considered.