Evaluation of the RealStar® SARS-CoV-2 RT-PCR kit RUO performances and limit of detection.

BACKGROUND: The use of efficient, reliable and sensitive PCR assays is a cornerstone in the race to contain the SARS-CoV-2 pandemic. In this work we performed an independent evaluation of the RealStar® SARS-CoV-2 RT-PCR Kit Researh Use Only (Altona) for SARS-CoV-2 detection.
METHODS: A comparative limit of detection (LoD) assessment was performed between RealStar® SARS-CoV-2 and the currently WHO recommended RT-PCR (WHO-PCR) workflow using a quantified clinical sample. Assessment of the RealStar® SARS-CoV-2 assay was also performed using 83 primary clinical samples in comparison with the WHO-PCR.
RESULTS: The RealStar® SARS-CoV-2 demonstrated a slightly higher sensitivity than the WHO recommended assay with a limit of detection at 625 copies/mL instead of 1250 copies/mL for the WHO-PCR in our conditions. The overall percent agreement between RealStar® SARS-CoV-2 and WHO-PCR on 83 clinical samples was 97.6 % (81/83) with a sensitivity at 97.8 % (45/46) and specificity at 97.3 % (36/37). No cross reaction was encountered for the other human coronaviruses (HKU1, OC43, NL63, 229E).
CONCLUSIONS: In this comparison of the RealStar® SARS-CoV-2 assay with the reference WHO assay, we observed a slightly better sensitivity of the RealStar® assay. It provides a robust option for all molecular biology laboratories, with a strong real-life LoD and is compatible with various real-time PCR platforms.

Introduction

The novel coronavirus identified in China in December 2019 as the etiological agent of the COVID-19 disease has been declared as pandemic by the WHO the 11th of March 2020 [1]. The etiological agent of this new infection is the Severe Acute Respiratory Syndrome coronavirus 2 (SARS-CoV-2) [2]. Availability and reliability of robust assays providing sensitive identification of infected individuals are the first steps allowing patient isolation, diagnosis and proper therapeutic measures. Several in-house and commercial RT-PCR assays have been quickly developed [3] but still require to be independently assessed and evaluated. The RealStare® SARS-CoV-2 RT-PCR kit (Altona) is a Research Usage Only (RUO) product providing the detection of SARS-CoV-2 and compatible with various real-time PCR platforms. It allows detection and differentiation of lineage B-betacoronavirus (B-ßCoV), by targeting the E gene from B-ßCoV, and SARS-CoV-2 specific RNA, by targeting the S gene. Evaluating the sensitivity of RT-PCR assays is an essential point as very low viral loads can be identified in patients nasopharyngeal swabs, especially in later disease stages [4,5]. To date, the RealStar® SARS-CoV-2 assay has been compared to several other commercial assay [6,7], however thorough limit of detection assessment and performances comparison with the currently WHO recommended assay have not been provided.

Here, we report an independent validation of this new commercial PCR assay for SARS-CoV-2 detection, determining its limit of detection and comparing its performances on clinical samples with the currently WHO recommended RT-PCR assay [3].

Methods

Analytical sensitivity analysis of the RealStar® SARS-CoV-2 assay

A preliminary sensitivity analysis was conducted using serial dilution of a SARS-CoV-2 positive nasopharyngeal sample. This sample was serially diluted at 1:10 up to 1/100 000 before being serially diluted at 1:5 up to 1/12 500 000. All those dilutions were tested both with RealStar® SARS-CoV-2 and the currently WHO recommended RT-PCR assay (WHO-RT-PCR). The same sample was quantified using a standardised RNA transcript control obtained from the European Virus Archive Program. The limit of detection (LoD) was then determined for the two assays by testing multiple replicates of serial two fold dilutions of the quantified sample around the expected LoD. As recommended by the European Network of GMO Laboratories for LoD definition [8], the LoD was defined as the concentration until obtaining 10 out of 10 positive replicates (10/10). All dilutions were done using Virocult transport medium and kept at +4 °C before testing within 24 h.

Briefly, for both assays the viral RNA was extracted from 200 μL of clinical samples with the MagNA Pure LC Total Nucleic Acid Isolation Kit - Large Volume (Roche Diagnostics) and eluted in 50 μL. The WHO RT-PCR, targeting E and Orf1 genes, was performed as described by Corman et al. [3] and the RealStar® SARS-CoV-2 was performed according to the manufacturer recommendations. Both PCR assays were performed on an ABI 7500 plateform (Applied Biosystems®). For both assay, a signal with a cycle threshold (Ct) value above 40 was considered as negative.

Evaluation of specificity

To assess the specificity of the RealStar® SARS-CoV-2 assay towards other coronaviruses we used using 2 pools of 3 positive samples for each of them (HKU1, NL63, OC43, 229E) obtained between October and December 2019.

Performance comparison using clinical samples

We included 83 nasopharyngeal swabs specimens from patients hospitalized in Bichat Claude Bernard teaching hospital, Paris, France. These specimens were taken from patients suspected of COVID-19, collected in Virocult® viral transport media (Sigma). Samples for the specificity evaluation were obtained for mPCR point of care evaluation approved by the Bichat Claude Bernard ethic committee (N2019−050). All other hospitalized patients, included in this study, were included in the national French−COVID19 cohort and written consent was obtained for clinical and biological sub-studies.

Results

All results obtained from initial serial dilutions are depicted in Table 1 . The virus was detected for all genes of both methods for all dilutions up to 1:100 000. At 1:500 000 dilution, all targets were positive for RealStar® SARS-CoV-2 but only the E gene for the WHO RT-PCR. At 1:2 500 00 dilution, only the two targets of the RealStar® SARS-CoV-2 assay were positive. As we previously observed, the RdRp gene of the WHO RT-PCR assay presented a lower sensitivity than the E gene (Visseaux et al. JCM, in revision).

Table 1

Results of the preliminary evaluation of the sensitivity of RealStar® SARS-CoV-2 assay and the currently recommended WHO assay.

Tested dilution	WHO assay E gene				WHO assay RdRp gene		RealStar SARS-CoV-2 E gene		RealStar SARS-CoV-2 S gene
	Ct	Interpretation	copies/PCR	copies/mL	Ct	Interpretation	Ct	Interpretation	Ct	Interpretation
1/10	20	Positive	6 254 420	312 721 000	19	Positive	19.6	Positive	19.2	Positive
1/100	24.3	Positive	412 044	20 602 200	24.4	Positive	23.1	Positive	22.7	Positive
1/1000	28.7	Positive	25 648	1 282 400	29.5	Positive	26.6	Positive	26.4	Positive
1/10 000	32.8	Positive	1 877	93 850	33.9	Positive	31	Positive	30.7	Positive
1/100 000	36.4	Positive	198	9 900	40.6	Negative	33	Positive	32.7	Positive
1/500 000	38.9	Positive	41	2 050	N.D.	Negative	36.4	Positive	36	Positive
1/2 500 000	N.D.	Negative	–	–	N.D.	Negative	40.1	Negative	38.6	Positive
1/12 500 000	N.D.	Negative	–	–	N.D.	Negative	N.D.	Negative	N.D.	Negative

For the E gene with the WHO assay, used as the gold standard method, the quantification estimated using a standardised RNA transcript control has been calculated. N.D.: Not Detected.

For the LoD assessment, all results obtained with the serial 1:2 dilutions around the expected LoD are depicted in Table 2 . The detection of 10/10 replicates by the WHO RT-PCR assay was achieved for the dilution containing 1250 copies/mL (i.e. for an input of 25 copies/PCR in our conditions) whereas it was achieved for the dilution containing 625 copies/mL with both E and S genes of the RealStar® SARS-CoV-2 assay (i.e. for an input of 13 copies/PCR).

Table 2

Limit of detection assessment for RealStar® SARS-CoV-2 and the currently recommended WHO assay.

Dilution	Viral load (copies/mL)	WHO assay	RealStar SARS-CoV-2
		E gene	E gene	S gene
1/100 000	10 000	36.8	34.5	33
		36.8	34.9	33
		37.7	34.7	33.3
1/200 000	5000	37.3	35.5	33.9
		38.3	36	34.4
		37	36	34.5
1/400 000	2500	38.4	37.4	35.5
		37.9	37.3	35.2
		38.2	36.8	35.1
1/800 000	1250	39.8	39.5	37.8
		39.6	38.2	36.8
		39.4	37.5	35.5
		39	37.9	35.6
		38.9	38.2	37.4
		37.4	39	38.2
		39.4	37.7	37.2
		39.7	39.6	36.4
		38.7	38	39.3
		39.1	38.4	37.1
1/1 600 000	625	39.5	39	36.3
		39.4	39.5	39.2
		39.3	38.6	38.3
		39.4	38.1	36.2
		N.D.	38.6	37.9
		N.D.	38.8	39.2
		N.D.	39	37.4
		N.D.	40.8	38.2
		N.D.	38.1	37.6
		N.D.	38.3	36.4
1/3 200 000	312.5	39.1	40.3	39.1
		N.D.	40.9	37.9
		N.D.	40.9	N.D.
		N.D.	N.D.	N.D.
		N.D.	N.D.	N.D.
		N.D.	N.D.	N.D.
		N.D.	N.D.	N.D.
		N.D.	N.D.	N.D.
		N.D.	N.D.	N.D.
		N.D.	N.D.	N.D.

N.D.: Not Detected.

No cross-reaction was detected neither for human coronaviruses (229E, OC43, NL63 and HKU1, n = 2 pools of three samples for each targets) nor other respiratory viruses (influenza A:H1 N1, A:H3N2 and B, respiratory syncytial virus, rhinovirus, parainfluenza virus 1 and 4, and human metapneumovirus; n = 1 pool for each).

All results from this comparison are given in Fig. 1 and supplementary Table 1. Among the 83 clinical samples tested in this study, 45/83 and 36/83 were identified positive and negative with both methods, respectively. Among the two remaining samples: (i) one was identified as positive with the WHO assay (E gene at 37.9 Ct and undetectable RdRp gene) but negative with the RealStar® SARS-CoV-2 assay; and (ii) the second one was identified as negative with the WHO assay (E gene above 40 Ct at 40.9 and undetectable RdRp gene) but positive with the RealStar® SARS-CoV-2 assay (E gene at 34.4 and S gene at 36.9). Thus, when taking the WHO assay as a gold standard, the RealStar® assay demonstrated a sensitivity at 97.8 % (45/46) and a specificity at 97.3 % (36/37).

Fig. 1

Concordance of cycle threshold values obtained for the E gene with RealStar® SARS-CoV-2 assay and the WHO recommended RT-PCR.

Discussion

In this work we assessed the performances and limit-of-detection on clinical samples of the RealStar® SARS-CoV-2 RT-PCR Kit RUO in comparison to the currently recommended WHO RT-PCR assay [3]. The LoD obtained for the E gene with the WHO RT-PCR assay, at 50 copies/PCR, was slightly higher than previously estimated in the initial description at 5 copies/PCR [3]. This may be explained in part by the use of SARS-CoV-2 clinical samples in the current work instead of transcript RNA and the use of different LoD estimation methods. The LoD of the RealStar® SARS-CoV-2 assay was in the same range and even slightly higher than the WHO assay in our work at 25 copies/PCR. The poor sensitivity of the WHO assay for the RdRp gene has been previously evidenced in our previous works [9] and a recent pre-publication comparing the performances of the main reference assays [10]. The sensitivity and specificity of both the RealStar® SARS-CoV-2 RT-PCR RUO assay on clinical samples appears similar with 100 % of concordance on 83 clinical nasopharyngeal samples. Thus, the RealStar® SARS-CoV-2 RT-PCR RUO assay provides a promising commercial alternative for SARS-CoV-2 detection with a slightly better sensitivity than the WHO currently recommended assay. It can be used in most laboratories with various extraction and real-time PCR platforms. To date, RT-PCR assays remain the methods of choice for COVID-19 diagnosis even if it can lack of sensitivity in swab samples from some patients, mostly presenting late stage diseases and for which chest CT or other respiratory samples can be of valuable help [[11], [12], [13]]. This point also highlights the necessity of having highly sensitive PCR assays and the critical importance of checking and comparing their performances. If the RealStar® SARS-CoV-2 assay will not fulfil all our needs, especially for high-throughput and highly automatized assays, it provides a robust, versatile and sensitive options for all molecular biology laboratories.

CRediT authorship contribution statement

Benoit Visseaux: Conceptualization, Writing - original draft. Quentin Le Hingrat: Writing - original draft, Investigation. Gilles Collin: Investigation, Conceptualization, Methodology. Valentine Ferré: Investigation, Writing - review & editing. Alexandre Storto: Methodology, Writing - review & editing. Houria Ichou: Resources, Conceptualization, Writing - review & editing. Donia Bouzid: Resources, Writing - review & editing. Nora Poey: Resources, Writing - review & editing. Etienne de Montmollin: Resources, Writing - review & editing. Diane Descamps: Supervision, Writing - review & editing. Nadhira Houhou-Fidouh: Supervision, Writing - review & editing.

Declaration of Competing Interest

BV received travel accommodations and personal fees from Qiagen and Biomérieux. The other authors have no conflict of interest to declare.