Evaluation of the Roche antigen rapid test and a cell culture-based assay compared to rRT- PCR for the detection of SARS-CoV-2: A contribution to the discussion about SARS-CoV-2 diagnostic tests and contagiousness.

Background: The most sensitive method to detect SARS-CoV-2 relies on rRT-PCR; however, viral RNA can be detected weeks/months after clinical resolution. Since rRT-PCR cannot discern between non- and infectious virus, it is unclear whether the presence of viral RNA after recovery reflects infectious SARS-CoV-2. However, recent studies suggest a positive correlation between antigen rapid tests (Ag-RDT) and virus isolation that is more suited to assess contagiousness.
Objectives: To assess the utility of SARS-CoV-2 diagnostic tests in different settings we evaluated the performance of Ag-RDT-based and a cell culture-based SARS-CoV-2 assay in comparison to rRT-PCR. Study design: A total of 61 Nasopharyngeal-Swabs tested positive by cobasⓇ SARS-CoV-2 rRT-PCR were in parallel evaluated with the Roche Ag-RDT and a cell culture-based assay to detect SARS-CoV-2.
Results: SARS-CoV-2 was successfully isolated in 51/61 samples corresponding to 83.6%, which was 97.3% or 96.2% when considering samples with E-gene Ct-value <25 and <28, respectively. In comparison, the Ag-RDT showed an overall sensitivity of 85.2%, that increased to 100% and 96.2% using an E-gene Ct-value cut-off of <25 and <28, respectively. There was an overall good agreement between the commercial Ag-RDT and our in-house cell culture-based SARS-CoV-2 detection assay. However, SARS-CoV-2 could be isolated from two samples that tested negative by Ag-RDT. Conclusions: Our results support the use of the Roche Ag-RDT to detect SARS-CoV-2 exposure in large scale populations. However, it is recommended to use rRT-PCR, potentially in conjunction with cell culture-based SARS-CoV-2 assay, to support clinicians in making decisions regarding fragile patient groups.

Background

In December 2019, a new zoonotic coronavirus emerged in Wuhan, Hubei Province, China named Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), which is the etiological agent of Coronavirus Disease 2019 (COVID-19) [[1], [2], [3]]. The clinical features of SARS-CoV-2 infected patients range from mild cold-like symptoms to severe illness ultimately leading to acute respiratory distress syndrome (ARDS) [2,4]. Patients at older age and with underlying comorbidities are at higher risk for developing severe courses of COVID-19 [5]. One of the most profound phenotypic characteristics of fulminant SARS-CoV-2 is the early replication in the upper respiratory tract of infected individuals, followed by prolonged shedding of viral RNA in patients with COVID-19 weeks after clinical resolution and development of neutralizing antibodies [4,[6], [7], [8], [9]]. Based on other respiratory viruses, this prolonged detection of SARS-CoV-2 is likely more evident among hospitalized or immunocompromised individuals [[10], [11], [12], [13], [14]].

The current reference method for SARS-CoV-2 detection relies on real-time reverse transcriptase polymerase chain reaction (rRT-PCR) [15]. Nonetheless, it is unclear whether the prolonged detection of viral RNA reflects the actual presence of infectious SARS-CoV-2 virus, as molecular diagnostic methods cannot differentiate between non- and infectious virus. Therefore, it remains elusive how long symptomatic and asymptomatic carriers remain a potential transmission reservoir for SARS-CoV-2, influencing mitigation strategies against SARS-CoV-2 in a hospital setting.

Cell culture-based assays are more poised to differentiate between non- and infectious virus [16]. However, cell culture-based isolation of SARS-CoV-2 requires a dedicated BSL3 infrastructure [13].

Meanwhile, immunologic based SARS-CoV-2 antigen-detecting rapid diagnostic tests (Ag-RDT) are also being used for diagnosis of SARS-CoV-2, and would potentially allow treating physicians diagnosing patients at their bedside or can be employed in large community settings for contact tracing. However, Ag-RDTs are reported to be less sensitive compared to rRT-PCR diagnostic assays, especially in samples with a relatively low viral load. Nonetheless, Ag-RDTs may perform well in patients with high viral loads (Ct-values ≤25), who are probably infectious [17]. However, since immunological diagnostics cannot differentiate between non- and infectious virus their applicability in a hospital setting remains to be evaluated.

Objectives

In order to assess the utility of SARS-CoV-2 diagnostic tests in different settings we evaluated the performance of Ag-RDT-based and cell culture-based SARS-CoV-2 diagnostics in comparison to the current molecular reference method. For this, Nasopharyngeal Swabs (NPS) collected from patients undergoing SARS-CoV-2 diagnostic that tested positive with cobasⓇ SARS-CoV-2 rRT-PCR were subsequently evaluated with the Roche Ag-RDT and a cell culture-based assay to detect SARS-CoV-2.

Study design

rRT-PCR analysis and sample selection

A total of 5121 NPS samples were collected between the 14th and 27th of October 2020 from patients undergoing SARS-CoV-2 diagnostic and first evaluated with the cobasⓇ SARS-CoV-2 real-time reverse transcriptase polymerase chain reaction (rRT-PCR) diagnostic test on a cobasⓇ 6800 System according to the manufacturer's instructions (Roche). Only positive tested samples (Ct-value varying between >10 and <35 with E-gene rRT-PCR) with enough leftover Viral Transport Medium (> 1 ml) that were stored at 4°C for a maximum of 24 hours after collection were included in this study. For assessing the specificity of the Ag-RTD we also incorporated 31 samples that were tested negative by rRT-PCR.

Antigen-detecting rapid diagnostic test (Ag-RDT)

The detection of SARS-CoV-2 antigens was performed with the commercially available SARS-CoV-2 Rapid Antigen Test (Roche). All NPS were vortexed prior to transferring 350 µl of sample into the provided extraction buffer of the Ag-RDT kit. After mixing, three drops of the extracted sample were applied to the specimen portal of the lateral-flow test device. Readout and interpretation of the results was done according to the manufacturer's instructions.

Cell lines

Vero E6 cells (kindly provided by M. Müller and C. Drosten, Charité, Berlin, Germany) were propagated in Dulbecco's modified Eagle Medium–GlutaMAX, 10% (v/v) heat-inactivated fetal bovine serum (FBS), 100 mg/mL streptomycin, 100 IU/mL penicillin, 1% (v/v) non-essential amino acids and 15 mM 4-(2-hydroxyethyl)-1-piperazineethanesulfonic acid (HEPES, Gibco, Gaithersburg, MD, USA). Cells were maintained at 37°C in a humidified incubator with 5% CO2.

Cell culture-based detection of SARS-CoV-2

For SARS-CoV-2 isolation, 400 µl of NPS sample was vortexed and centrifugated at 2900 x g for 5 minutes at room temperature and supplemented with 10% v/v of penicillin and streptomycin mixture. A volume of 35 ul of processed sample was inoculated on 15.000 Vero E6 cells in a 96-cluster well plate, this was done in three technical replicates. Inoculated plates were centrifuged at 724 x g for 40 minutes at room temperature followed by the removal of the viral inoculum and supplementation of 175 µl of Earle's minimal essential medium, containing 1% L-glutamine, 1% penicillin/streptomycin/fungizol, and 1% heat-inactivated FBS. Cell culture plates were incubated at 37°C in a humidified incubator with 5% CO2 and visually inspected every 1-2 days for the appearance of a cytopathic effect (CPE). Cells were fixed in 4% neutral buffered formalin solution when at least 70% of the monolayer displayed CPE, or in the absence of CPE 8 – 10 days after inoculation and processed for immunofluorescence analysis, as previously described but without counterstaining [18]. SARS-CoV-2 antigen-positive cells were detected using a rabbit polyclonal anti-SARS-CoV nucleocapsid protein (Rockland, Limerick, PA, USA 200-401-A50) and a secondary Donkey F(ab')2 Anti-Rabbit IgG Antibody Fluorescein Conjugated Pre-Adsorbed (Rockland Inc., Pennsylvania, USA; cat. no. 711-702-127) as conjugate. Stained plates were then examined under a fluorescent microscope, using a 20x objective.

Statistical analysis

Statistical analyses were performed using the software package MedCalc for Windows, version 19.0.4 (MedCalc Software, Ostend, Belgium). The sensitivity of the Ag-RDT and cell culture-based assay were calculated with a 95%CI in relation to rRT-PCR. The agreement between Ag-RDT and culture was analysed using the Cohen's weighted kappa.

Results

In order to directly compare the sensitivity of SARS-CoV-2 Ag-RDT and cell culture-based SARS-CoV-2 detection with the SARS-CoV-2 rRT-PCR test we chose to only include NPS samples that had an E-gene rRT-PCR Ct-value below 35, and were stored at 4°C for a maximum of 24 hours after collection. Importantly, for the comparison of cell culture-based SARS-CoV-2 diagnostic assay with both rRT-PCR and Ag-RDT assays within a single NPS sample the residual volume after rRT-PCR needed to exceed 1 ml. Based on the selection criteria we further evaluated a total of 61 NPS.

Performance of Ag-RDT compared to rRT-PCR

Out of the 61 samples tested positive by the cobasⓇ rRT-PCR, 52 were also found to be positive by Ag-RDT and thereby revealed an overall sensitivity of 85.2% (95%CI 73.8-93%) for the Roche Ag-RDT (Table 1 ). Because Ag-RDTs have shown to have a relatively poor performance among samples with a relatively low viral load, we also assessed the sensitivity at different arbitral cut-offs. When we only consider samples with E-gene Ct-value of <25, <28 or <30 the positive samples by Ag-RDT compared to rRT-PCR were 38/38, 51/53 and 52/56 resulting in a sensitivity of 100% (95%CI, 90.7-100%), 96.2% (95%CI, 87.8-99.5%) and 92.9 (95%CI, 82.7-98%), respectively (Table 1). Interestingly, the nine samples that were tested negative with the Ag-RDT all had an E gene Ct-value higher than 25. No rRT-PCR negative sample was positive by Ag-RDT (Suppl. Table 1). These results demonstrate the detection sensitivity of the Roche Ag-RDT depends on the viral load in the NPS sample.

Table 1.

Results of Ag-RDT and viral culture of rRT-PCR positive samples by different E-gene Ct-values intervals.

PCR E-gene Ct-value	<20	20-25	25-28	28-30	≥30	Total
Ag-RDT pos / culture pos	13	24	12	0	0	49
Ag-RDT pos / culture neg	0	1	1	1	0	3
Ag-RDT neg / culture pos	0	0	2	0	0	2
Ag-RDT neg / culture neg	0	0	0	2	5	7
	13	25	15	3	5	61

Performance of cell culture-based detection of SARS-CoV-2 compared to rRT-PCR

In parallel to the SARS-CoV-2 Ag-RDT, we also performed cell culture-based detection of SARS-CoV-2 of the 61 rRT-PCR positive NPS samples. In order to confirm that the observed CPE in the Vero E6 cell cultures is induced by SARS-CoV-2, we used an immunofluorescent readout based on the SARS-CoV-2 nucleocapsid antigen [18]. This revealed that productive SARS-CoV-2 infection/replication was detected in 51 out of the 61 samples, resulting in an overall sensitivity of 83.6%. (95%CI, 71.9-91.8%) (Table 1). From the 10 NPS samples that did not show any SARS-CoV-2 antigen-positive fluorescent signal, nine had an E gene Ct-value >26, whereas one had an E-gene Ct-value of 21 (Suppl. Table 1). A repeated culture attempt from leftover material stored at -80°C confirmed this negative result. When we, similar to the Ag-RDT, only consider samples with an E-gene Ct-value of <25, <28 or <30, the overall performance of the cell culture based-detection compared to PCR were 37/38, 51/53 and 51/56, which translates into in a sensitivity of 97.3% (95%CI, 86.1-99.9%), 96.2% (95%CI, 87.8-99.5%) and 91% (95%CI, 80.3-97%), respectively (Table 1). This indicates that SARS-CoV-2 can be readily be isolated on Vero E6 cells from samples with a Ct-value lower than 28.

Comparison of Ag-RDT and cell culture-based detection of SARS-CoV-2

After the individual performance comparison from the Ag-RDT and cell culture-based SARS-CoV-2 assay to the rRT-PCR diagnostic test, we also compared the results of the Ag-RDT and cell culture-based assay with each other. This revealed that 49 of the 61 NPS samples were tested positive in both assays, whereas seven samples (Ct-value >28) tested negative in both the Ag-RDT and cell culture-based SARS-CoV-2 assays (Table 1 and Suppl Table 1). In contrast, five samples showed a discrepancy in the outcome between both assays. Namely, two NPS with an E-gene Ct-values of 25.4 and 26.0, respectively, were only found positive in the cell culture-based assay (Fig. 1 ), whereas three samples with an E-gene Ct-values of 21.0, 26.3 and 28.4, respectively, were only found positive with the Ag-RDT. Nonetheless, we found that there was an overall good agreement between the Ag-RDT and cell culture-based SARS-CoV-2 detection assay. This was corroborated by the Cohen's weighted kappa value of 0.69 (95% CI 0.43-0.94). In Fig. 2 , Ag-RDT and cell culture results are plotted against E-gene and RdRp Ct-values by rRT-PCR. Of note is that CT-values for E-gene and RdRp are almost overlapping (for more details see Suppl Table 1).

Fig. 1

Immunofluorescence staining of Vero E6 cells 5 days after inoculation with sample 49 that tested negative by Ag-RDT. Cells were fixed in 4% neutral buffered formalin solution, stained using a rabbit polyclonal anti-SARS-CoV nucleocapsid protein, and examined under a fluorescent immunofluorescence microscope. Scale bar is 50 μm.

Fig. 2

Results of Ag-RDT and cell culture-based assay are plotted against RdRp and E-gene Ct-values of rRT-PCR positive samples.

Discussion

In the current study we used rRT-PCR positive NPS samples to evaluate the sensitivity of the Roche Ag-RDT and cell culture-based isolation of SARS-CoV-2 to that of a rRT-PCR diagnostic test using matched clinical specimens.

We successfully isolated SARS-CoV-2 from 51 of 61 rRT-PCR positive NPS samples, which translates to an overall sensitivity of 83.6%. This further increases to 96.2% when only considering rRT-PCR positive samples with a Ct-value for the viral E-gene <28. Nevertheless, no SARS-CoV-2 could be isolated from two specimens with E-gene Ct-value <28, of which one even had a Ct-value of 21.0. The reason for this discrepancy remains unclear but is in line with other studies that describe some rare failure of SARS-CoV-2 isolation from clinical material with a low rRT-PCR Ct-value [[19], [20], [21], [22], [23]]. In contrast to some of those studies, we did not isolate SARS-CoV-2 from clinical samples with E gene Ct-values above 28. This might be due to the general low successful rate of isolating SARS-CoV-2 from clinical samples with a high Ct-value [[20], [21], [22]]. Moreover, it is also important to mention that in the current study only a limited number of NPS samples with an E gene Ct-value >28 were included, and therefore the success rate might further increase when a large number of samples is tested.

Nowadays, there are several studies that have evaluated the sensitivity of SARS-CoV-2 isolation in comparison to rRT-PCR detection. However, because different types of transport media, cell lines, storage conditions, inoculation (e.g., on tubes or plates, with and without centrifugation after inoculation), and confirmation methodologies were used, it is difficult to make a direct comparison between the studies [20,21,[23], [24], [25], [26], [27], [28]]. Nonetheless, despite these intrinsic differences our cell culture-based SARS-CoV-2 detection results are in line with other studies, which all demonstrate that the success rate of SARS-CoV-2 isolation is related to the viral load/Ct-value within a clinical specimen [[19], [20], [21], [22],[24], [25], [26], [27], [28], [29]]. Because cell culture-based virus isolation methods are suited to discern between non- and infectious virus it has been suggested that a certain rRT-PCR Ct-value cut-off potentially could be used as a surrogate to define when a patient with COVID19 can be released from isolation [20,25,30]. However, due to the reported differential Ct-value cut-offs (ranging from 24 to 34) between infectious and non-infectious SARS-CoV-2 [19,20,22,24] it will be necessary to standardize and potentially improve the detection sensitivity of SARS-CoV-2 cell culture-based diagnostic assays prior to implementing such cut-offs in practice.

Using matching NPS samples we found that the commercially available Roche Ag-RDT in comparison to the rRT-PCR has an overall sensitivity of 85.2%, which is in good agreement with other studies that report a sensitivity between 68.8% and 92.9% for the Roche Ag-RDT [21,[31], [32], [33], [34], [35], [36], [37], [38]]. However, due to the non-homogenous sample distribution, the use of a specific Ct-value cut-off (e.g., <25) allows a better comparison between the different studies. For example, using an E-gene <25, <28 and <30 the sensitivity is 100%, 96.2% and 92.9%, respectively, which are comparable to the 96.6 - 99.1% (Ct-values <25) [21,[34], [35], [36]], 100% (Ct-values <28) [31], and 94.3% (Ct-values <30) [21] reported previously. Important to note is that in our study we did not detect any false positive, although this might be due to the small sample number as other studies that analysed more samples reported a specificity ranging between 92 and 100% for the Roche Ag-RDT [21,[31], [32], [33], [34], [35], [36], [37], [38]]. False-positive Ag-RDT results have also been reported for other commercially available tests [39] and could potentially lead to unnecessary isolation. However, this problem seems to be rare using the Roche Ag-RDT, and is outweighed by the benefits of low costs and rapidly screening and identifying individuals with high viral load among large populations.

Because we used matching NPS samples to evaluate the performance of either the Roche Ag-RDT and the cell culture-based isolation of SARS-CoV-2 with the rRT-PCR diagnostic assay, we can also make a pairwise comparison between the commercial antigen-based assay and our in-house cell culture-based SARS-CoV-2 assay. Hereby we found a good correlation between Ag-RDT positivity and the success rate of detecting viral replication in Vero E6 cells. With the exception of a single sample, all other specimens with a rRT-PCR Ct-value <25 were found to be positive in both assays. These results are consistent with recent studies that used a similar experimental setup and combined demonstrate that there is a strong relationship between a positive Ag-RDT result and the detection of infectious SARS-CoV-2 in cell culture when testing clinical samples with a high viral load [21,22,27,29,39]. It is important to note that it is possible that infectious virus can be detected even when the Ag-RDT result is negative. Such cell culture positive and Ag-RDT negative samples were in our study only detected in 2 out of the 51 samples, and despite methodological differences, is also rarely observed in other studies [21,22,29,39]. Whether this is due to the detection sensitivity or incompatibility between antibody and antigen in the Ag-RDT remains to be determined.

Although we used matched clinical specimens, we only evaluated the performance of diagnostic tests in this study. Therefore, we cannot measure or determine the influence of pre-analytics, the presence or absence of symptoms and underlying diseases, and the duration of clinical symptoms on the sensitivity on the outcome of the SARS-CoV-2 Ag-RDT and cell culture-based diagnostic assay, and would require a larger prospective study.

Combined, our results demonstrate that the performance of the commercially available Roche Ag-RDT assay and our in-house cell culture-based assay to detect SARS-CoV-2 are in good agreement. This supports the use of the Roche Ag-RDT to detect SARS-CoV-2 exposure in large scale populations (i.e., school, airport, military) with limited logistics effort at rather low cost. However, a negative Ag-RDT result does not mutually exclude the presence of infectious SARS-CoV-2 in a clinical specimen. It is therefore recommended to use the rRT-PCR diagnostic method, potentially in conjunction with cell culture-based SARS-CoV-2 detection assay, to support clinicians in making decisions regarding fragile patient groups that undergo or require medical intervention in course of their diagnostic pathway.

Funding

This research did not receive any specific funding.

Declaration of Competing Interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.