Evaluation of a SARS-CoV-2 rapid antigen test: Potential to help reduce community spread?

BACKGROUND: Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) can spread from symptomatic patients with COVID-19, but also from asymptomatic individuals. Therefore, robust surveillance and timely interventions are essential for the control of virus spread within the community. In this regard the frequency of testing and speed of reporting, but not the test sensitivity alone, play a crucial role.
OBJECTIVES: In order to reduce the costs and meet the expanding demands in real-time RT-PCR (rRT-PCR) testing for SARS-CoV-2, complementary assays, such as rapid antigen tests, have been developed. Rigorous analysis under varying conditions is required to assess the clinical performance of these tests and to ensure reproducible results.
RESULTS: We evaluated the sensitivity and specificity of a recently licensed rapid antigen test using 137 clinical samples in two institutions. Test sensitivity was between 88.2-89.6 % when applied to samples with viral loads typically seen in infectious patients. Of 32 rRT-PCR positive samples, 19 demonstrated infectivity in cell culture, and 84 % of these samples were reactive with the antigen test. Seven full-genome sequenced SARS-CoV-2 isolates and SARS-CoV-1 were detected with this antigen test, with no cross-reactivity against other common respiratory viruses.
CONCLUSIONS: Numerous antigen tests are available for SARS-CoV-2 testing and their performance to detect infectious individuals may vary. Head-to-head comparison along with cell culture testing for infectivity may prove useful to identify better performing antigen tests. The antigen test analyzed in this study is easy-to-use, inexpensive, and scalable. It can be helpful in monitoring infection trends and thus has potential to reduce transmission.

Introduction

Since the beginning of COVID-19 outbreak in December 2020, the global demand for the Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) testing has been steadily increasing. Already back in March 2020, hospitals and laboratories around the world announced their concerns about reagent, consumable material shortages, and limited personal protective equipment. Yet, timely detection and isolation of SARS-CoV-2 infected cases and identification of their contacts are pivotal to slowing down the pandemic.

The main public health strategy during a pandemic relies on robust and easy to perform diagnostic tools that can be used to test large number of samples in a short time. To date the gold standard diagnostic method for SARS-CoV-2 detection [1] is based on real time reverse transcription-PCR (rRT-PCR) technology which has been promptly implemented by the World Health Organization (WHO) [2], Center for Disease Control and Prevention (CDC) [3] protocols, and a number of commercial assays [4]. The SARS-CoV-2 rRT-PCR has high specificity and sensitivity [5,6]. However, the type and quality of the patient specimen [7,8], stage of the disease, and the degree of viral replication and/or clearance have an impact on the test outcome [9]. These factors are critical not only for PCR-based but also for other diagnostic test systems aiming to detect the presence of the virus. Hence interpreting a test result for SARS-CoV-2 depends on the accuracy of the test, but the prevalence and the estimated risk of disease before testing should also be taken into consideration.

In many countries SARS-CoV-2 testing is extended to asymptomatic population, e.g. in schools, airports, nursing-homes, and workplaces. This leads to a growing gap between the large number of demand and the laboratory capacities to preform rRT-PCR tests, especially in developing countries. Despite high specificity and sensitivity, rRT-PCR has a disadvantage in point of care testing, because it usually requires professional expertise, expensive reagents and specialized equipment. Therefore, alternative assays, such as rapid antigen detection tests, which can also detect the presence of the virus directly in respiratory samples, have been developed [4] and tested by different groups [[10], [11], [12], [13], [14]]. However, it is vital to determine the sensitivity, specificity of such tests relative to standard rRT-PCR in order to identify the ideal circumstances that their application would be beneficial.

This study was performed to evaluate a novel antigen test produced by R-Biopharm for the detection of SARS-CoV-2 in different specimens and to identify its limitations and potential usage. Different types of materials and verification analysis were used by two institutions independently to assure the reproducibility of the testing and to analyze the potential caveats.

Materials and methods

Specimen collection

At the Institute of Virology, Charité Berlin stored specimens taken after routine diagnostic were used with no extra procedures required for the study. Cell culture supernantants of respiratory viruses other than SARS-CoV-2 were available at the Institute of Virology, Charité through a EVD-LabNet EQA (https://www.evd-labnet.eu/) [25].

At the Institute of Virology, Frankfurt, the clinical samples were collected from subjects as part of registered protocols. Combined oropharyngeal/nasal swabs were collected, stored in 2 ml PBS at 4 °C and processed for further analysis within 24 h.

Cell culture and virus stocks

Caco-2 (human colon carcinoma) were cultured in Minimum Essential Medium (MEM) supplemented with 10 % fetal calf serum (FCS), 100 IU/mL of penicillin and 100 g/mL of streptomycin. All culture reagents were purchased from Sigma (St. Louis. MO. USA). The Caco-2 cells were originally obtained from DSMZ (Braunschweig, Germany, no.: ACC 169), differentiated by serial passaging and selected for high permissiveness to virus infection. Caco-2 cells were infected with different viral isolates (FFM1-FFM7) [15] at an MOI 0.1. Cell culture supernatant was harvested 48 h after infection, precleared at 2000×g for 10 min at room temperature. Aliquots of virus particle containing supernatant were kept at −80 °C.

Detection of infectious virus in cell culture

Of the swab-dilution, 500 μL were mixed with 1.5 ml of MEM containing 1% FCS (Sigma-Aldrich; St. Louis, Missouri, USA), 7.5 μg/mL Amphotericin B, and 0.1 mg/mL Primocin, (InvivoGen; San Diego, California, USA). Swab-inoculums were transferred to Caco-2 cells seeded in 5.5 cm2 culture tubes. Cytopathogenic effect (CPE) was assessed daily for up to seven days or until cell lysis occurred.

Rapid antigen test

Rapid antigen test was provided by R-Biopharm. Test was performed according to the manufacturer’s recommendations and evaluated visually by four or six-eye principle. Briefly, samples were vortexed for 20 s. 50 μl from Solution A (blue) and B (yellow) were dispensed in clean 1.5 ml reaction tubes which leads to green coloring. Immediately 50 μl of the test samples were added to the reaction mixture, mixed briefly and incubated for 10 min at room temperature. Test strips were placed into the mixture vertically to allow absorption. Test results were evaluated after 10 min. Intensities of the test bands were compared to control bands and categorized as follows: +++ (test band intensity stronger than the control), ++ (test and control bans intensity are similar), + (test band intensity is weaker than the control). Antigen testing for viable SARS-CoV-2 and SARS-CoV-1 cell culture supernatants was performed in a BSL-3 laboratory.

RNA extraction and rRT-PCR analysis

At the Institute of Virology, Charité Berlin, stored samples (swab resuspended in 1.5 mL of phosphate-buffered saline) were anonymized before testing. After thawing at room temperature all samples were analyzed by antigen test and rRT-PCR in parallel. RNA extraction for rRT-PCR was done by using the MagNA Pure 96 system, using 100 μl of sample, eluted in 100 μl. The rRT-PCR was performed as published previously [1].

At the Institute of Virology in Frankfurt the SARS-CoV-2 rRT-PCR analysis (Cobas, Roche, Basel, Switzerland) was performed on the automated Cobas 6800 system. Of the swab-dilution, 1000 μl aliquots were mixed with lysis buffer (1:1 ratio) and 500 μL aliquots were transferred to barcoded secondary tubes, loaded on the Cobas 6800 system, tested with Cobas SARS-CoV-2 master mix containing an internal RNA control, and primer-probe sets towards ORF1 and E-gene according to the manufacturer’s instructions.

Within seven days of virus inoculation using clinical sample material, culture supernatant was collected to perform rRT-PCR in order to confirm productive virus replication. RNA was isolated from 100 μL cell culture supernatant using the QIAcube HT instrument and QIAamp 96 Virus QIAcube HT Kit (Qiagen; Hilden, Germany) according to the manufacturer’s instructions. SARS-CoV-2 RNA was analyzed by rRT- PCR using the Luna Universal One-Step RT-qPCR Kit (New England Biolabs; Ipswich, Massachusetts, USA) and primers targeting RNA-dependent RNA polymerase (RdRp) as described [15]. RdRP_SARSr-F2 GTGARATGGTCATGTGTGGCGG), RdRP_SARSr-R1 CARATGTTAAASACACTATTAGCATA.

Statistical analysis

The number of positive samples were compared using two by two contingency table. The agreement between the antigen test and rRT‐PCR techniques was evaluated using the Cohen’s weighted kappa index (K value) [16]. K value interpretations were categorized as follows: <0.20 is poor, 0.21−0.40 is fair, 0.41−0.60 is moderate, 0.61−0.80 is substantial, and 0.81–1.00 is almost perfect agreement [17].

Ethical statement

The use of stored clinical samples for validation of diagnostic methods without person related data is covered by Section 2.5 of the Berlin hospital law and does not require ethical or legal clearance.

The use of anonymized clinical samples for validation of diagnostic methods does not require ethical clearance by the Goethe University, Frankfurt.

Results

Rapid antigen test sensitivity and specificity were evaluated by two independent institutions using various number of clinical samples. rRT-PCR was used as a reference test system. We deemed individuals to be uninfected with SARS-CoV-2 when a negative result was obtained by rRT-PCR.

In the Institute of Virology, Charité, Berlin, a total of 67 stored patient samples were available for the study. Of these, 58 were rRT-PCR positive with cycle threshold (cT) range between 18.77–40 corresponding to 2.5 × 109 -1380 RNA copies/mL (Table S1), representing 86.6 % (58/67) of the clinical samples analyzed (Fig. 1 A). When the rRT-PCR results were used as a reference, the antigen test diagnosed SARS-CoV-2 infection status with a sensitivity of 77.6 % (45/58) and a specificity of 100 % (9/9) (Table 1 ). After re-evaluating the data based on the acceptable analytic sensitivity and limit of detection suggested by WHO [18], we identified 48 samples with ≥106 RNA copies/mL. Rapid antigen test performed with 89.6 % sensitivity for this sample set (Table 1). Of these, 40 samples had ∼2.23 × 106 or more RNA copies/mL and reacted positive with the antigen test)Table S1). In contrast samples with less than 7.63 × 105 RNA copies/mL were negative (Fig. 1A, Table S1). Cohen’s weighted kappa value of 0.482 indicated moderate agreement between the rRT-PCR and the rapid antigen test (Table 2 ). The overall concordance between the rRT-PCR and the antigen test was 80.6 % (54/67).

Fig. 1

Antigen test analysis performed in Berlin (A) and Frankfurt (B). A. Log10 RNA copies/mL and corresponding antigen (Ag) detection test results (red circles positive n: 45, blue circles negative n: 13) for each rRT-PCR positive sample (n: 58). B. Cycle threshold (cT) value and corresponding antigen (Ag) detection test results (red circles positive n: 16, blue circles negative n: 16) for each rRT-PCR positive sample (n: 32). 32 rRT-PCR positive samples were tested in cell culture for infectivity. All Ag-test positive (n:16, red circles) and three Ag-test negative (red-filled blue circles) samples displayed CPEs after inoculating in Caco-2 cells (Table S2). Intensities of the test bands were compared to control band and designated as follows: +++ (test band intensity stronger than the control), ++ (test and control bans intensity are similar), + (test band intensity is weaker than the control).

Table 1

Sensitivity and specificity of the antigen detection test in comparison to rRT-PCR.

		Antigen test
		Negative	Positive	Marginal row	Sensitivity (%)	Specificity (%)
rRT-PCR	Negative	9	0	9		100
	Positive	13	45	58	77.6
genome copies/mL	≥1.09 x 106	5	43	48	89.6
	1.38 x 103–9.78 x 105	8	2	10	20

Table 2

Cohen’s weighted kappa coefficient between rapid antigen test and rRT-PCR.

	Antigen test
rRT-PCR	Negative	Positive	Row marginal
Negative	9	0	9 (13.4%)
Positive	13	45	58 (86.5%)
Column marginal	22 (32.8%)	45 (67.2%)	67
Weighted Kappa	0.482
Standard error	0.110
95% CI	0.266 to 0.698

Certain rapid tests may be used at the point-of-care and thus offer benefits for the detection and management of infectious diseases. In order to assess the potential of the rapid antigen test in this context, 70 nasopharyngeal samples freshly collected from individuals living in a shared housing were analyzed head to head by rRT-PCR using Cobas 6800 system, rapid antigen test, and cell culture using Caco-2 cells to determine the infectivity (Institute of Medical Virology, Goethe University, Frankfurt). 45.7 % (32/70) of the clinical samples were diagnosed positive for SARS-CoV-2 by rRT-PCR with cT values ranging between 18.01–35.98 (Fig. 1B, Table S2). The antigen test diagnosed the infection status with a sensitivity of 50 % (16/32) and a specificity of 100 % (Table 3 ). Re-evaluating the data based on the limit of detection, sensitivity was determined to be 88.2 % for samples with cT values <28, and it was reduced in the group of samples with cT values ≥28 (6.7 %) (Table 3). Cohen’s weighted kappa value of 0.521 indicated moderate agreement between rRT-PCR and the rapid antigen test (Table 4 ). The overall concordance between the rRT-PCR and the antigen test was 77.1 % (54/70) (Table 4).

Table 3

Comparison of the clinical diagnostic performance of rapid antigen test with rRT-PCR.

		Antigen test
		Negative	Positive	Marginal row	Sensitivity (%)	Specificity (%)
rRT-PCR	Negative	38	0	38		100
	Positive	16	16	32	50
cT<28	Positive	2	15	17	88.2
cT≥28	Positive	14	1	15	6.7

Table 4

Cohen’s weighted kappa coefficient between rapid antigen test and  rRT-PCR.

	Antigen test
rRT-PCR	Negative	Positive	Row marginal
Negative	38	0	38 (54.3%)
Positive	16	16	32 (45.7%)
Column marginal	54 (77.1%)	16 (22.9%)	70
Weighed Kappa	0.521
Standard error	0.092
95% CI	0.339 to 0.702

rRT-PCR is a highly sensitive method to detect viral RNA molecules from clinical samples. However, viral RNA can persist in different body parts and can be detected in specimens for much longer than the presence of viable virus [19]. Thus demonstration of infectivity on permissive cell lines in vitro is a more reliable surrogate for infectivity and virus transmission. Therefore, we attempted virus isolation by inoculating rRT-PCR positive samples in Caco-2 cells. Cytopathic changes were monitored daily by microscopy for a week and subsequently aliquots of culture supernatant were tested to verify viral RNA copies (Table S2). For samples that are positive for both antigen test and rRT-PCR (16/32, cT 18.01–28.45), we observed cytopathic effects (CPE) in cell culture 1–3 days after inoculation (Fig. 1B, Table S2). Three samples that had a negative result in the antigen test, but were positive by rRT-PCR (cT values 26.69, 30.12, and 32.13) displayed CPE as well. Other 13 antigen-test negative samples with higher cT values (indicating lower viral load) between 28.34–34.12 were not infectious in cell culture. Interestingly, one sample with a relatively low cT value 25.53, did not show any CPE in cell culture and was also negative for the antigen test (Table S2).

In order to investigate potential cross reactivity among common coronaviruses and other respiratory viruses, infectious and heat inactivated (4 h at 60 °C) cell culture supernatants were tested (Table 5 ). SARS-CoV-1 and SARS-CoV-2 tested positive with the antigen test, as expected. The antigen test did not display any cross-reactivity with the other respiratory and endemic corona viruses listed in Table 5.

Table 5

Rapid Antigen Test results using different respiratory virus cell culture supernatant stocks.

Cell culture supernatant with virus	Antigen-Test
SARS-CoV-1	+
SARS-CoV-2	+
HCoV-229E	–
HCoV-NL63	–
MERS-CoV	–
Enterovirus	–
Rhinovirus	–
Parainfluenzavirus 1	–
Parainfluenzavirus 2	–
Parainfluenzavirus 3	–
Parainfluenzavirus 4	–
hMPV A	–
hMPV B	–
RSV	–
Influenzavirus A H1N1	–
Influenzavirus A H3N2	–
Influenza B	–

We further evaluated the detection sensitivity among different SARS-CoV-2 isolates. Here we used cell culture supernatant collected from Caco-2 cells infected with seven different isolates [15] and SARS-CoV-1 (Fig. 2 ). The virus stocks were thawed at room temperature and a total of six 10-fold dilutions were prepared in PBS. The antigen test was performed and evaluated immediately (Fig. 2A). In parallel, aliquots of the dilutions were mixed with lysis buffer used for RNA extraction to inactivate the virus. rRT-PCR was performed for two different gene targets ORF1 and E-gene that resulted in similar cT values (Fig. 2B, Table S3). 10-fold serial dilutions led to ∼3 cT difference in rRT-PCR for each set as anticipated. According to our results the limit of detection was between 100–560 RNA copies/mL which is in line with the manufacturer’s findings. We previously identified RG203KR mutations in FFM3, FFM4 and FFM6 and S→L mutation in FFM1 within the nucleocapsid protein coding region [15]. According to GISAID classification the GR clade, carrying the combination of Spike D614G and nucleocapsid RG203KR mutations, is currently the most common representative of the SARS-CoV-2 population worldwide [20]. Our results suggest that the presence of the RG203KR mutation did not interfere with the antigen test performance.

Fig. 2

Rapid Antigen Test Results for SARS-CoV-1 and SARS-CoV-2 isolates. A. Representative lateral flow assay using serially diluted virus stocks. Intensities of the test bands were compared to control band and designated as follows: +++ (test band intensity stronger than the control), ++ (test and control bans intensity are similar), + (test band intensity is weaker than the control). B. TCID50/mL values and corresponding antigen (Ag) detection test intensity for serially diluted SARS-CoV-2 isolates FFM1-7 and SARS-CoV-1 are shown. Representative result of two experiments.

Discussion

In this study we validated the assay performance of a recently approved rapid antigen test in two independent institutions using a total of 137 clinical samples. Although the test specificity was 100 % for this particular sample set, overall sensitivity was low (50–77.6 %), yet re-analyzing samples with higher viral loads, typically observed during early stages of the infection, showed good correlation (88.2–89.6 %). Previous studies reported that lower cT values are associated with higher viral culture positivity [21,22]. There is currently no direct evidence whether cell culture positivity or higher viral load correlates with contagiousness of an individual, however, it is commonly recognized as the surrogate of infectivity [23]. Since an important aspect of using point-of-care testing is to able to identify infected individuals who are infectious and can potentially transmit the virus, we performed correlation analysis within a group of clinical samples tested. 19 out of 32 SARS-CoV-2 infected individuals were positive in cell culture. The antigen test detected 16 out of 19 these (84 %). In contrast 43.7 % (14/32) of the samples were not infectious in cell culture, yet positive by rRT-PCR, probably due to persisting genomic and subgenomic viral RNA within the collected sample. We detected an excess amount of viral RNA in cell culture supernatants due to high replication capacity of the virus in permissive cells, despite a negative antigen test result. This might explain the cT discrepancy between the cell culture supernatant and clinical samples. Small sample size is the major limitation of this study. Future efforts should aim to monitor frequent sampling of larger groups and to compare different rapid antigen tests, different sampling sites along with infectivity correlation in cell culture.

Our results suggest that the rapid antigen test can detect SARS-CoV-2 infected individuals with high viral loads and has potential in determining highly contagious individuals. Despite low analytic sensitivity, rapid antigen tests are inexpensive and therefore can be used frequently for detecting infected individuals who are asymptomatic, pre-symptomatic and without known or suspected exposure to SARS-CoV-2 [24]. They can be beneficial in congregate settings, such as a long-term care facility or a correctional facility, workplace, or a school testing its students, faculty, and staff. Rapid antigen tests likely perform best during the early stages of SARS-CoV-2, when the viral loads are higher.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.