Evaluation of the Roche-SD Biosensor rapid antigen test: Antigen is not reliable in detecting SARS-CoV-2 at the early stage of infection with respiratory symptoms.

We evaluated a rapid antigen test against SARS-CoV-2 virus (Roche-SD Biosensor; RSDB-RAT) in children and adults with respiratory symptoms compared to those with nonrespiratory symptoms or asymptomatic. Also the performance of RSDB-RAT with respect to the duration of respiratory symptoms was assessed. A viral cross-reactivity panel was included. RSDB-RAT was reliable in detecting SARS-CoV-2 in children and adults if the respiratory symptoms had endured 1 to 7 days. If the respiratory symptoms had lasted less than 1 day, the sensitivity was significantly lower. No cross-reactivity with other respiratory viruses was observed.

Introduction

The laboratory diagnostics of coronavirus disease 2019 (COVID-19) caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) rests on nucleic acid amplification tests (NAATs) based mainly on reverse-transcriptase polymerase chain reaction (RT-PCR), which is the most sensitive and highly specific golden standard method. However, its limitations in the point-of-care (POC) testing are a relatively long turn-around-time, high test cost per sample, and the need for specialized equipment. Rapid antigen tests (RATs), in turn, are less sensitive, but usually faster and cheaper than NAATs. (European Centre for Disease Prevention and Control 2020; Vandenberg et al., 2021)

According to the guidelines, the use of RATs is acceptable in certain conditions, if the NAAT test is not reasonably feasible (European Centre for Disease Prevention and Control 2020; Hanson et al., 2021; WHO, 2020). The European Centre for Disease Prevention and Control (ECDC) guideline allows the use of RATs even in pre-symptomatic and early symptomatic phase up to 5 days from the onset of symptoms, the guideline being exceptionally more front-loaded in the testing than the others (European Centre for Disease Prevention and Control 2020). According to the World Health Organization (WHO), the testing should be conducted within the first 5 to 7 days after the onset of symptoms (WHO, 2020). The Infectious Diseases Society of America (IDSA) guideline concluded that RATs should be used within 7 days after symptom onset (Hanson et al., 2021). They also concluded that the RATs perform equally in adults and children, but in children the data is still very limited (Eleftheriou et al., 2021; Jung et al., 2021; L'Huillier et al., 2021). The guidelines also allow the use of RATs in screening of asymptomatic cases in certain conditions like in high prevalence of COVID-19 or in identifying highly infectious cases (European Centre for Disease Prevention and Control 2020; Hanson et al., 2021; WHO, 2020).

We evaluated the performance of Roche-SD Biosensor SARS-CoV-2 (RSDB) RAT both in children and adults with respiratory symptoms compared to other cases with non-respiratory symptoms or asymptomatic, and to the duration of the respiratory symptoms. A cross-reactivity panel containing positive samples regarding other respiratory viruses was included in the material. The test was reliable in detecting SARS-CoV-2, if the respiratory symptoms had endured 1 to 7 days. However, if the respiratory symptoms had lasted less than 1 day, the sensitivity was significantly lower. No cross-reactivity with other respiratory viruses was observed.

Material and methods

The study material comprised of routine nasopharyngeal samples of total 238 persons [aged 26 years (mean, range 0.5–90 years); 112 males]. Samples of 172 persons mainly of African ethnicity [aged 22 years (mean, range 0.5–90 years); 81 males] had been collected in Fimlab Laboratories for SARS-CoV-2 NAAT testing during a COVID-19 outbreak in Central Finland in November 2020. Of these samples, 38 were COVID-19 NAAT positive and 134 negative. During the same month, the material was supplemented with consecutive routine samples from 66 other COVID-19 cases mainly of Finnish ethnicity referred to Fimlab Laboratories for sampling [aged 37 years (mean, range 1.8–88 years); 31 males]. From all COVID-19 NAAT positive cases, the onset time of possible respiratory symptoms was known.

Primary COVID-19 diagnosis had been based on Cobas® SARS-CoV-2 (Roche Diagnostics International AG, Rotkreuz, Switzerland; N = 211), AllplexTM 2019-nCoV (Seegene Inc., Seoul, South Korea; N = 26), or Abbott RealTime SARS-CoV-2 (Abbott Laboratories, Illinois, U.S.A; N = 1) NAAT assays.

The NAAT samples had been collected to 2 mL VACUETTE® Virus Stabilization Tube (Greiner Bio-One GmbH, Kremsmünster, Austria), and the Roche-SD Biosensor SARS-CoV-2 rapid antigen test [manufactured by SD Biosensor (Republic of Korea) and distributed by Roche Diagnostics (Mannheim, Germany); referred as RSDB-RAT in the text] was done from the residual sample after NAAT testing. RSDB-RAT is an immunochromatographic lateral flow cassette test for the qualitative detection of SARS-CoV-2 nucleocapsid (N) antigen from the nasopharyngeal sample. The test is interpreted visually within 15 to 30 minutes after sample application. All RATs were completed not later than 48 hours after sample collection. For the detection of possible cross-reactions in the RAT, respiratory samples positive for other viruses than SARS-CoV-2 from 56 cases [aged 43 years (mean, range 0.4–90 years); 26 males; Finnish ethnicity] collected to eSwab® (Copan, Italy) were included in the study material. The samples had been collected before the COVID-19 pandemic or tested negative for SARS-CoV-2, and stored at -70°C.

The study was based on a standard clinical validation procedure from the residual SARS-CoV-2 samples of the test intended for clinical use in the laboratory, and the approval of the ethical committee was not required. The statistical analyses were performed with IBM® SPSS® Statistics software Version 26 and www.medcalc.org free statistical calculators. The proportions were compared with 2-tailed Fisher's exact test and the cycle threshold (Ct) value levels with Mann-Whitney U test. The positive and negative predictive values (PPV and NPV, respectively) were calculated with different assumed COVID-19 prevalence (2% and 5%) in the population to demonstrate the performance of the test in different epidemic situations.

Results

The sensitivities and specificities of the RSDB-RAT in different case groups are shown in Table 1 . The overall specificity was 99.3%, and the only false positive result was detected in an adult male. The sensitivity of the test was significantly higher among all COVID-19 cases including children whose respiratory symptoms had endured 1-7 days (94.6%) before sampling compared to those with the duration of less than 1 day [64.3%, confidence interval (CI) 35.1-87.2%; P = 0.007], or to those with non-respiratory symptoms or whom were asymptomatic (77.4%; P = 0.031).

Table 1

Sensitivities and specificities of the Roche-SD Biosensor SARS-CoV-2 rapid antigen test in different case groups.

Antigen test resulta	COVID-19 NAAT test result		Sensitivity (%)	Spesificity (%)	PPV (%) (COVID-19 prevalence 2/5%)	NPV (%) (COVID-19 prevalence 2/5%)
	Positive	Negative
All cases
Positive	88	1	84.6(CI: 76.2−90.9)	99.3(CI: 95.9−100)	69.8/85.7	99.7/99.2
Negative	16	133
All respiratory cases
Positive	64		87.7(CI: 77.9−94.2)		70.6/86.1	99.8/99.4
Negative	9
Respiratory symptoms 1-7daysbefore sampling
Positive	52		94.6(CI: 84.9−98.9)		72.1/87.0	99.9/99.7
Negative	3
Respiratory symptoms <1 day before sampling
Positive	9		64.3(CI:35.1−87.2)		63.7/81.9	99.3/98.1
Negative	5
Nonrespiratory cases (other symptoms or asymptomatic)
Positive	24		77.4(CI: 58.9−90.4)		67.9/84.5	99.5/98.8
Negative	7

Roche SARS-CoV-2 Rapid Antigen Test; NAAT, nucleic acid amplification test; PPV, positive predictive value; NPV, negative predictive value; CI, 95% confidence interval.

The positive (PPV) and negative (NPV) predictive values with different assumed COVID-19 prevalence in the population are also shown in the table. PPVs and NPVs are calculated against the test´s overall specificity.

The majority of the false negative RAT results occurred in the cases where the respiratory symptoms had continued less than 1 day before sampling (Fig. 1 ). When estimating the analytical sensitivity of RSDB-RAT against NAAT test, 2 samples gave a false RAT negative result with the NAAT mean Ct values of 25.18 and 29,47, respectively. The first case had had respiratory symptoms less than 1 day and the other was asymptomatic. Otherwise, the NAAT Ct values of all other RAT false negative samples (N = 14) were above 30. When comparing the Ct values (Cobas®; N = 75) of positive NAAT samples between different case groups, no statistically significant difference between children and adults, nor between respiratory patients and the others (asymptomatic or non-respiratory symptoms) was detected.

Fig. 1

Roche-SD Biosensor SARS-CoV-2 rapid antigen test (RSDB-RAT) results of the COVID-19 cases with respiratory symptoms compared to positive NAAT results as a function of time. Ct, cycle threshold; ○ Roche Cobas® SARS-CoV-2 NAAT assay; ∆ Seegene AllplexTM 2019-nCoV NAAT assay; fulfilled symbols (●▲), RSDB-RAT negative.

The study population contained 84 children under the age of 15 years (median age 10 years, quartiles 5–13 years; 36 boys). Among the 18 NAAT positive children, 6 had had respiratory symptoms 1 to 7 days before sampling, of whom the RAT was positive in all (100.0%). Of the other 12 NAAT positive children, 2 had had respiratory symptoms less than 1 day before sampling, of whom the RAT was positive in none (0%), and 10 had had only nonrespiratory symptoms or they had been asymptomatic, of whom the RAT was positive in 7 (70.0%).

The results of the cross-reactivity testing against respiratory samples containing other viruses than SARS-CoV-2 are shown in Table 2 . No cross-reactivity against other viruses was detected.

Table 2

Cross-reactivity testing for SD Biosensor SARS-CoV-2 rapid antigen test against 56 PCR positive samples containing other respiratory viruses than SARS-Cov-2 to evaluate potential interference.

Possible cross-reactive samplesa	n	SD Biosensor SARS-CoV-2 rapid antigen test positive
Human coronavirus OC43	17	0
Human coronavirus NL63	5	0
Human coronavirus 229E	2	0
Human coronavirus OC43 and human rhinovirusa	1	0
Influenza A virusb	6	0
Influenza A virus and bocavirus	1	0
Influenza B virus	5	0
Respiratory syncytial virus (RSV)c	4	0
RSV A and Parainfluenzavirus 1	1	0
Parainfluenzavirusd	7	0
Rhinovirus	6	0
Enterovirus	1	0

Tested positive by Allplex Respiratory Panel 1 & 3 (Seegene Inc., Seoul, South Korea) or Xpert® Xpress Flu/RSV (Cepheid, Sunnyvale, CA, USA).

All strains were H1N1v type.

One strain was A type and 3 were B type.

Six strains were type 1 and one was type 3.

The samples had been collected before the COVID-19 era or tested negative for SARS-CoV-2, and stored at -70°C before analysis.

Discussion

The RSDB-RAT has been included in the common list of COVID-19 RATs that are considered mutually appropriate for use in context of the situations by the European Commission's (EC) Health Security Committee, the minimum performance requirements of being ≥90% sensitivity and ≥97% specificity compared to NAAT (2021). EC has also purchased RSDB RATs to the European Union member states via Emergency Support Instrument (European Commission - Press release: 2020).

According to our study, the RSDB-RAT was reliable with >94% sensitivity and >99% specificity in detecting SARS-CoV-2 in both children and adults, if the respiratory symptoms had endured 1 to 7 days. In recent Cochrane report, RSDB-RAT's specificity was observed high (>98%) in every case group (Dinnes et al., 2020). In the same report, the overall sensitivity in symptomatic patients was 80%, but if restricted to instructions for use (IFU) compliant studies the sensitivity was 88%. Our sample handling deviated from IFU regarding the sampling tube and the time delay of making the test, but these did not seem to have effect on the results. In contrast, the strength of our study was that the RAT and NAAT had been made from the same sample.

Less information is available of the RAT performance in children. L'Huillier et al. concluded that the sensitivity of RAT in symptomatic children was 73% and peaked at the day 2 after the onset of symptoms (L'Huillier et al., 2021). Eleftheriou et al. observed that the overall sensitivity of RAT in children was 82%, and the sensitivity in symptomatic cases was even >95% (Eleftheriou et al., 2021). In one cohort of symptomatic children, the overall sensitivity of RAT was 88% compared to NAAT (Jung et al., 2021). Thus, our findings in children are in line with current literature, with RATs revealed to be reliable also in children with respiratory symptoms endured 1 to 7 days.

The overall sensitivity of the RSDB-RAT was significantly lower if the respiratory symptoms had lasted less than one day (64%), or the patients had had only non-respiratory symptoms or they had been asymptomatic (77%). According to Cochrane report, the overall sensitivity of the RSDB-RAT in asymptomatic patients was 61%, and in IFU compliant studies 69% (Dinnes et al., 2020). Thus, the decreased sensitivity of the RATs in asymptomatic COVID-19 patients is well known. However, the decreased sensitivity of the RSDB-RAT was more striking in the cases with the duration of respiratory symptoms less than 1 day, since all guidelines consider RATs reliable already at the very early stage of the respiratory symptoms or even before the symptom onset (European Centre for Disease Prevention and Control 2020; Hanson et al., 2021; WHO, 2020, 2021). L'Huillier et al. has observed in children that RAT sensitivity peaks at the day 2 after the onset of symptoms (L'Huillier et al., 2021). Berger et al. has observed in community-based setting that the sensitivity of RAT is over 95% 1 to 5 days after the onset of symptoms, but only 88% at the day of onset (Berger et al., 2021). Thus, we conclude, that if the respiratory symptoms have lasted less than 1 day, RAT is not reliable in detecting SARS-CoV-2, and the RAT should be repeated next day.

Most of the false negative RAT samples gave NAAT Ct values above 30. This finding is in concordance with other literature (Berger et al., 2021). According to WHO, infectiousness is associated with high viral loads resulting in NAAT Ct values below 25 to 30. (WHO, 2020) Based on our findings, the identification of potentially infectious cases by RSDB-RAT is possible in those with respiratory symptoms with timely sampling, but RSDB-RAT is not suitable for detecting cases with low viral loads.

In our study, only one sample was considered as a false positive in RAT, since the more sensitive NAAT result from the same sample had remained negative. The cause of this false positive result is unknown. Corman and colleagues observed that some other factor than the tested cross-reacting pathogens were likely to have caused the false positive signals (Corman et al., 2021). Chaimayo et al. observed that thick and highly viscous mucous may cause false positive results (Chaimayo et al., 2020). Recent study with another RAT showed that the changes in the test buffer pH due to the added sample may cause non-specific interactions between SARS-CoV-2 specific conjugated antibodies and capture antibodies and cause a false positive result (Patriquin et al., 2021).

Cross-reactivities of the RATs with the other respiratory viruses has not been profoundly investigated. SARS-CoV-2 belongs to Betacoronaviruses like HCoV-OC43 and HCoV-HKU1. Thus cross-reactions might be expected especially with these (Huang et al., 2020). Corman et al. tested 100 samples containing other respiratory viruses than SARS-CoV-2, and only one sample with human parainfluenzavirus 3 cross-reacted with RSDB-RAT. However, they tested only one sample containing Betacoronavirus (HCov-OC43) (Corman et al., 2021). In our cross-reactivity testing – including also 18 HCoV-OC43 samples - no cross-reactivity against any of the viruses was detected.

Conclusions

The SARS-CoV-2 testing strategy in developed countries is changing – mainly due to the intensive vaccination campaign - from the era of the epidemic control to the direction of disease diagnostics in individual patients, and the significance of POC and home RATs are increasing. Thus, the detailed knowledge of the RAT performance both in children and adults is crucial. According to our study, RSDB-RAT was reliable in detecting SARS-CoV-2 both in children and adults, if the respiratory symptoms had endured 1 to 7 days. However, if the respiratory symptoms had lasted less than one day, the sensitivity of the RAT was significantly lower. No cross-reactivity of the RSDB-RAT was observed with the samples containing other respiratory viruses.

Acknowledgments

We thank all who have enabled this study and taken part in developing COVID-19 diagnostics in Fimlab Laboratories operation region.

Funding

The study was supported by an institutional grant of Tampere Tuberculosis Foundation (Tampere, Finland). The funder had no role of study design, data collection and interpretation, or the decision to submit the work for publication.

Declaration of competing interest

The authors report no conflicts of interest relevant to this article.

Author contributions

HF has participated in planning the study, collecting the samples, setting up the rapid antigen test, analyzing the samples, and participated in analyzing the results and writing the manuscript; DK has participated in setting up the rapid antigen test and analyzing the samples; BL has set up and evaluated the commercial NAAT methods; TS has participated in writing the manuscript; JA has participated in planning the study, collecting the samples, setting up the rapid antigen test, analyzing the samples, and participated in analyzing the results and writing the manuscript