Performance of SARS-CoV-2 rapid antigen test compared with real-time RT-PCR in asymptomatic individuals.

Screening, testing and contact tracing plays a pivotal role in the control of COVID-19 pandemic. To carry out this strategy it is necessary to increase the testing capacity. Here, we compared a SARS-CoV-2 rapid antigen test (RAT) and RT-PCR in 842 asymptomatic individuals from Tarapacá, Chile. We report a sensitivity of 69.86%, a specificity of 99.61%, PPV of 94.44% and NPP of 97.22% with Ct values (Ct > 27) that were significantly higher among individuals with false-negative RAT. These results support the fact that RAT might have a significant impact in the identification of asymptomatic carriers in areas that lack suitable laboratories to perform SARS-CoV-2 real-time RT-PCR diagnostics, or the results take more than 24-48 hours, as well as zones with high traffic of individuals, such as border/customs, airports, interregional bus, train stations or in any mass testing campaign requiring rapid results.

Introduction

Given the increase in cases of SARS-CoV-2 infections worldwide, there is a need for a reliable rapid diagnostic test in addition to existing gold standard real-time RT-PCR. Rapid antigen tests (RAT) for SARS-CoV-2 can be performed onsite in mass testing, are inexpensive compared to real-time RT-PCR, do not require specific and expensive equipment, and the results are available within 15 min (CDC, 2021), which could serve to evaluate chains of infection and their interruption. A recent meta-analysis revealed that the average sensitivity and specificity of RAT for SARS-CoV-2 were 56.2% and 99.5%, respectively (Dinnes et al., 2020). To date, most of these validations were carried out in symptomatic individuals or using previously collected samples (Cerutti et al., 2020, Kruttgen et al., 2021, Porte et al., 2020, Weitzel et al., 2020, Yamayoshi et al., 2020). In contrast, onsite test validation studies in asymptomatic individuals, to support the use of RAT in mass testing and epidemiological surveillance, are limited (Jakobsen et al., 2021, Mina et al., 2020, Pollock et al., 2021, Schildgen et al., 2021, Toptan et al., 2021). This study performed a mass comparison of RAT and real-time RT-PCR test in asymptomatic individuals from a Chilean region.

Materials and methods

Sample collection

Sample collection was coordinated by a specialized team from SEREMI de Salud Tarapacá. Two nasopharyngeal swabs (NSS) samples from asymptomatic individuals were collected by healthcare workers at Iquique city, Tarapacá Region, Chile between 14–17 January 2021, where the prevalence of SARS-CoV-2 infection was 11% according to data provided by the Chilean Ministry of Health (Minsal, 2021). Taking into consideration that the expected prevalence of positive cases for COVID-19 in asymptomatic individuals can vary from 8–12% and establishing a sampling error of 1.5% and a type I error of 5%, the minimum sample size required was 864 ± 69.7. All participants completed a questionnaire and provided information on demographic characteristics, current and past (14 days) symptoms, and recent exposure to people with COVID-19 (Table 1 ). One swab was immediately tested at the facility using the SARS-CoV-2 RAT (SD Biosensor, Inc. Republic of Korea) and the result was interpreted according to the manufacturer’s guidelines. The second swab was preserved in specimen transport medium and transported at 4 °C to Laboratorio Médico Bioclinic and Hospital Regional de Iquique for RNA extraction and RT-PCR testing.

Table 1

Characteristics of individuals providing paired NSS (n = 842) by results for SARS-CoV-2 real-time reverse transcription-polymerase chain reaction (RT-PCR) and Rapid Antigen Test (RAT) – January 2021.

Characteristic			True positives (n = 51, %)	False negatives (n = 22, %)	False positives (n = 3, %)	True negatives (n = 7 66, %)	Total (n = 842, %)
Testing Site	Context	Date
A	Workers	January 14	1			55	56 (6.7%)
B	Sanitary residence	January 14				51	51 (6.1%)
C	Sanitary residence	January 14		1		29	30 (3.6%)
D	General public	January 15	13	5		193	211 (25.1%)
E	Sanitary residence	January 15	9	2	2	145	158 (18.8%)
F	General public	January 16	23	14	1	210	248 (29.5%)
G	General public	January 17	5			83	88 (10.5%)
Total			51 (6.05%)	22 (2.61%)	3 (0.35%)	766 (91%)	842 (100%)
Sex
Male			27	9	1	392	429 (51.0%)
Female			23	13	2	313	351 (41.7%)
#N/D			1			61	62 (7.4%)
Total			51 (6.05%)	22 (2.61%)	3 (0.35%)	766 (91%)	842 (100%)
Nationality
Bolivia			3			18	21 (2.5%)
Brazil						1	1 (0.1%)
Chile			37	19	1	476	533 (63.3%)
China						1	1 (0.1%)
Colombia						11	11 (1.3%)
Cuba						4	4 (0.5%)
Ecuador						2	2 (0.2%)
Paraguay						1	1 (0.1%)
Perú						10	10 (1.2%)
Venezuela			10	3	2	171	186 (22.1%)
#N/D			1			71	72 (8.6%)
Total			51 (6.05%)	22 (2.61%)	3 (0.35%)	766 (91%)	842 (100%)

N/D: no data.

Rapid antigen test

The SD Biosensor, Inc. Antigen Test (Republic of Korea, Catalog number 9901-NCOV-01G) is a rapid lateral flow immunoassay for the qualitative detection of SARS-CoV-2-specific antigens present in the human nasopharynx, with sensitivity of 96.52% and specificity of 99.68% (SARS-CoV-2 Rapid Antigen Test Package Insert 2020-08, V 1.0). According to the manufacturer, the results are available within 30 min and all necessary reagents to perform the assay are provided. The assay kits are stable when stored at 2–30 °C.

RT-PCR

Viral RNA was extracted using the Mag-Bind Viral DNA/RNA 96 kit (Omega Bio-Tek, Catalog number M6246) on the Kingfisher Flex Magnetic Particle Processor (Thermo Fisher Scientific). Real-time RT-PCR was performed at Laboratorio Médico Bioclinic and Hospital Regional de Iquique using the GenomeCov19 Detection Kit ABM (Applied Biological Materials Inc, Canada, Catalog number G628.v2), with cycle threshold (Ct) values ≤ 40 considered positive for the N and S viral gene regions.

Statistical analysis

Statistical analysis considered sensitivity, specificity, Positive Predictive Value (PPV) and Negative Predictive Value (NPV), accuracy, Kappa coefficient, and Wilson score Confidence Interval at 95% (GraphPad Prism version 9.0.1).

Results

This study evaluated the performance of the SARS-CoV-2 RAT (SD Biosensor, Inc. Republic of Korea) compared with the real-time RT-PCR for SARS-CoV-2 detection among asymptomatic individuals at Iquique city, Tarapacá Region, Chile. The sampling was carried out in seven testing sites corresponding to i) workers (n = 56; 6.7%), ii) sanitary residence (n = 239; 28.4%) and iii) general public (n = 547; 65%). A total of 854 individuals were included (mean age: 36.67 years; SD: 16.48 years; males: 51%; females: 41.6%; N/A: 7.4%). Two NSS samples from each individual were collected by healthcare workers at testing sites. Among a total of 854 NSS submitted, 12 (1.4%) were excluded for lacking real-time RT-PCR results. Among 842 paired NSS from asymptomatic individuals, 54 (6.17%) were antigen-positive and 73 (8.6%) were real-time RT-PCR-positive. Antigen testing sensitivity was 69.86% (51 of 73), specificity was 99.61% (766 of 769), PPV was 94.44% (51 of 54), and NPV was 97.22% (766 of 789). Three paired (0.35%) NSS were antigen-positive and real-time RT-PCR-negative. Accuracy between the two techniques was 97.04% (Kappa coefficient = 0.78, 95% CI: 0.70–0.86) (Table 2 ). Given that the Ct value is inversely related to the viral load (Rao et al., 2020), the PCR Ct value data of the 73 samples that tested positive for NSS SARS-CoV-2 real-time RT-PCR detection were reviewed: Ct values were significantly higher among individuals with false-negative RAT (viral gene N: 28.07 ± 4.343; viral gene S: 28.81 ± 3.873) compared with true positives (viral gene N: 19.99 ± 4.535; viral gene S: 20.93 ± 4.487) (Figure 1A). The k-means clustering analysis was then reviewed to find clusters in an iterative way using the Ct value of viral gene N (Cerutti et al., 2020). Three clusters were identified: c1 = 16.87 (n = 31, strongly positive), c2 = 24.5 (n = 30, moderately positive) and c3 = 31.67 (n = 12, weakly positive), where 96.77% (30/31) of the samples with low Ct value also tested positive for RAT (Figure 1B, cluster 1), while 66.6% (20/30) of the samples with medium Ct value also tested positive for RAT (Figure 1B, cluster 2). In contrast, 8.33% (1/12) of the samples with high Ct value tested positive for RAT (Figure 1B, cluster 3). Three RATs-positive and SARS CoV-2 real-time RT-PCR-negative samples were identified (Table 1).

Table 2

Agreement between RT-PCR test results and antigen test results overall.

January 2021 – Tarapacá Region
	RT-PCR (+)	RT-PCR (−)	Total
RAT (+)	51	3	54
RAT (−)	22	766	788
Total	73	769	842

Figure 1

(A) Difference in viral cycle threshold (Ct) value between asymptomatic individuals with positive (black) and negative (red) RAT among real-time RT-PCR-positive (n = 73). Analysis for statistical difference was performed by Wilcoxon test. (B) Plot of K-means clustering results of NSS real-time RT-PCR Ct values used to compare to RAT (black dots: RAT+/red dots: RAT−).

Discussion

Improvements in SARS-CoV-2 diagnosis with easy, rapid and cost-efficient approaches are urgently required to control the COVID-19 pandemic. As of April 2021, more than 170 assays are on the market (FINDdx, 2020) and few have been extensively validated. Recently, the second iteration of a Cochrane living review summarized the accuracy of multiple RATs. The average sensitivity reported in symptomatic individuals from 37 evaluations was 72.0% (95% CI: 63.7–79.0%), while that in asymptomatic individuals from 12 evaluations was 58.1% (95% CI: 40.2–74.1%) (Dinnes et al., 2021). The SD Biosensor RAT (Inc., Republic of Korea) manufacturer reported a higher sensitivity (96.52%; 95% CI: 91.33–99.04%) obtained in prospective, randomized, single blinded studies conducted in Brazil and India in symptomatic and asymptomatic individuals (SARS-CoV-2 Rapid Antigen Test Package Insert 2020-08, V 1.0). In addition, a mass screening campaign in Catalonia (North-East Spain) evaluated four RATs in a cohort of 286 asymptomatic individuals. The sensitivity of each was 38.6% for Abbott assay, 51.5% for Siemens, 45.5% for Lepu and 43.6% for Roche (the same used in this study), with 83.3% of specimens with Ct < 30 (Baro et al., 2021).

This study comprised asymptomatic individuals in three different contexts with an 8.64% prevalence of SARS-CoV-2 infection. In agreement with the Centers for Disease Control and Prevention (CDC, 2021) recommendation on the use of antigen testing, the sensitivity of 69.86% indicates that RAT should not replace real-time RT-PCR in diagnosis and surveillance of SARS-CoV-2 infection (CDC, 2021). However, the PPV of antigen testing was 94.44%, indicating that asymptomatic persons with positive antigen results are infected with SARS-CoV-2 and would not require confirmatory real-time RT-PCR. Likewise, the NPV of antigen testing was 97.21%, indicating that asymptomatic individuals with negative antigen results are unlikely to be infected with SARS-CoV-2. Furthermore, individuals with false-negative results of the RAT had significantly higher Ct values (Ct > 27), which can be related to lower viral loads and less infectiousness in general (Bullard et al., 2020, Singanayagam et al., 2020). Given the high predictive values in asymptomatic individuals and the fast test result implying faster tracing of infected individuals, these results support and provide policymakers with evidence that RAT might have a significant role in COVID-19 screening, testing and contact tracing strategies to control the COVID-19 pandemic in i) areas that lack suitable laboratories to perform SARS-CoV-2 real-time RT-PCR diagnostics, ii) areas where results take more than 24–48 h and iii) areas with high traffic of individuals such as border/customs, airports, interregional bus and train stations or in any mass testing campaign requiring rapid results.

Author contributions

FV-E, MJF, SS, PG, MSV, and JA participated in the study design. MP, MA and CG performed the experiments. RL provided the NSS from COVID-19 asymptomatic individuals. FP performed the statistical analysis. FV-E, AG, SS, and RS-R wrote the manuscript. MSV and JA performed funding acquisition. All authors approved the final version of the manuscript.

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Funding source

We wish to thank Ministerio de Salud de Chile for funding this study. The authors also are supported by ANID Chile through Fondecyt grants Nº 1181656 (AG), 1190156 (RS-R) and 1180798/1211547 (FV-E).

Ethical statement

Ministerio de Salud de Chile instructed this study to implement the use of the RAT in asymptomatic individuals. Therefore, this work is considered as a public health intervention to improve diagnosis. The study described here was approved by the Ethics Committee of the Faculty of Medicine at Universidad de Chile (Project Nº 036-2020). NSS from COVID-19 positive and negative patients were anonymized.