Clinical assessment of SARS-CoV-2 antigen rapid detection compared with RT-PCR assay for emerging variants at a high-throughput community testing site in Taiwan.

OBJECTIVES: With the emergence of the severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) B.1.1.7 lineage in the ongoing coronavirus disease 2019 (COVID-19) pandemic, Taiwan confronted a COVID-19 flare up in May 2021. Large-scale, accurate, affordable and rapid diagnostic tests such as the lateral flow assay can help to prevent community transmission, but their performance characteristics in real-world conditions and relevant subpopulations remain unclear.
METHODS: The COVID-19 Antigen Rapid Test Kit (Eternal Materials, New Taipei City, Taiwan) was used in a high-throughput community testing site; the paired reverse transcription polymerase chain reaction (RT-PCR) results served as a reference for sensitivity and specificity calculations.
RESULTS: Of 2096 specimens tested using the rapid antigen test, 70 (3.33%) were positive and 2026 (96.7%) were negative. This clinical performance was compared with the RT-PCR results. The sensitivity and specificity of the rapid antigen test were 76.39% [95% confidence interval (CI) 64.91-85.60%] and 99.26% (95% CI 98.78-99.58%), respectively, with high sensitivity in subjects with cycle threshold values ≤24. Further, the rapid antigen test detected the SARS-CoV-2 B.1.1.7 lineage effectively.
CONCLUSIONS: Considering the short turnaround times and lower costs, this simple SARS-CoV-2 antigen detection test for rapid screening combined with RT-PCR as a double confirmatory screening tool can facilitate the prevention of community transmission during COVID-19 emergencies.

Introduction

A cluster of cases of pneumonia with unknown aetiology was reported and confirmed as 2019-nCoV in 2019 (Wu et al., 2020; Zhu et al., 2020). Severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2), the cause of coronavirus disease 2019 (COVID-19), has since spread globally. Reverse transcription polymerase chain reaction (RT-PCR), which is performed to detect viral nucleic acids, is currently considered as the diagnostic gold standard for early diagnosis in patients with suspected SARS-CoV-2 infection (Russo et al., 2020; Safiabadi Tali et al., 2021). As of 26 December 2020, Taiwan had only 783 confirmed cases of COVID-19 because of its effective centralized quarantine policies (Tsai et al., 2021). Household transmission accounts for the major infection sources apart from community-acquired COVID-19 outbreaks (Hsu et al., 2021). However, Taiwan confronted a COVID-19 flare-up in May 2021; the number of confirmed COVID-19 cases surged >10-fold compared with those in 2020, with confirmed cases reaching 15,000 by 15 July 2021 (https://covid19.mohw.gov.tw/en/sp-timeline0-206.html). In addition to the recent surge in the number of indigenous COVID-19 cases in northern Taiwan, new SARS-CoV-2 variants of concern (VOCs) with possibly enhanced transmissibility and/or severity, as well as diagnostic and/or treatment failure, are important issues (Boehm et al., 2021). SARS-CoV-2 lineages carrying the amino acid substitution N501Y spread rapidly in the UK during late autumn 2020, and thereafter in Taiwan (Leung et al., 2021). Taiwan faced a third epidemic wave in May–July 2021, forcing laboratories to the maximum of their testing capacity despite limited reagent supplies. Therefore, there is an increasing need for more rapid and feasible assays, such as lateral flow assays. Several reports have described the performance of rapid antigen test kits, most of which were derived from COVID-19 symptomatic subjects (Caruana et al., 2021). Previous studies reported that the clinical sensitivities of these assays vary widely (Dinnes et al., 2020; Hayer et al., 2021). However, the role of rapid antigen test kits as a screening tool in community-based test sites to test subjects with or without COVID-19 symptoms remains unclear. Thus, these SARS-CoV-2 rapid antigen tests with emergency use authorization require rigorous evaluation of their performance characteristics in different epidemiological settings.

The aim of this study was to evaluate the clinical performance of the COVID-19 Antigen Rapid Test Kit (Eternal Materials, New Taipei City, Taiwan) compared with nucleic acid amplification testing (NAAT) using a multiplex quantitative RT-PCR kit with dual-target genes (E, N1) on the LabTurbo AIO 48 system (LabTurbo, Taipei City, Taiwan). This study also independently evaluated the performance characteristics of the rapid antigen test for detecting SARS-CoV-2, particularly the VOC B.1.1.7 lineage.

Materials and methods

Study design

This study was performed in the Wanhua District of Taipei City in Taiwan between 17 and 22 May 2021 when the prevalence of COVID-19 was 3.5% because of a sudden increase in local cases; subsequently, the Central Epidemic Command Centre of Taiwan announced a Level 3 epidemic warning nationwide. This study included 2096 eligible subjects (symptomatic or non-symptomatic with a history of contact with a confirmed COVID-19 case); a flowchart of the screening and confirmatory strategy is shown in Figure 1 . Two simultaneous nasopharyngeal swabs were collected for symptomatic or asymptomatic subjects aged 8–99 years at community testing sites using a standard procedure. The first swab was analysed using the COVID-19 Antigen Rapid Test Kit which has a turnaround time of 15 min. Subjects with positive results were quarantined immediately and asked to wait for NAAT confirmation, performed on the LabTurbo AIO 48 system, an automated high-throughput and sample-to-results diagnostic platform. Residents with negative results were asked to self-monitor their health until they received the NAAT report. This study was approved by the Institutional Review Board of the Tri-Service General Hospital (TSGHIRB No.: C202005041) and was registered on 8 February 2021. Informed consent was obtained for this study. Positive cases were defined using NAAT as the reference method.

Figure 1

Flowchart of the study using the rapid antigen test (ART) as a screening tool in combination with reverse transcription polymerase chain reaction (RT-PCR) for severe acute respiratory syndrome coronavirus-2 infection in densely populated areas. CECC, Central Epidemic Command Centre in Taiwan.

SARS-CoV-2 rapid antigen test

The COVID-19 Antigen Rapid Test Kit is a membrane-based immunochromatographic assay that has been granted emergency use authorization by the Taiwan Food and Drug Administration. It detects the nucleocapsid protein of SARS-CoV-2 in nasopharyngeal samples. A healthcare professional first collected the nasopharyngeal swab samples using the materials in the rapid antigen test. The assay was performed and interpreted by on-site technicians according to the manufacturer's instructions. Positive results were scored as positive and weakly positive based on the intensity of the test line compared with the control line. Band intensities stronger than the control line were scored as positive, and those stronger than the background but weaker than the control line were scored as weakly positive. The rapid test results were compared with those of the multiplex real-time RT-PCR kit with dual target genes (E, N1) performed on the LabTurbo AIO 48 system.

SARS-CoV-2 real-time RT-PCR

Another nasopharyngeal swab specimen was collected from residents with suspected COVID-19 using LIBO specimen collection and transport swab kits in 2 mL Universal Transport Medium (LIBO Medical Products Inc., New Taipei City, Taiwan). An automated sample-to-result SARS-CoV-2 RT-PCR assay for high-throughput testing was performed using the LabTurbo AIO 48 system, with the LabTurbo AIO COVID-19 RNA testing kit for SARS-CoV-2 multiplex real-time RT-PCR, containing reverse transcriptase, primer/probe mixture and 2X PCR master mix, in accordance with the manufacturer's instructions.

Detection of SARS-CoV-2 VOCs

SARS-CoV-2-positive samples detected by both the rapid antigen test and RT-PCR were analysed. For rapid screening for SARS-CoV-2 VOCs (B.1.1.7 lineage), VirSNiP SARS-CoV-2 Spike N501Y and Spike del H69/V70 (TIB Molbiol, Berlin, Germany) were used to detect the N501Y and del 69–70 mutations, respectively, in SARS-CoV-2-positive specimens. Briefly, an RT-PCR variant assay with melting curve analysis was used to detect the spike gene mutations on a LightCycler 480 (Roche, Basel, Switzerland) as described previously (Jian et al., 2021; Ong et al., 2021).

Statistical analysis

Specificity and sensitivity with 95% confidence intervals and positive and negative predictive values of the rapid antigen test were calculated using the RT-PCR results as a reference method. Analyses were performed using Excel (Microsoft Corp, Redmond, WA, USA) and GraphPad Prism Version 8.0 (GraphPad, Inc., San Diego, CA, USA).

Results

Community RT-PCR testing results

Individuals at the community testing site were aged 8–99 years, and the majority were male (61.7%). Overall, 2096 subjects from the community testing site were included in the study, of which 72 tested positive for SARS-CoV-2 by RT-PCR (Table 1 ); the prevalence rate was approximately 3.5%. The cycle threshold (Ct) values of the positive results ranged from 12 to 29.

Table 1

Comparison of coronavirus disease 2019 (COVID-19) rapid antigen test and reverse transcription polymerase chain reaction (RT-PCR).

		COVID-19 rapid antigen test
		Negative	Positivea	Total
RT-PCR	Negative	2009	15	2024
	Positive	17	55	72
	Total	2026	70	2096

Including weakly positive (n=23) and positive (n=47).

Rapid antigen test results

Of the 2096 nasopharyngeal specimens, 70 (3.3%) were weakly positive or positive and 2026 (96.7%) were negative by the rapid antigen test detecting the SARS-CoV-2 nucleocapsid protein (Table 1). Further investigation of the clinical performance of the test was compared with the RT-PCR assay results. The overall sensitivity and specificity of the rapid SARS-CoV-2 antigen detection test were 76.39% and 99.26%, respectively (Table 2 ). The rapid SARS-CoV-2 antigen test implemented in this study showed moderate sensitivity but high specificity. Further, this test had a moderate positive predictive value (78.90%) and a high negative predictive value (99.14%) (Table 2). The sensitivity of the antigen test was stratified by the Ct values, and the test sensitivity was related to the viral load; the highest sensitivity was observed for RT-PCR Ct values <20, moderate sensitivity for Ct values between 20 and 25, and sensitivity decreased dramatically at Ct values >25 (Table 3 ).

Table 2

Characteristics of the Coronavirus Disease 2019 (COVID-19) Rapid Antigen Test Kit (ART) compared with reverse transcription polymerase chain reaction (RT-PCR) in a high-throughput community testing site.

RT		95% CI
Sensitivity	76.39%	64.91–85.60%
Specificity	99.26%	98.78–99.58%
Positive likelihood ratio	103.07	61.26–173.42
Negative likelihood ratio	0.24	0.16–0.36
Positive predictive value	78.90%	68.96–86.28%
Negative predictive value	99.14%	98.71–99.43%
Accuracy	98.46%	97.83–98.94%

CI, confidence interval.

Table 3

Sensitivity of the Coronavirus Disease 2019 (COVID-19) Rapid Antigen Test Kit (ART) by reverse transcription polymerase chain reaction (RT-PCR) cycle threshold (Ct) intervals.

		n	ART positive	Sensitivity % (95% CI)	ART negative	False-negative rate (%)
RT-PCR Ct value	≤20	39	39	100 (88.8–100)	0	0
	20–≤ 25	19	12	63.15 (38.63–82.77)	7	36.84
	25–<30	14	4	28.57 (9.58–57.99)	10	71.43

CI, confidence interval.

Rapid antigen test performance in detecting VOC B.1.1.7

To determine the clinical performance of the rapid antigen test for the detection of VOCs, 55 specimens that yielded positive results in both the rapid antigen test and RT-PCR were evaluated. As VOC B.1.1.7 began circulating in early September 2020, previously stored 50 SARS-CoV-2-specific RT-PCR-positive specimens collected between June and July 2020 were also used for further identification as wild-type (non-B.1.1.7) or VOC (B.1.1.7) strains. The rapid antigen test showed 100% positive agreement with RT-PCR as a reference method for the 55 positive VOC (B.1.1.7) SARS-CoV-2-positive specimens (Table 4 ). Thus, the antigen assays used in this study were able to detect the B.1.1.7 variant of this outbreak strain.

Table 4

Performance of the Coronavirus Disease 2019 (COVID-19) Rapid Antigen Test Kit (ART) for the detection of severe acute respiratory syndrome coronavirus-2 variants of concern (VOCs).

		VOC mutants distinguished on RT-PCR
		Wild-typea	VOCb
ART positive		n=50	n=55

RT-PCR, reverse transcription polymerase chain reaction.

Wild-type (non-B.1.1.7): spike protein without mutations on either N501Y or del 69-70.

VOC (B.1.1.7): spike protein with both N501Y and del 69-70 mutations.

Discordant analysis between rapid antigen test and RT-PCR

Across all 2096 tested subjects, 17 false-negative results were observed with Ct values between 21 and 29, and 15 false-positive results were observed for RT-PCR-negative results (Figure 2 ). Further investigation of the 15 false-positive cases showed that five cases were scored as weakly positive with faint bands by two independent readers, and the remaining 10 were scored as positive with clearly visible bands (Figure 2). When paired rapid antigen detection and molecular diagnostic methods were performed simultaneously, false-negative results were corrected rapidly by the confirmatory RT-PCR results the following day, and the subjects were quarantined or admitted to dedicated hospital wards. The 15 subjects with presumed positive results by the rapid antigen test were isolated until they received the confirmatory RT-PCR results the following day. Following a negative RT-PCR confirmatory result, the subjects discontinued isolation and continued self-management of their health.

Figure 2

Discordant analysis between coronavirus disease 2019 rapid antigen test and reverse transcription polymerase chain reaction (RT-PCR). Ct, cycle threshold.

Discussion

This study comprehensively and systematically evaluated the clinical performance of the COVID-19 Antigen Rapid Test Kit in a community setting including symptomatic and asymptomatic subjects. In this real-life evaluation of a rapid antigen test kit in the community, SARS-CoV-2-infected subjects with low Ct values were identified by RT-PCR (Ct values <20). As expected, the 100% specificity and 100% sensitivity mentioned in the package inserts were not observed in the clinical evaluation in a community setting. This study revealed the actual test performance, particularly in symptomatic or asymptomatic subjects. Previous studies demonstrated the advantages and clinical performance of rapid antigen test devices for detecting circulating VOCs (Jungnick et al., 2021; Rodgers et al., 2021); however, these data showed a favourable performance for the detection of SARS-CoV-2 VOCs using rapid antigen testing with only limited clinical samples in a UK population or viruses derived from cell culture. The rapid antigen test was able to detect the SARS-CoV-2 B.1.1.7 variant, particularly during emergence of B.1.1.7 during the third pandemic wave in Taiwan. As lateral flow assays suffer from subjective interpretation, which may lead to difficulties in analysing weakly positive bands, weakly positive results were also addressed along with the extended edge-to-edge of the test strip, which may pose major questions for visual interpretation by technicians. For example, 23 weakly positive results were reported by the rapid antigen test in this study, although only 56% of these results were true-positives using RT-PCR as the reference method. Subjective interpretation of uncertain results is a major limitation of lateral flow assays (Pegoraro et al., 2021). Peeling et al. (2021) showed that most detected cases representing false-positives rather than true infections may require a two-tier approach for molecular confirmation. As the inherent lower sensitivity may be offset by combining this approach with NAAT methods, paired RT-PCR was performed in this study to counter the limitation of rapid antigen tests because of its lower sensitivity in blocking COVID-19 community transmission. Lateral flow antigen detection diagnostics have long been used for various infectious diseases considering their convenience and short turnaround times (≤15 min) (Dinnes et al., 2021). In the current COVID-19 pandemic, rapid SARS-CoV-2 testing kits aid in the rapid assessment of infectiousness in densely populated areas affected by the epidemic, and have been widely applied in many countries to contain the pandemic. In general, lateral flow assays are effective when subjects are symptomatic or have a history of contact with a confirmed COVID-19 case. However, SARS-CoV-2 infection can be asymptomatic, and the viral load in the upper respiratory tract in these cases is low compared with that of symptomatic cases (Han et al., 2020; Ra et al., 2021).

In summary, the COVID-19 rapid antigen test evaluated in this study was able to detect SARS-CoV-2 infection with high viral loads in both asymptomatic and symptomatic individuals. Thus, this test can serve as a rapid tool for blocking community spread of SARS-CoV-2. The limitations of this study include its cross-sectional design and the fact that no distinction was made between symptomatic and non-symptomatic participants, which may influence the sensitivity of this rapid antigen assay. Additional studies of the field performance of this assay in different settings is needed to develop strategies for optimal use.

Declaration of Competing Interest

The authors declare no conflicts of interest.