Clinical sensitivity of rapid antigen test during a COVID-19 outbreak in Taipei, May to June 2021.

BACKGROUND/
PURPOSE: This population-based study aimed to compare the accuracy of Rapid antigen detection (RAD) and reverse transcription-polymerase chain reaction (RT-PCR) assays for diagnosing individuals infected with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) during the COVID-19 outbreak in Taipei, from May to June 2021.
METHODS: In response to the outbreak of COVID-19 in mid-May 2021, Taipei City Hospital set up 12 citywide proactive community testing (PCT) stations for early identification of infected individuals from May 17 to June 20, 2021. Individuals with RAD positivity were isolated and later confirmed by RT-PCR. The c-statistic value was estimated to indicate the level of diagnostic accuracy of RAD tests.
RESULTS: Of the 33,798 individuals who were evaluated for SARS-CoV-2 infection, 4.4% tested positive for RAD. There was a moderate concordance (kappa = 0.67) between the RAD tests and RT-PCR assay for identifying infectious individuals. The c-statistic value of the RAD test for the diagnosis of SARS-CoV-2 infection was 0.8. There was a positive linear trend between the accuracy of the RAD tests and the prevalence of SARS-CoV-2 infection in the study population (β = 0.04; p = .03). As the cycle threshold value decreased, the sensitivity rate of the RAD tests increased (p < .001). After implementation of the PCT program, the prevalence of COVID-19 decreased from 8.4% to 3.3% (p < .001).
CONCLUSION: Proactive community testing for SARS-CoV-2 infection using RAD tests could rapidly identify and quarantine the most infectious patients in the early phase of COVID-19 outbreak.

Introduction

The coronavirus disease 2019 (COVID-19), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has been spreading worldwide since January 2020. As of January 3, 2022, 281.8 million individuals have been infected with SARS-CoV-2, with the number of deaths reaching 5.4 million.

SARS-CoV-2 is highly contagious and thus susceptible to outbreaks. Early identification of patients with COVID-19 is essential to control the outbreak. A previous report indicated that proactive testing for SARS-CoV-2 in a population could identify symptomatic and asymptomatic infectious individuals early and prevent the onward infection of others. World Health Organization (WHO) also recommended extensive testing as an important strategy to control the increasing incidence of COVID-19 during the pandemic.

The current standard test for laboratory diagnosis of SARS-CoV-2 infection is the real-time reverse transcription-polymerase chain reaction (RT-PCR) assay. Although the results of the RT-PCR assay are normally available within 4 h, some reports showed that the results of the RT-PCR assay could be delayed by three to seven days. A long waiting period to obtain results increases the risk of virus transmission and causes more challenges to controlling the COVID-19 outbreak.

Rapid antigen detection (RAD) tests are an alternative assay for diagnosing active infection by detecting SARS-CoV-2 viral antigens. The test results of RAD can be interpreted without specialized instruments and are available within 15–20 min. However, in contrast to the analytical sensitivity (positive rate to detect standard laboratory SARS-CoV-2 samples) required for FDA licensure, the public health usefulness of RAD tests actually depends on their clinical sensitivity (positive rate to detect all patients with SARS-CoV-2 infection, including those in latency period), , which has not been assessed in an outbreak situation. A previous study in China showed that the clinical sensitivity of RAD tests for the diagnosis of COVID-19 ranged from 22.9 % to 71.4% when different RAD kits were used. Although RAD tests have been recommended for the use of mass screening to assist in controlling outbreaks, their clinical sensitivity during different phases of the COVID-19 outbreak has not been extensively studied.

Taiwan experienced its first large-scale outbreak of SARS-CoV-2 infection on May 15, 2021, with 180 laboratory-confirmed indigenous COVID-19 cases reported to the Taiwan Centers for Disease Control (CDC) in a single day. By May 14, 2021, only 1290 laboratory-confirmed COVID-19 cases were reported to the Taiwan CDC, including 164 (12.7%) local cases and 1073 (83.2%) imported cases. As the capacity of RT-PCR during the COVID-19 outbreak in Taiwan could not accommodate the testing demand from the rapid increase in the number of cases, the Taipei City government adopted a new quarantine policy based on a single RAD test. Individuals who tested positive on RAD were isolated and their results were later confirmed by RT-PCR during the COVID-19 outbreak in May, 2021.

When the community is threatened by the spread of COVID-19, the most important measure is the early identification and isolation of individuals infected with the disease. Previous studies found that RAD tests are an important tool in assisting the control of COVID-19 outbreaks. , However, previous reports were unable to evaluate the accuracy of RAD tests for the diagnosis of SARS-CoV-2 during the different stages of the COVID-19 outbreak. , Moreover, it remains unclear whether RAD tests are actually useful in identifying the most infectious patients during a COVID-19 outbreak, and thus facilitate early quarantine and contact tracing of the suspected infectious individuals. Therefore, we conducted a population-based study to compare the accuracy of RAD and RT-PCR assays in diagnosing individuals infected with SARS-CoV-2 during the COVID-19 outbreak in Taipei, from May to June 2021.

Materials and methods

Background information

In mid-May 2021, Taiwan had its first huge outbreak of SARS-CoV-2, which was particularly severe in Taipei. Taipei City Hospital (TCH), a healthcare organization affiliated with the Taipei City Government, set up 12 extensive proactive community testing (PCT) stations to identify and isolate individuals infected with SARS-CoV-2. People who had been in contact with a patient with COVID-19 were informed by the Taipei Bureau of Public Health to receive free-of-charge RAD tests and RT-PCR assays. The RAD test at the PCT stations was also conducted for those who had symptoms of COVID-19. When the RAD test in an individual showed positivity for SARS-COV-2 infection, the patient was admitted to the designated isolation centers in Taipei.

Study subjects

This study recruited individuals who were tested for SARS-COV-2 infection at various Taipei PCT stations from May 17 to June 20, 2021, using RAD and RT-PCR assays. During the RAD tests, participants’ demographic data (e.g., age and sex) were collected. Individuals who did not complete all of the demographic questions were excluded from the analysis. This study was approved by the Institutional Review Board of Taipei City Hospital (TCH) (no. TCHIRB-10904014-E). All procedures in this study were performed in accordance with the Declaration of Helsinki guidelines.

Clinical specimen collection

At the time of the RAD tests, respiratory samples, mainly nasopharyngeal and throat swabs, were collected. Samples in study individuals were mixed in 2 mL of viral transport media, comprising Hanks’ balanced salt, 0.4% fetal bovine serum, and HEPES, as well as antifungal and antibiotic agents. Samples were transported at 2–8 °C to the Microbiology Laboratory, Taiwan CDC and then were processed within a few hours. All samples were processed in biosafety level-2 enhanced (BSL-2 +) and biosafety level-3 (BSL-3) facilities with proper personal protective equipments.

Rapid SARS-CoV-2 antigen detection assay

The RAD tests in this study were rapid chromatographic immunoassays for the detection of SARS-CoV-2 nucleocapsid (N) antigens in respiratory specimens. The RAD test device had two precoated lines on the result window: test (T) and control (C) lines. The test (T) region was coated with mouse monoclonal anti-SARS-CoV-2 antibody against SARS-CoV-2 N antigen, and the control (C) region was coated with mouse monoclonal anti-chicken Igγ antibody. For COVID-19 antigen testing positivity, two colored lines of test (T) and control (C) lines are presented.

SARS-CoV-2 RNA detection using real-time RT-PCR

At the time of the RAD tests, participants also received an RT-PCR assay to confirm SARS-CoV-2 infection. After sampling, the oropharyngeal swabs were placed in a viral transport medium for RNA extraction. We used an RNA purification kit (QIAmp Viral RNA Mini Kit, Qiagen, Germany) to extract the viral RNA. RT-PCR was performed by amplifying the RNA-dependent RNA polymerase gene (RdRp), envelope (E), and nucleocapsid (N) genes. The E gene and N gene assays were used as first-line screening targets, which was then followed by confirmatory testing with the RdRp gene assay.

Statistical analyses

First, we analyzed the demographic data of the study individuals. Then we presented the continuous data as the mean (standard deviation [SD]), and used a two-sample t-test to compare groups. We also used Pearson's χ2 test to analyze the categorical data, where appropriate.

The concordance between the RAD test and the RT-PCR in terms of diagnosing SARS-CoV-2 infection was calculated as the overall percentage agreement using 2 × 2 contingency tables. The strength of this agreement was assessed using the kappa statistics.

To assess the diagnostic accuracy of RAD tests, we calculated the sensitivity (the ability to identify individuals infected with SARS-CoV-2), specificity (the ability to identify those not infected with SARS-CoV-2), positive predictive value (PPV) (the proportion of individuals who were infected with SARS-CoV-2 when the RAD tests were positive), and negative predictive value (NPV) (the proportion of individuals who were not infected with SARS-CoV-2 when the RAD tests were negative). The c-statistic value, also known as the area under the curve, was estimated to indicate the diagnostic accuracy of RAD tests. A score of 0.5 suggests a test with poor diagnostic value, meaning that RAD tests are no better than chance at identifying an individual infected with SARS-CoV-2. An increase in the c-statistic value indicates an increase in the diagnostic accuracy. A good diagnostic instrument requires a c-statistic score >0.7.

Linear regression analysis was used to assess the associations between the accuracy of the RAD tests and the prevalence of SARS-CoV-2 infection in the study participants. All data management and analyses were performed using the Statistical Analysis System (SAS) version 9.4 statistical software package (SAS Institute, Cary, NC, USA).

Results

Patient selection

From May 17 to June 20, 2021, 33,800 individuals were tested for SARS-COV-2 at the PCT stations in Taipei. After excluding those with missing data (n = 2), the remaining 33,798 individuals were included in the analysis (Fig. 1 ). The overall mean (SD) age was 45.0 (16.5) years, 55.0% of the subjects were male, and 4.4% of the individuals were positive for RAD.

Figure 1

The process of enrollment in individuals tested for SARS-COV-2 infection. SARS-COV-2 = severe acute respiratory syndrome coronavirus 2; RAD = Rapid antigen detection; RT-PCR = reverse transcription-polymerase chain reaction.

Characteristics of study subjects by rapid antigen detection test results

Table 1 shows the characteristics of the study participants with positive and negative RAD tests. Compared to individuals with RAD test negativity, those with RAD test positivity were older. In terms of the RT-PCR assay, individuals with RAD test positivity were more likely to show RT-PCR positivity.

Table 1

Baseline characteristics of the study participants based on rapid antigen detection test (RAD) results.

	Total number of subjects	Positive RAD test, n (%)	Negative RAD test, n (%)	p value
Demographics
Age (years)
Mean ± SD	44.95 ± 16.53	52.61 ± 17.84	44.60 ± 16.38	<0.001
<18	608	54 (8.88)	554 (91.12)	<0.001
18-29	6494	139 (2.14)	6358 (97.86)
30-39	7218	172 (2.38)	7051 (97.62)
40-49	6929	223 (3.22)	6712 (96.78)
50-59	6018	311 (5.16)	5712 (94.84)
60-69	4120	370 (8.97)	3753 (91.03)
≥70	2389	214 (8.96)	2175 (91.04)
Sex
Female	15,224	699 (4.59)	14,525 (95.41)	0.098
Male	18,574	784 (4.22)	17,790 (95.78)
RT-PCR assay
Negative	31,899	331 (1.04)	31,568 (98.96)	<0.001
Positive	1899	1152 (60.66)	747 (39.34)

RAD, rapid antigen detection; SD, standard deviation; RT-PCR, reverse transcription-polymerase chain reaction.

∗Unless stated otherwise.

Correlation between RAD tests and RT-PCR assay

Table 2 shows the kappa statistics between the RAD and RT-PCR assays in terms of diagnosing SARS-CoV-2 infection. A moderate concordance (kappa = 0.67) was found between RAD tests and the RT-PCR assay to identify individuals with COVID-19 infection. Participants with both RAD test negativity and RT-PCR positivity were the major contributors to the discrepancy between these two tests.

Table 2

Agreement between rapid antigen detection (RAD) test and RT-PCR assay among the study participants in the proactive community testing program in Taipei, Taiwan.

	RAD test positivity	RAD test negativity	Total number	Agreement (%)	kappa	p value
RT-PCR positivity	1152	747	1899
RT-PCR negativity	331	31,546	31,877
Number	1483	32,293	33,776	96.8	0.67	<0.001

RAD, rapid antigen detection; RT-PCR, reverse transcription-polymerase chain reaction.

Accuracy of rapid antigen detection tests for diagnosis of SARS-CoV-2 infection

Table 3 shows the accuracy of RAD tests for the diagnosis of SARS-CoV-2 infection. The c-statistic value of the RAD test for the diagnosis of SARS-CoV-2 infection was 0.8. There was a linear trend between the accuracy of the RAD tests and the prevalence of SARS-CoV-2 infection in the study participants (β = 0.04; p = .03).

Table 3

Accuracy of the rapid antigen detection test (RAD) for the diagnosis of SARS-CoV-2 infection.

Study period	Total number of subjects	Number of RT-PCR positivity, n (%)	Sensitivity % (95% CI)	Specificity	Positive predictive value	Negative predictive value	C-statistic
Overall	33,776	1899 (5.60)	60.7 (58.4–62.9)	99.0 (98.8–99.1)	77.7 (75.7–79.6)	97.7 (97.6–97.8)	0.80 (0.78–0.81)
First week of the study	7392	623 (8.43)	72.1 (68.4–75.6)	99.0 (98.7–99.2)	87.0 (84.0–89.5)	97.5 (97.1–97.8)	0.86 (0.83–0.88)
Second week	6623	477 (7.20)	67.1 (62.7–71.3)	98.0 (97.6–98.3)	71.9 (68.0–75.5)	97.5 (97.1–97.8)	0.83 (0.80–0.85)
Third week	6031	356 (5.90)	60.4 (55.1–65.5)	98.9 (98.5–99.1)	76.8 (71.9–81.0)	97.6 (97.2–97.8)	0.80 (0.77–0.83)
Fourth week	8215	264 (3.21)	40.5 (34.6–46.7)	99.2 (99.0–99.4)	62.2 (55.4–68.6)	98.1 (97.9–98.2)	0.70 (0.66–0.74)
Fifth week	5515	179 (3.25)	34.1 (27.2–41.5)	99.8 (99.7–99.9)	87.1 (77.4–93.1)	97.8 (97.6–98.1)	0.67 (0.62–0.72)

SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; RT-PCR, reverse transcription-polymerase chain reaction; CI, confident interval.

Association between sensitivity of rapid antigen detection tests and cycle threshold value

Of 1899 individuals with RT-PCR positivity, 1397 (73.6%) had a cycle threshold (Ct) value. The overall mean (SD) Ct value was 24.4 (6.4). RAD tests had the highest sensitivity (81.1%) to identify individuals infected with SARS-CoV-2 when patients with COVID-19 had Ct values less than 20 (Table 4 ). There was a negative association between the sensitivity of the RAD tests and the Ct value in the study participants (p < .001).

Table 4

Sensitivity of the rapid antigen detection (RAD) test for the diagnosis of SARS-CoV-2 infection, by the cycle threshold value.

Ct value	Total number of COVID-19 cases	Number of rapid antigen test positivity	Sensitivity % (95% CI)
Overall	1397	833	59.60
<20	397	107	81.1%
20–24	326	370	71.8%
25–29	284	226	43.6%
≥30	390	107	24.2%

SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; COVID-19, coronavirus disease 2019; Ct, cycle threshold; CI, confident interval.

Trend for prevalence and number of COVID-19 cases in Taipei

Fig. 2 shows the prevalence and number of COVID-19 cases in a proactive community testing program in Taipei. The prevalence of SARS-COV-2 infection in Taipei significantly decreased from 8.4% at the beginning of the outbreak to 3.3% at the end of the study (p < .001). During the implementation of the PCT program, the number of newly reported COVID-19 cases per week significantly decreased (β = −110; p < .001).

Figure 2

Trend for the prevalence and number of COVID-19 cases in Taipei. COVID-19 = coronavirus disease 2019.

Discussion

This study showed that proactive community testing for SARS-CoV-2 infection using the RAD tests played an important role in the rapid identification and quarantine of the most infectious patients during the early phase of the COVID-19 outbreak in Taipei, from May to June 2021. Overall, the accuracy of RAD tests for the diagnosis of SARS-CoV-2 infection was good, with a c-statistic value of 0.8. There was a moderate concordance between RAD and RT-PCR assays to identify individuals with COVID-19 infection. Moreover, the lower the Ct value, the higher were the sensitivity rates of the RAD tests.

This study found that proactive community testing for SARS-CoV-2 infection played an important role in controlling the COVID-19 outbreak in Taipei. While Taipei experienced its first large-scale outbreak of SARS-CoV-2 infection in mid-May 2021, Taipei City Hospital set up 12 PCT stations and used the RAD for early identification of patients with COVID-19. When an individual's RAD test result was positive for SARS-COV-2 infection, the patient was admitted to the designated isolation centers. The result was later confirmed by the RT-PCR assay. Previous reports have shown that early identification and isolation of infectious individuals significantly reduce the prevalence of COVID-19. , Our study found that, after the implementation of large-scale PCT programs, the prevalence of SARS-COV-2 infection in Taipei significantly decreased from 8.4% at the beginning of the outbreak to 3.3% at the end of this study. These findings suggest that proactive testing for SARS-CoV-2 infection is an effective strategy to control the outbreak of COVID-19.

This study revealed that the RAD tests had good accuracy for the diagnosis of SARS-CoV-2 infection during the early phases of the COVID-19 outbreak. Despite RAD tests not being the “gold standard” when diagnosing SARS-CoV-2, they could provide results within 15–20 min, thus facilitating early quarantine of infectious individuals. Moreover, the RAD test results can be interpreted without specialized instruments, whereas RT-PCR assays require laboratory facilities with robust infrastructure and a highly trained staff. Our findings suggest that RAD tests could be strategic in identifying and isolating the most infectious individuals during the early phases of a COVID-19 outbreak.

This study showed a significant decrease in the accuracy of the RAD tests for the diagnosis of SARS-CoV-2 infection in the later phase of eliminating the virus from communities. Although the RAD tests could show the results quickly, the sensitivity and positive predictive values (PPVs) of the RAD tests vary according to the prevalence of SARS-CoV-2 and the level of Ct value in the screening population. , A recent meta-analysis reported that, at 5% prevalence of SARS-CoV-2 infection in symptomatic individuals, the PPVs of the RAD tests ranged from 84% to 90%, which means that between 1 in 10 and 1 in 6 positive results will be a false positive. However, when the prevalence of SARS-CoV-2 infection decreased to 0.5% in the screening population, the PPVs of the RAD tests were between 11% and 28%, which means that between 7 in 10 and 9 in 10 positive results will be false positives. Our study found that RAD tests had the high sensitivity and PPVs at the 8.4% prevalence of SARS-CoV-2 infection during the first week of the outbreak. Our findings suggest that RAD tests could identify the most infectious individuals in the early phase of the outbreak, while RT-PCR and comprehensive contact tracing strategy play more important roles in the control of SARS-CoV-2 in the end phase of the outbreak.

The accuracy of the RAD tests also varies according to the level of Ct value in the screening population. A previous report showed that the sensitivity of RAD tests ranged from 92.3% to 97.8% in patients with the Ct value < 25, while the sensitivity of RAD tests was between 35.6% and 65.8% in patients with the Ct value ≥25. Our study found that the lower the Ct value, the higher the sensitivity rate of the RAD tests. The high sensitivity rate of RAD tests among COVID-19 patients with low Ct values indicated that RAD tests could identify infectious individuals with high viral load in the early phase of the disease. Because the early diagnosis of patients at high risk of transmitting the infection to others is important to contain the spread of SARS-CoV-2, large-scale RAD test programs must be implemented to identify patients with COVID-19 at the beginning of the outbreak.

This study showed that proactive community testing for SARS-CoV-2 infection is an important strategy to control the COVID-19 outbreak in Taipei. Nevertheless, the present study has two limitations. First, information about the RAD test manufacturers were not available. As a result, the analysis cannot compare the sensitivity of RAD tests diagnosing SARS-CoV-2 infections among different RAD test manufacturers. Second, the external validity of our findings may be a concern because all our patients were Taiwanese. The generalizability of our results to other non-Asian ethnic groups requires further verification.

Conclusion

This study showed that the Taipei City government’ innovative use of RAD tests was able to rapidly identify and quarantine the most infectious patients during the early phase of the COVID-19 outbreak in Taipei, from May to June 2021. However, the accuracy of the RAD tests in diagnosing SARS-CoV-2 infection was poor in the later phase of eliminating the virus from communities, indicating that RT-PCR and comprehensive contact tracing strategy play more important roles on the control of SARS-CoV-2 in the end phase of outbreak. As early diagnosis of patients with COVID-19 is the key point to contain the spread of SARS-CoV-2, our findings suggest that it is imperative to implement large-scale PCT programs and use RAD tests to identify and isolate infectious individuals in the early phase of a COVID-19 outbreak.

Funding/support

This study was not funded.

Declaration of competing interest

The authors have no conflicts of interest relevant to this article.