Prospective analytical performance evaluation of the QuickNavi™-COVID19 Ag for asymptomatic individuals.

INTRODUCTION: Antigen testing may help screen for and detect severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infections in asymptomatic individuals. However, limited data regarding the diagnostic performance of antigen tests for this group are available.
METHODS: We used clinical samples to prospectively evaluate the analytical and clinical performance of the antigen test QuickNavi™-COVID19 Ag. This study was conducted at a PCR center between October 7, 2020 and January 9, 2021. Two nasopharyngeal samples per patient were obtained with flocked swabs; one was used for the antigen test, and the other for real-time reverse transcription PCR (RT-PCR). The diagnostic performance of the antigen test was compared between asymptomatic and symptomatic patients, and the RT-PCR results were used as a reference.
RESULTS: Among the 1934 collected samples, 188 (9.7%) demonstrated detection of SARS-CoV-2 by real-time RT-PCR; 76 (40.4%) of these 188 samples were from asymptomatic individuals, and over half of the total samples were asymptomatic (1073; 55.5%). The sensitivity of the antigen test was significantly lower for the asymptomatic group than for symptomatic patients (67.1% vs. 89.3%, respectively, p < 0.001). The specificity was 100% for both groups, and no false positives were observed among all 1934 samples. The median cycle threshold value for the asymptomatic group was significantly higher than that of the symptomatic group (24 vs. 20, p < 0.001).
CONCLUSIONS: The QuickNavi™-COVID19 Ag showed lower sensitivity for the asymptomatic group than for symptomatic patients. However, its specificity was consistently high, and no false positives were found in this study.

Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), which causes coronavirus disease 2019 (COVID-19), has overwhelmed healthcare systems globally [1]. The early identification and isolation of patients infected with SARS-CoV-2 are essential for constraining COVID-19 transmission.

Travel restrictions have been enforced worldwide to impede the spread of SARS-CoV-2 [2], and many countries have implemented immigration screening measures to minimize the risk of travelers bringing the virus into the country with them [3]; however, the need for resuming domestic and international movement is growing. According to recent data, symptom-based screening, including body-temperature screening, fails to detect a substantial number of SARS-CoV-2-infected patients who have no or mild symptoms [4]. Thus, more accurate screening methods, ideally ones that are convenient and provide rapid results, are desired to detect such individuals.

Although nucleic acid amplification tests (NAATs) are considered highly reliable for detecting SARS-CoV-2, the disadvantages of their finite availability, long turnaround time, and requirement for skilled technicians to perform them have limited their utility for screening purposes [5]. Antigen tests are more accessible point-of-care tests, and they generally take less than an hour to produce results. They can therefore be more beneficial for use in SARS-CoV-2 screening, if their diagnostic performance is sufficient. However, data on the performance of antigen tests in asymptomatic individuals is currently scarce.

Our previous study demonstrated that the antigen test QuickNavi™-COVID19 Ag (Denka, Tokyo, Japan) had good performance in the detection of patients with COVID-19, with a sensitivity of 86.7% (95% CI: 78.6%–92.5%) and specificity of 100% (95% CI: 99.7%–100%) in 1186 patients [6]. However, only a few asymptomatic subjects were included in that study, so the diagnostic performance of the QuickNavi™-COVID19 Ag in asymptomatic individuals could not be thoroughly evaluated.

In the present prospective study, we aimed to evaluate the analytical and clinical performance of the QuickNavi™-COVID19 Ag in asymptomatic individuals. This study was conducted as an extension study of our previous report [6].

Patients and methods

The details of our study protocol were described previously [6]. Briefly, we prospectively performed this study between October 7, 2020 and January 9, 2021. Sample collection was performed at the PCR center in Tsukuba Medical Center Hospital (TMCH). We enrolled participants with suspected SARS-CoV-2 infection based on their symptoms or known contact histories with COVID-19 confirmed/suspected patients. The included participants were TMCH healthcare workers and those who referred from a local public health center, and 97 primary care facilities. Clinical information was obtained from each participant.

All samples from the same patients collected at different timepoints were included in the analysis. The ethics committee of TMCH approved the present study (approval number: 2020–033).

Sample collection and antigen test procedure

For sample collection, we simultaneously obtained two nasopharyngeal samples: one sample for use in the antigen test, and the other for use in a PCR examination. The antigen test was performed using the QuickNavi™-COVID19 Ag in accordance with the manufacturers’ instructions. The other swab sample was transferred to an in-house microbiology laboratory within an hour of sample collection.

PCR examinations for SARS-CoV-2

Purification and RNA extraction was performed with magLEAD 6gC (Precision System Science, Chiba, Japan) for in-house reverse transcription PCR (RT-PCR) with GENECUBE® (TOYOBO Co., Ltd., Osaka, Japan). Although GENECUBE assays had been approved for In Vitro Diagnostic, the method developed by the National Institute of Infectious Diseases (NIID), Japan [7] is the national gold standard method of RT-PCR for SARS-CoV-2 in Japan. Hence, we additionally performed reference real-time RT-PCR with residual samples, using a method developed by NIID. Since the possibility of false negatives with the NIID method was reported [8], we underwent re-evaluation with GeneXpert® for SARS-CoV-2 (Cepheid, Sunnyvale, CA, USA) for samples with discrepant results between the two assays.

Statistical analyses

The sensitivity and specificity of the QuickNavi™-COVID19 Ag were calculated using the Clopper and Pearson method, with 95% confidence intervals (CIs). Cycle threshold (Ct) values were compared between groups using Mann-Whitney U tests, and p-values of <0.05 were considered to indicate statistically significant differences. Categorical variables were compared by using Fisher's exact test. All calculations were conducted using the R 4.0.3 software program (www.r-project.org).

Results

During the study period, we evaluated 1939 nasopharyngeal samples taken from 1881 participants. After excluding those collected from subjects for whom symptom data were unavailable (n = 5), we were left with 1934 samples for the final analysis. Of these 1934 samples, 1073 (55.5%) were from asymptomatic individuals.

SARS-CoV-2 was detected by both in-house and reference real-time RT-PCR in 187 samples. One discordant sample showed positive results with in-house RT-PCR but negative results with reference real-time RT-PCR. This sample was deemed positive for SARS-CoV-2 following an additional examination using GeneXpert for SARS-CoV-2 (Ct value of the N2 gene: 42.6). Thus, 188 of 1934 total samples (9.7%) were assessed as positive for SARS-CoV-2. Of the 188 SARS-CoV-2-positive samples, 76 (40.4%) were from asymptomatic individuals. The median Ct values for the N2 gene in samples from symptomatic and asymptomatic participants were 20 and 24, respectively (Fig. 1 ).

Fig. 1

Ct values of samples from symptomatic and asymptomatic subjects. The number next to each box indicates the median Ct value.

Sensitivity and specificity of the antigen test in asymptomatic individuals

Table 1 compares QuickNavi™-COVID19 Ag assay results with reference real-time RT-PCR results in asymptomatic and symptomatic participants. The sensitivity and specificity for the asymptomatic group were 67.1% (95% CI: 55.4%–77.5%) and 100% (95% CI: 99.4%–100.0%), respectively. Of the 25 samples with discordant results between these two assays, all were assessed as negative by the QuickNavi™-COVID19 Ag assay and assessed as positive by the reference real-time RT-PCR assay (Table 1).

Table 1

Comparison of QuickNavi™-COVID19 Ag performance between asymptomatic and symptomatic subjects.

		Asymptomatic participants (n = 1073)		Symptomatic participants (n = 861)
		Real-time RT-PCR
		positive	negative	positive	negative
QuickNavi™-COVID19 Ag	Positive	51	0	100	0	p-value
	Negative	25	997	12	749
Sensitivity (%)		67.1 (55.4–77.5)		89.3 (82.0–94.3)		<0.001
Specificity (%)		100 (99.4–100)		100 (99.3–100)		1.000
Positive predictive value (%)		100 (89.7–100)		100 (94.6–100)		1.000
Negative predictive value (%)		97.6 (96.4–98.4)		98.4 (97.3–99.2)		0.919

Reverse transcription-polymerase chain reaction was used as the reference.

RT-PCR, reverse transcription-polymerase chain reaction.

Data in parentheses are 95% confidence intervals.

The overall sensitivity and specificity of the antigen test were 80.3% (95% CI: 73.9–85.7%) and 100% (95% CI: 99.7–100%), respectively, and no false-positive results were identified among the 1934 samples. The sensitivity of QuickNavi™-COVID19 Ag was 89.3% (95% CI: 82.0%–94.3%) for symptomatic patients, which is significantly higher than its sensitivity for asymptomatic individuals (p < 0.001). Additionally, the QuickNavi™-COVID19 Ag sensitivities stratified by symptom occurrence and Ct value are shown in Table 2 .

Table 2

Antigen test sensitivities stratified by Ct values.

Ct value (N2 gene)	Total (n = 187a)		Asymptomatic (n = 75a)		Symptomatic (n = 112)
	Sensitivity	n	Sensitivity	n	Sensitivity	n
<20	100 (91.7–100)	64	100 (71.3–100)	16	100 (89.1–100)	48
20–24	98.2 (90.6–100)	57	100 (78.1–100)	22	97.1 (85.1–99.9)	35
25–29	86.2 (68.3–96.1)	29	90.9 (58.7–99.8)	11	83.3 (58.6–96.4)	18
≥30	16.2 (6.2–32.0)	37a	11.5 (2.4–30.2)	26a	27.3 (6.0–61.0)	11

Data in parentheses are 95% confidence intervals.

Ct, cycle threshold.

Results omitted for one discordant sample with positive results by in-house RT-PCR and GeneXpert for SARS-CoV-2 but negative results by reference real-time RT-PCR.

Discussion

Among the 1073 samples collected from asymptomatic participants, the QuickNavi™-COVID19 Ag showed a sensitivity of 67.1% (95% CI: 55.4–77.5%) and a specificity of 100% (95% CI: 99.4–100.0%). The sensitivity of this test for asymptomatic individuals was significantly lower than its sensitivity for symptomatic patients. The antigen test yielded no false-positive results in any of the 1934 samples tested.

Studies of SARS-CoV-2 outbreaks have found that asymptomatic individuals comprise a significant portion of the infected population. [9]. A study of the outbreak on the Japanese cruise ship Diamond Princess reported that 328 out of the 634 confirmed SARS-CoV-2-infected patients were asymptomatic at the time of diagnosis [10]. It has been suggested that infections in asymptomatic individuals play an essential role in the SARS-CoV-2 epidemic [11], and efficient detection of asymptomatic individuals is necessary to control outbreaks [12]. Because a symptom-based SARS-CoV-2 screening approach is incapable of detecting asymptomatic infected individuals, large-scale testing is needed for successful contact tracing.

There is limited data on the performance of antigen tests in asymptomatic SARS-CoV-2-infected individuals [13]. The current study found that the sensitivity of our antigen test was lower in asymptomatic individuals than in symptomatic patients (67.1% vs. 89.3%). A few studies have observed a similar trend of lower antigen test sensitivity in asymptomatic individuals (45.4% vs. 79.1% [14], 53.3% vs. 84.6% [15]). The median viral load in our UTM samples was significantly lower in participants without symptoms (Fig. 1) [15], which could explain the differences in antigen test sensitivity between symptomatic and asymptomatic subjects. Nevertheless, the viral loads in the UTM samples may not directly reflect those in the samples used for antigen testing, since two samples were separately collected from each participant for RT-PCR analysis and antigen testing.

Among the samples with Ct values of <30, the QuickNavi™-COVID19 Ag showed a sensitivity of >80% for both symptomatic and asymptomatic subjects. This sensitivity met the performance requirement by the World Health Organization, which suggests that a sensitivity of ≥80% is “acceptable” in samples [16]. Of the 37 false-negative samples, the majority had a low viral concentration, and only five had Ct values of <30. As indicated in a previous report, patients with low viral shedding have low infectivity [17]. Thus, the risk of overlooking high infectivity patients with COVID-19 in cases when the antigen test provides negative results seems limited. However, asymptomatic individuals in the early phase of SARS-CoV-2 infection may later develop symptoms with progressively increasing viral shedding [9]. Therefore, in addition to conducting COVID-19 screenings, symptom follow-up in asymptomatic individuals is essential.

A controversy exists over the use of antigen tests for screening purposes. The World Health Organization (WHO) has opposed using antigen testing as a screening tool [18]. By contrast, the United States Centers for Disease Control and Prevention (CDC) has acknowledged that screening may be conducted with antigen testing in high-risk congregate settings such as nursing homes [19,20]. In this study, the sensitivity of QuickNavi™-COVID19 Ag in asymptomatic participants was lower than 80%, even though the pre-test probabilities in our participants were considered high based on their symptoms or close contact histories. Next action after obtaining a negative antigen test result may differ according to the individual's pre-test probability. The United States Food and Drug Administration suggests that a negative antigen test result should be followed by more sensitive NAATs in high-risk settings, whereas serial antigen testing is sufficient for screening in the general population [21]. This strategy seems to be supported by a modeling study whose results showed that a high testing frequency was more critical than high test sensitivity when screening the general population [21]. Still, the real-life effectiveness of antigen tests as a screening tool has not been established, and the United States National Institutes of Health and CDC are currently investigating the benefits of mass screening with antigen testing for residents in communities [22].

Several limitations associated with the present study warrant mention. First, the results were obtained at a single PCR center during one epidemic season. Whether the same results would be obtained in other regions or during other epidemics requires additional validation. Second, this study did not include patients who had received the vaccine, and the accuracy of the test on SARS-CoV-2-infected patients after vaccination needs to be verified in the future. Third, we did not perform a genetic analysis of the detected SARS-CoV-2 variants and did not study the effect of genetic mutation on the antigen test results. Nevertheless, according to manufacturer's information for use (version 4.0), QuickNavi™-COVID19 Ag reacts with both the SARS-CoV-2 UK variant (VOC-202012/01) and the Brazilian variant (501Y·V3, P.1), and the degrees of reaction with these variants are the same as those with the Wuhan strain.

In conclusion, despite showing very high specificity, the QuickNavi™-COVID19 Ag has lower sensitivity for detecting SARS-CoV-2 in asymptomatic individuals as compared with its sensitivity in symptomatic patients. Nevertheless, given its high convenience and specificity, this antigen test can be used as a supplementary COVID-19 assessment for asymptomatic individuals under certain settings or circumstances, as long as prevalence and testing frequency are considered and most of all, the results are interpreted appropriately.

Author statement

Contributor Yoshihiko Kiyasu drafted the manuscript and performed the statistical analyses. Yuto Takeuchi and Yusaku Akashi analyzed the data and revised the manuscript. Hiromichi Suzuki supervised the project. All authors contributed to the writing of the final manuscript.

Declaration of competing interest

Denka Co., Ltd. Provided fees for research expenses and provided the QuickNavi-COVID19 Ag kits without charge. Hiromichi Suzuki received a lecture fee from Otsuka Pharmaceutical Co., Ltd. regarding this study. Daisuke Kato, Miwa Kuwahara, and Shino Muramatsu belong to Denka Co., Ltd., the developer of the QuickNavi™ -COVID19 Ag.