The evaluation of a newly developed antigen test (QuickNavi™-COVID19 Ag) for SARS-CoV-2: A prospective observational study in Japan.

INTRODUCTION: Several antigen tests for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) have been developed worldwide, but their clinical utility has not been well established. In this study, we evaluated the analytical and clinical performance of QuickNavi™-COVID19 Ag, a newly developed antigen test in Japan.
METHODS: This prospective observational study was conducted at a PCR center between October 7 and December 5, 2020. The included patients were referred from a local public health center and 89 primary care facilities. We simultaneously obtained two nasopharyngeal samples with flocked swabs; one was used for the antigen test and the other for real-time reverse transcription PCR (RT-PCR). Using the results of real-time RT-PCR as a reference, the performance of the antigen test was evaluated.
RESULTS: A total of 1186 patients were included in this study, and the real-time RT-PCR detected SARS-CoV-2 in 105 (8.9%). Of these 105 patients, 33 (31.4%) were asymptomatic. The antigen test provided a 98.8% (95% confidence interval [CI]: 98.0%-99.4%) concordance rate with real-time RT-PCR, along with a sensitivity of 86.7% (95% CI: 78.6%-92.5%) and a specificity of 100% (95% CI: 99.7%-100%). False-negatives were observed in 14 patients, 8 of whom were asymptomatic and had a low viral load (cycle threshold (Ct) > 30). In symptomatic patients, the sensitivity was 91.7% (95% CI: 82.7%-96.9%).
CONCLUSION: QuickNavi™-COVID19 Ag showed high specificity and sufficient sensitivity for the detection of SARS-CoV-2. This test is a promising potential diagnostic modality especially in symptomatic patients.

Introduction

The pandemic of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), which causes coronavirus disease 2019 (COVID-19), has laid a detrimental burden on the healthcare system [1]. The effective isolation and early treatment of SARS-CoV-2 patients require rapid and accurate diagnostic methods [2].

Nucleic acid amplification tests (NAATs) for upper respiratory samples have been the mainstay for the identification of infected individuals [3]. However, while these assays are considered the gold-standard examinations, the disadvantages of their finite availability, long turnaround time, and need for skilled technicians have limited their clinical utility [4]. The number of patients eligible to undergo these tests may overwhelm the test capacity in outbreak settings [3]. Antigen tests are cheaper, more accessible point-of-care tests and take a shorter time to produce results; they can therefore be more useful in limited-resource settings, provided they reliably detect SARS-CoV-2.

The reported sensitivity of antigen tests has ranged from 0% to 94%, whereas the specificity is consistently high at >97% [3]. QuickNavi™-COVID19 Ag (Denka Co., Ltd., Tokyo, Japan) is a newly developed antigen test in Japan and employs a sandwich immunochromatography method with mouse monoclonal antibodies against SARS-CoV-2. The test result is available within 15 min after samples diluted in the buffer have been placed in a well of the test kit. Nevertheless, no study has yet examined its utility.

In the present study, we evaluated the analytical and clinical performance of QuickNavi™-COVID19 Ag using prospectively collected clinical samples. Furthermore, we analyzed the factors that might influence the sensitivity and specificity.

Patients and methods

We prospectively performed the study between October 7 and December 5, 2020, at a PCR center in Tsukuba, located in the southern part of Ibaraki Prefecture, Japan. During the COVID-19 endemic period, sample-collecting for PCR in the Tsukuba district was intensively performed with a drive-through-type method at the PCR center in Tsukuba Medical Center Hospital (TMCH). During the study period, additional samples for antigen test were collected from patients who have been referred from a local public health center and 89 primary care facilities (Supplementary Table 1) and healthcare workers of TMCH, and their clinical information was obtained after receiving the subjects’ informed consent. If patients had no clinical data, we excluded them from this study. In cases where patients participated in the current study more than once, only the first evaluation was included in this study.

The ethics committee of TMCH approved the present study (approval number: 2020-033).

Sample collection and procedures for antigen test

For sample collections, we simultaneously obtained two nasopharyngeal samples for antigen test and PCR examination with FLOQSwab™ (Copan Italia S.p.A., Brescia, Italy) as previously described [5]. Antigen test was performed immediately after sample collection according to the manufacturers’ instructions, described in Supplementary Figure 1, and the results were obtained by the visual interpretation of each examiner. Another swab sample was diluted in 3 mL of Universal Transport Medium™ (UTM™) (Copan Italia S.p.A., Brescia, Italy), and the UTM™ was transferred to an in-house microbiology laboratory located next to the drive-through sample-collecting place of the PCR center within an hour of sample collection.

PCR examinations for SARS-CoV-2 in this study

After the arrival of the UTM™ samples, purification and RNA extraction were performed with magLEAD 6gC (Precision System Science Co., Ltd., Chiba, Japan) from 200 μL aliquots of UTM™ for in-house reverse transcription PCR (RT-PCR) on the same day as sample collection. The RNA was eluted in 100 μL and stored at −80 °C after in-house RT-PCR. The eluted samples were transferred to Denka Co., Ltd., every week for reference real-time RT-PCR of SARS-CoV-2 using a method developed by the National Institute of Infectious Diseases, Japan [6]. If discordance was recognized between the reference real-time RT-PCR and in-house RT-PCR, a re-evaluation was performed with a BioFire® Respiratory Panel 2.1 and FilmArray® systems (BioFire Diagnostics, LLC, UT, USA), and the final judgment was made.

Limit of detection of QuickNavi™-COVID19 Ag

The limit of detection of QuickNavi™-COVID19 Ag was investigated as follows: the 2019-nCoV/JPN/TY/WK-521 strain (4.2 × 105 TCID50/mL) cultured in VeroE6/TMPRSS2 cells were diluted two-fold stepwise with QuickNavi™ specimen buffer and used as samples. Each sample with different concentrations was tested in triplicate. As shown in Table 1 , the limit of detection was 5.3 × 101 TCID50/mL and was consistent throughout the test.

Table 1

The limit of detection test results of three repetitive tests for each sample.

Concentration	Results
(TCID50/mL)	Sample A	Sample B	Sample C
2.1 × 102	+	+	+
1.1 × 102	+	+	+
5.3 × 101	+	+	+
2.6 × 101	–	–	–

TCID50, median tissue culture infectious dose.

Statistical analyses

The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of antigen test were calculated using the Clopper and Pearson method, with 95% confidence intervals (CIs). Categorical variables were compared by Fisher’s exact test. P-values <0.05 were considered to represent statistically significant differences. All calculations were conducted using the R 3.3.1 software program (The R Foundation, Vienna, Austria).

Results

Of the 2079 referred patients and 45 healthcare workers, a total of 1208 individuals who had nasopharyngeal samples collected for antigen test and had provided their informed consent were initially included. We excluded the patients who were duplicates (n = 18) or missing symptom data (n = 4). We finally included 1186 subjects for the analysis.

Most samples were obtained at the drive-through PCR center, and only 15 were obtained after hospitalization. Of the 1186 subjects, SARS-CoV-2 was detected in 105 (8.9%) by reference real-time RT-PCR. There was one discordant sample that showed positivity on in-house RT-PCR and negativity on reference real-time RT-PCR. The sample was deemed negative by an additional BioFire® Respiratory Panel 2.1 examination. Of the 105 subjects, 72 (68.6%) were symptomatic, and 33 (31.4%) were asymptomatic (Table 2 a). Asymptomatic patients were examined for the purpose of contact tracing for COVID-19.

Table 2a

Demographic data of the whole study population and cases infected with SARS-CoV-2.

	Total	SARS-CoV-2
		Positive	Negative
n	1186	105	1081
Age (years, median [IQR])	36.5 [23.0, 50.0]	47.0 [30.0, 58.0]	36.0 [23.0, 49.0]
<18	164 (13.8)	11 (10.5)	153 (14.2)
18–64	898 (75.7)	79 (75.2)	819 (75.8)
≥ 65	124 (10.5)	15 (14.3)	109 (10.1)
Sex (Female, %)	539 (45.4)	43 (41.0)	496 (45.9)
Asymptomatic patients	415 (35.0)	33 (31.4)	382 (35.3)
Symptomatic patients	771 (65.0)	72 (68.6)	699 (64.7)

SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

The characteristics of the symptomatic subjects and cases infected with SARS-CoV-2 are described in Table 2 b. Of the symptomatic SARS-CoV-2-positive cases (n = 72), the most common symptom was fever (72.2%), followed by cough or sputum production (41.7%), sore throat (23.6%), fatigue (18.1%) and headache (18.1%).

Table 2b

Characteristics of symptomatic patients and cases infected with SARS-CoV-2.

	Total	SARS-CoV-2
		Positive	Negative
n	771	72	699
Days from symptom onsetto sample collection [IQR]	2.0 [1.0, 4.0]	3.0 [1.0, 4.8]	2.0 [1.0, 4.0]
Signs and symptoms (%)
Fever	617 (80.0)	52 (72.2)	565 (80.8)
Cough/sputum production	294 (38.1)	30 (41.7)	264 (37.8)
Runny nose/nasal congestion	196 (25.4)	10 (13.9)	186 (26.6)
Loss of taste or smell	33 (4.3)	9 (12.5)	24 (3.4)
Dyspnea	6 (0.8)	3 (4.2)	3 (0.4)
Fatigue	77 (10.0)	13 (18.1)	64 (9.2)
Diarrhea	44 (5.7)	4 (5.6)	40 (5.7)
Sore throat	149 (19.3)	17 (23.6)	132 (18.9)
Headache	83 (10.8)	13 (18.1)	70 (10.0)

SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

Sensitivity, specificity, PPV, and NPV of antigen test

Of the 105 cases that were positive on reference real-time RT-PCR, antigen test was also positive in 91 (Table 3 a). The concordance rate between antigen test and real-time RT-PCR was thus 98.8% (95% CI: 98.0%–99.4%). The sensitivity, specificity, PPV, and NPV were 86.7% (95% CI: 78.6%–92.5%), 100% (95% CI: 99.7%–100%), 100% (95% CI: 96.0%–100%), and 98.7% (95% CI: 97.9%–99.3%), respectively (Table 3a).

Table 3a

Sensitivity and specificity of the QuickNavi™-COVID19 Ag among overall patients.

		real-time RT-PCR
		Positive	Negative
Antigen test	Positive	91	0
	Negative	14	1081
Sensitivity (%)		86.7 (78.6–92.5)
Specificity (%)		100 (99.7–100)
Positive predictive value (%)		100 (96.0–100)
Negative predictive value (%)		98.7 (97.9–99.3)

Sensitivity, specificity, positive predictive value, and negative predictive value are provided with 95% confident intervals.RT-PCR, reverse transcription polymerase chain reaction

Of the 72 symptomatic cases that were positive on reference real-time RT-PCR, antigen test was also positive in 66 (Table 3 b). The sensitivity, specificity, PPV, and NPV were 91.7% (95% CI: 82.7%–96.9%), 100% (95% CI: 99.5%–100%), 100% (95% CI: 94.6%–100%), and 99.1% (95% CI: 98.2%–99.7%) (Table 3b).

Table 3b

Sensitivity and specificity of the QuickNavi™-COVID19 Ag among symptomatic patients.

		real-time RT-PCR
		Positive	Negative
Antigen test	Positive	66	0
	Negative	6	699
Sensitivity (%)		91.7 (82.7–96.9)
Specificity (%)		100 (99.5–100)
Positive predictive value (%)		100 (94.6–100)
Negative predictive value (%)		99.1 (98.2–99.7)

Sensitivity, specificity, positive predictive value, and negative predictive value are provided with 95% confident intervals.

RT-PCR, reverse transcription polymerase chain reaction.

In asymptomatic patients, the sensitivity, specificity, PPV, and NPV were 75.8% (95% CI: 57.7%–88.9%), 100% (95% CI: 99.0%–100%), 100% (95% CI: 86.3%–100%), and 92.0% (95% CI: 89.0%–94.5%), respectively.

Detailed data of discrepant cases between antigen test and real-time RT-PCR examinations

Among the 14 discrepant cases, 8 were asymptomatic, and 4 of the 6 symptomatic cases had their nasopharyngeal samples taken ≥6 days after the onset of symptoms. The N2-gene was detected in all cases, but the N1-gene was not detected in 7 cases. One patient had a history of preceding favipiravir administration (Table 4 ).

Table 4

Detailed data of the 14 cases with discrepant findings between antigen test and real-time RT-PCR.

Case number	Symptoms	Days from thesymptom onsetto sample collection	Ct value		Notes
			N1a	N2a
1	+	7	ND	34
2	+	3	ND	40
3	+	6	31	24
4	+	6	35	30	Preceding favipiravir administration
5	+	4	38	35
6	–	NA	ND	36
7	–	NA	ND	39
8	–	NA	ND	37
9	–	NA	37	31
10	–	NA	ND	38
11	+	7	36	30
12	–	NA	35	30
13	–	NA	41	39
14	–	NA	ND	34

Ct, cycle threshold; NA, not available; ND, not detected.

Real-time reverse transcription polymerase chain reaction examinations of SARS-CoV-2 developed by the National Institute of Infectious Diseases, Japan [6].

Change of sensitivities of antigen test stratified by cycle threshold (Ct) value

The sensitivity of Ct value (N2) <20 was 100% (95% CI: 91.0%–100%), that of Ct 20–24 was 96.7% (95% CI: 82.8%–99.9%), and that of Ct 25–29 was 100% (95% CI: 83.2%–100%) (Table 5 ). In contrast, the sensitivity of Ct ≥ 30 was 18.8% (95% CI: 4.0%–45.6%) (Table 5).

Table 5

Sensitivities of antigen test stratified by Ct value.

Ct value (N2)	Sensitivity (%)	p
<20	100 (91.0–100)	<0.001
20–24	96.7 (82.8–99.9)
25–29	100 (83.2–100)
≥30	18.8 (4.0–45.6)

Sensitivity is provided with 95% confidence intervals.

Ct, cycle threshold.

Discussion

Among 1186 subjects referred from clinics and a local healthcare center in the southern part of Ibaraki Prefecture, Japan, this prospective study indicated that QuickNavi™-COVID19 Ag has satisfactory performance for the detection of SARS-CoV-2. Of note, the test provided no false-positive results in our study population. False-negatives were detected in 14 subjects, over half of whom were asymptomatic.

False-positives should be avoided due to concerns about unnecessary further examinations or application of quarantine measures [7]. NAATs are highly specific for SARS-CoV-2, and positive results are usually definitive for the diagnosis of COVID-19 [3]. False-positives are rare and they tend to only be observed under exceptional conditions such as cross contaminations, erroneous handling of samples, or a breakdown in test reagents or equipment [8]. Similar to NAATs, antigen tests generally have high specificities of >99% [9]. Nevertheless, some false-positive results have been reported in other antigen tests [10,11]. While definitive proof is lacking, possible causes for the false-positives include the high viscosity of specimens and interference of human antibodies [12]. The QuickNavi™-COVID19 Ag provided 100% specificity in our study, which exceeded the performance recommended by the World Health Organization (WHO) [3]. However, the test may show false-positives in the general population outside this study, and any positive results should be carefully adjudicated if the diagnosis of COVID-19 is unlikely. Whether or not a similar result can be obtained in different settings needs to be confirmed.

The sensitivity of QuickNavi™-COVID19 Ag was 86.7% overall, and the positive detection rate was comparable to the real-time RT-PCR in patients with Ct < 30. Eight of 14 false-negative subjects had no symptoms and a low viral load, although conflicting evidence exists regarding the relationship between symptom severity and viral shedding [13,14]. All samples were collected from a nasopharynx with flocked swabs, which may have increased the viral load and improved antigen test sensitivity in our study. The viral load on the nasopharynx is generally higher than in the nasal cavity or saliva [15,16], and flocked swabs can yield more samples than rayon swabs [17].

The utility of antigen tests for screening purposes is controversial. The WHO guidelines basically recommend against antigen tests for screening purposes [3]. In contrast, European countries allow antigen tests for screening or serial testing [18]. Recent studies may support this use of antigen tests, showing the frequency and turnaround time of the tests to be great contributors to an effective screening strategy [19]. Since the QuickNavi™-COVID19 Ag may effectively identify highly infectious patients (generally Ct < 25 [20]), the test may be beneficial for screening purposes.

Several limitations associated with the present study warrant mention. First, reference real-time RT-PCR examinations employed frozen samples. Despite all samples being frozen at −80 °C, their viral load may have been reduced through the storage process. Second, although we investigated whether or not the intervals between the symptom onset and examination timing influenced the performance of the antigen test, the sample size was not sufficient to draw a definitive conclusion (Supplementary Figure 2). Third, using anterior nasal samples was beyond the scope of this study. Sample collection from the anterior nasal cavity is less invasive than that from the nasopharynx and is now approved for QuickNavi™-COVID19 Ag [21]. The clinical performance of the test with these samples has not yet been evaluated, and further research is necessary.

In conclusion, the QuickNavi™-COVID19 Ag showed very high specificity and sufficient sensitivity for the detection of SARS-CoV-2. Given the simple procedures and shorter turnaround time involved with this test, it is a promising option as an alternative diagnostic modality especially in symptomatic patients.

Author statement

All authors meet the ICMJE authorship criteria.

Contributor Yuto Takeuchi drafted the manuscript and performed the statistical analyses. Yusaku Akashi was the chief investigator and responsible for the data analysis. Hiromichi Suzuki supervised the project. All authors contributed to the writing of the final manuscript.

Declaration of competing interest

Denka Co., Ltd., provided fees for research expenses and the QuickNavi™-COVID19 Ag kits without charge. Hiromichi Suzuki received a lecture fee from , regarding this study. Daisuke Kato, Miwa Kuwahara and Shino Muramatsu belong to Denka Co., Ltd., the developer of the QuickNavi™-COVID19 Ag.