A prospective evaluation of diagnostic performance of a combo rapid antigen test QuickNavi-Flu+COVID19 Ag.

INTRODUCTION: Since respiratory sample collection is an uncomfortable experience, simultaneous detection of pathogens with a single swab is preferable. We prospectively evaluated the clinical performance of a newly developed antigen test QuickNavi-Flu+COVID19 Ag (Denka Co., Ltd., Tokyo, Japan) which can detect severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and influenza viruses at the same time with a single testing device.
METHODS: We included those who were suspected of contracting coronavirus disease 2019 (COVID-19) and were referred to a PCR center at Ibaraki prefecture in Japan, between August 2, 2021 to September 13, 2021, when the variant carrying L452R spike mutation of SARS-CoV-2 were prevalent. Additional nasopharyngeal samples and anterior nasal samples were obtained for the antigen test and were compared with a reference real-time reverse transcription PCR (RT-PCR) using nasopharyngeal samples.
RESULTS: In total, 1510 nasopharyngeal samples and 862 anterior nasal samples were evaluated. During the study period, influenza viruses were not detected by QuickNavi-Flu+COVID19 Ag and reference real-time RT-PCR. For SARS-CoV-2 detection in nasopharyngeal samples, the sensitivity and specificity of the antigen test were 80.9% and 99.8%, respectively. The sensitivity and specificity using anterior nasal samples were 67.8% and 100%, respectively. In symptomatic cases, the sensitivities increased to 88.3% with nasopharyngeal samples and 73.7% with anterior nasal samples. There were three cases of discrepant results between the antigen test and the real-time RT-PCR. All of them were positive with the antigen test but negative with the real-time RT-PCR in SARS-CoV-2 detection.
CONCLUSION: A combo kit, QuickNavi-Flu+COVID19 Ag, showed an acceptable sensitivity and sufficient specificity for SARS-CoV-2 detection, especially using nasopharyngeal sample collected from symptomatic patients.

The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic is still continuing worldwide and remains a major health problem [1]. In addition, although there is a concern about the simultaneous outbreak of SARS-CoV-2 and influenza viruses in winter season, it can be difficult to distinguish coronavirus disease 2019 (COVID-19) from influenza based on clinical symptoms [2].

Antigen testing is simpler and faster than PCR testing and therefore is expected to play a role in COVID-19 infection control [3]. In general, it is necessary to collect separate samples for SARS-CoV-2 and for influenza viruses to perform two different antigen testing, though the multiple collections of respiratory samples are accompanied by discomfort. The newly developed combo rapid antigen test QuickNavi-Flu+COVID19 Ag (Denka Co., Ltd., Tokyo, Japan) consists of the main parts of QuickNavi-COVID19 Ag (Denka Co., Ltd., Tokyo, Japan) and QuickNavi-Flu2 (Denka Co., Ltd., Tokyo, Japan) that are already in market. QuickNavi-Flu+COVID19 Ag can detect SARS-CoV-2 and influenza viruses simultaneously using nasopharyngeal or anterior nasal specimens, and the read time has been shortened compared with QuickNavi-COVID19 Ag [4]. Although QuickNavi-Flu+COVID19 Ag is commercially available in Japan, its usefulness in clinical practice has not yet been fully assessed.

In this study, we prospectively evaluated the clinical performance of the combo kit QuickNavi-Flu+COVID19 Ag using nasopharyngeal and anterior nasal samples, with a real-time reverse transcription PCR (RT-PCR) as a reference testing. In addition, we examined for agreement among the results of QuickNavi-Flu+COVID19 Ag, QuickNavi-COVID19 Ag and QuickNavi-Flu2.

This study was conducted at a PCR center in Tsukuba Medical Center Hospital (TMCH) located at Ibaraki prefecture, Japan between August 2, 2021 and September 13, 2021. We prospectively enrolled the individuals suspected to have COVID-19. Included individuals were those who were referred to the PCR center by nearby clinics and a local public health center as well as those who work at TMCH. The study includes asymptomatic individuals who had the history of close contacts with someone tested positive for COVID-19 and were referred by the local public health center. During this study period, Japan was affected by the fifth wave of SARS-CoV-2 outbreak mainly caused by the Delta variant [5]. Nasopharyngeal samples and clinical data were obtained from all patients suspected of COVID-19 and the samples were tested with an in-house PCR. In addition, we collected a nasopharyngeal sample and/or an anterior nasal sample for the evaluation of QuickNavi-Flu+COVID19 Ag with verbal informed consent. We excluded those who lacked clinical data. This study was approved by the ethics board of the University of Tsukuba Hospital Research Ethics Review Committee (approval number: R03-042).

We collected anterior nasal sample according to the manufacturer's instructions, namely, inserting a swab to 2 cm in depth to one nasal cavity, rotating it five times, and holding it in place for 5 s. After that, nasopharyngeal samples were collected by a recommended procedure [6]. All the anterior nasal and nasopharyngeal samples were collected with FLOQSwab (Copan Italia S.p.A., Brescia, Italy) by medical professionals. The samples for the antigen test were collected at the same time or on the same day the positive result of in-house PCR was found.

All antigen tests were performed on site. The swab samples were soaked in the suspension buffer included in QuickNavi-Flu+COVID19 Ag kit. From the same prepared sample, three drops were added to the device in the order of QuickNavi-Flu+COVID19 Ag (Lot No. 0041081), QuickNavi-COVID19 Ag (Lot No. 0971071), and QuickNavi-Flu2 (Lot No. 0559121).

Nasopharyngeal samples collected for in-house PCR of SARS-CoV-2 were suspended in 3 mL of Universal Transport Medium (UTM) (Copan Italia S.p.A.), and the RNA was extracted with magLEAD 6gC (Precision System Science Co., Ltd., Chiba, Japan). After performing the in-house PCR [7], the remaining eluted RNA were stored at −80 °C and were transferred to Denka Co., Ltd. for reference real-time RT-PCR of SARS-CoV-2 and influenza A/B virus. The reference real-time RT-PCR methods for SARS-CoV-2 and influenza A/B virus were developed by the National Institute of Infectious Diseases (NIID), Japan [8,9]. The result of SARS-CoV-2 was reported as positive when the cycle threshold (Ct) value for N2 set was ≤ 40 [10]. The result of influenza viruses was reported as positive when the Ct value of the reaction to M gene for type A or NS gene for type B was ≤ 40. In case of discrepancy between the in-house PCR and the reference real-time RT-PCR for SARS-CoV-2, re-evaluation using the Xpert Xpress SARS-CoV-2 and GeneXpert System (Cepheid, Sunnyvale, CA, USA) was performed, and the results were used as the final judgment (Supplementary Table 1).

The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of QuickNavi-Flu+COVID19 Ag were calculated for both nasopharyngeal and anterior nasal samples, separately using the results of the nasopharyngeal RT-PCR as comparator. Sensitivities stratified by Ct values based on N2 set in the reference real-time RT-PCR were also calculated for those with available data. If the results of in-house PCR and reference real-time RT-PCR deviated and the sample was determined to be positive by retesting with Xpert Xpress SARS-CoV-2, it was not included in the stratified data of Ct values. The 95% confidence intervals (CIs) were calculated with the Clopper and Pearson method. The Ct values between the symptomatic and asymptomatic cases were compared by Mann-Whitney U test. All statistical analyses were conducted using the R 3.5.2 software program (The R Foundation, Vienna, Austria).

During the study period, 2375 samples were tested. Three nasopharyngeal samples without clinical data were excluded. Finally, 1510 nasopharyngeal samples and 862 anterior nasal samples were included in the analysis. Of the 862 anterior nasal samples, 727 were collected simultaneously as nasopharyngeal samples for in-house PCR, and the rest of 135 were collected on the same day but after confirming SARS-CoV-2 positive results on in-house PCR. The number of symptomatic and asymptomatic cases in the nasopharyngeal and anterior nasal groups are shown in Table 1 and Table 2 , respectively. The median number of days from onset to sample collection for symptomatic cases were 2.0 (interquartile range [IQR]: 1.0–4.0) in nasopharyngeal sample group and 2.0 (IQR: 1.0–3.0) in anterior nasal sample group.

Table 1

SARS-CoV-2 diagnostic performance of QuickNavi-Flu+COVID19 Ag in nasopharyngeal samples.

		All (N = 1510)		Symptomatic (N = 636)		Asymptomatic (N = 874)
Results of RT-PCR using nasopharyngeal samples*		Positive	Negative	Positive	Negative	Positive	Negative
QuickNavi-Flu+COVID19 Ag	Positive	228	3	151	0	77	3
	Negative	54	1225	20	465	34	760
Sensitivity (%)		80.9 (75.8–85.3)		88.3 (82.5–92.7)		69.4 (59.9–77.8)
Specificity (%)		99.8 (99.3–99.9)		100 (98.8–100)		99.6 (98.9–99.9)
Positive predictive value (%)		98.7 (96.3–99.7)		100 (96.4–100)		96.3 (89.4–99.2)
Negative predictive value (%)		95.8 (94.5–96.8)		95.9 (93.7–97.5)		95.7 (94.1–97.0)
Sensitivity stratified by Ct value (NIID-N2 set) (%)	< 20	98.4 (121/123)		98.9 (86/87)		97.2 (35/36)
	20–24	92.9 (79/85)		95.8 (46/48)		89.2 (33/37)
	25–29	70.0 (21/30)		88.2 (15/17)		46.2 (6/13)
	≥ 30	18.9 (7/37)		25.0 (4/16)		14.3 (3/21)

* The results of RT-PCR were based on the results of the NIID, Japan method [8], while the discordant cases with in-house PCR [7] were determined by Xpert Xpress SARS-CoV-2 and GeneXpert System.

Sensitivity, specificity, positive predictive value, and negative predictive value are provided with 95% confidence intervals.

Sensitivity stratified by Ct value is provided with the number of the denominator and the numerator.

RT-PCR, reverse-transcription PCR; Ct, cycle threshold; NIID, National Institute of Infectious Diseases.

Table 2

SARS-CoV-2 diagnostic performance of QuickNavi-Flu+COVID19 Ag in anterior nasal samples.

		All (N = 862)		Symptomatic (N = 451)		Asymptomatic (N = 411)
Results of RT-PCR using nasopharyngeal samples*		Positive	Negative	Positive	Negative	Positive	Negative
QuickNavi-Flu+COVID19 Ag	Positive	158	0	98	0	60	0
	Negative	75	629	35	318	40	311
Sensitivity (%)		67.8 (61.4–73.8)		73.7 (65.3–80.9)		60.0 (49.7–69.7)
Specificity (%)		100 (99.1–100)		100 (98.3–100)		100 (98.2–100)
Positive predictive value (%)		100 (96.5–100)		100 (94.4–100)		100 (91.2–100)
Negative predictive value (%)		89.3 (86.8–91.5)		90.1 (86.5–93.0)		88.6 (84.8–91.7)
Sensitivity stratified by Ct value (NIID-N2 set) (%)	< 20	93.0 (106/114)		93.2 (69/74)		92.5 (37/40)
	20–24	63.1 (41/65)		55.3 (21/38)		74.1 (20/27)
	25–29	25.0 (6/24)		55.6 (5/9)		6.7 (1/15)
	≥ 30	14.3 (3/21)		20.0 (2/10)		9.1 (1/11)

* The results of RT-PCR were based on the results of the NIID, Japan method [8], while the discordant cases with in-house PCR [7] were determined by Xpert Xpress SARS-CoV-2 and GeneXpert System.

Sensitivity, specificity, positive predictive value, and negative predictive value are provided with 95% confidence intervals.

Sensitivity stratified by Ct value is provided with the number of the denominator and the numerator.

RT-PCR, reverse-transcription PCR; Ct, cycle threshold; NIID, National Institute of Infectious Diseases.

Overall, the results of QuickNavi-Flu+COVID19 Ag and QuickNavi-COVID19 Ag for SARS-CoV-2 detection were all consistent. For influenza viruses, one case in the nasopharyngeal group was positive for both influenza type A and B by QuickNavi-Flu2 but negative by QuickNavi-Flu+COVID19 Ag. This sample was confirmed to be negative by reference real-time RT-PCR for influenza viruses.

Table 1 shows the diagnostic performance of QuickNavi-Flu+COVID19 Ag for SARS-CoV-2 detection using nasopharyngeal samples. Totally, the sensitivity was 80.9% (95%CI: 75.8–85.3), specificity was 99.8% (95%CI: 99.3–99.9), PPV was 98.7% (95%CI: 96.3–99.7) and NPV was 95.8% (95%CI: 94.5–96.8). In symptomatic cases, the sensitivity was 88.3% (95%CI: 82.5–92.7) and specificity was 100% (95%CI: 98.8–100). In asymptomatic cases, the sensitivity was 69.4% (95%CI: 59.9–77.8) and specificity was 99.6% (95%CI: 98.9–99.9). For Ct values < 20, the sensitivity was greater than 95% regardless of symptoms. For Ct values 25–29, the sensitivity was decreased to 46.2% (95%CI: 19.2–74.9) in asymptomatic cases, and for Ct values ≥ 30, the sensitivity was even decreased to 25.0% (95%CI: 7.3–52.4) in symptomatic cases. In SARS-CoV-2 detection, there were three cases that were positive for both QuickNavi-Flu+COVID19 Ag and QuickNavi-COVID19 Ag, but negative for both the in-house PCR and the reference real-time RT-PCR. One of the three cases had an opportunity to recollect nasopharyngeal samples when the subject came to the hospital accompanied by the confirmed COVID-19 family members re-visiting for checking general condition, and both QuickNavi-Flu+COVID19 Ag and in-house PCR were positive. Subsequently, the reference real-time RT-PCR was negative, but Xpert Xpress SARS-CoV-2 was positive (Ct value of 36.2 for E target and 37.9 for N2 target).

Table 2 shows the diagnostic performance for SARS-CoV-2 detection using anterior nasal samples. In total, the sensitivity was 67.8% (95%CI: 61.4–73.8), specificity was 100% (95%CI: 99.1–100), PPV was 100% (95%CI: 96.5–100) and NPV was 89.3% (95%CI: 86.8–91.5). The sensitivity was 73.7% (95%CI: 65.3–80.9) in symptomatic cases and 60.0% (95%CI: 49.7–69.7) in asymptomatic cases. For Ct values < 20, the sensitivity was above 90% even in asymptomatic cases, but for Ct values 20–24, the sensitivity was decreased to 55.3% (95%CI: 38.3–71.4) in symptomatic cases, and for Ct values 25–29, the sensitivity was even decreased to 6.7% (95%CI: 0.2–31.9) in asymptomatic cases.

In this prospective evaluation, QuickNavi-Flu+COVID19 Ag showed sensitivity of 80.9% in nasopharyngeal samples and 67.8% in anterior nasal samples for the detection of SARS-CoV-2. Even though decreased sensitivity is known for anterior nasal samples when compared to nasopharyngeal samples [11], the sensitivity exceeds 90% with anterior nasal samples with high viral loads (Ct values < 20). In total of 2372 tests of QuickNavi-Flu+COVID-19 Ag, three cases were confirmed positive on the SARS-CoV-2 test line position but were negative for real-time RT-PCR. No positive cases of influenza viruses were detected by QuickNavi-Flu+COVID19 Ag and reference real-time RT-PCR.

QuickNavi-Flu+COVID19 Ag can similarly detect Alpha, Beta, Gamma, Delta, and Kappa variants of SARS-CoV-2 on experimental analysis [12]. The study period coincided with the time when the variant carrying L452R spike mutation accounted for approximately 80% in the community (Supplementary Fig. 1) [5]. Therefore, although SARS-CoV-2 variants testing were not performed in this study, our results along with the analysis indicated that the antigen test sufficiently detect the circulating SARS-CoV-2 variants.

There were large differences in sensitivities between symptomatic and asymptomatic cases in the 25–29 Ct value stratum (Table 1, Table 2). For both nasopharyngeal and anterior nasal sample groups, the median Ct values were lower for the symptomatic cases compared to the asymptomatic cases (nasopharyngeal samples: 26 [IQR: 25–27] vs 28 [27-29], p = 0.05; anterior nasal samples: 26 [IQR: 25–29] vs 28 [IQR: 27–29], p = 0.28). This could be attributed to the difference in sensitivity between the symptomatic and the asymptomatic cases in this Ct stratum.

Three cases with positive results of the antigen test but negative results of the real-time RT-PCR were observed in nasopharyngeal samples for SARS-CoV-2 detection. Interestingly, in one of the three cases, results of the re-test were positive for both the antigen test and the two of three PCR examinations. The cause of this discrepancy may be due to the insufficient amount or suboptimal quality of collected samples used for the first test. False positives of antigen test could occur due to cross-reactions, interfering substances, extreme temperature and humidity [13]. In addition, the manufacture has issued a recall on some lots of the QuickNavi-COVID19 Ag due to the concern for the increased false positive rate [14], but our study did not include the relevant batches. Depending on pre-test probabilities of patients or epidemic situations in neighboring community, positive results of antigen tests should be carefully judged, and necessity of retesting with PCR test should be considered.

In conclusion, QuickNavi-Flu+COVID19 Ag showed adequate sensitivity and sufficient specificity for SARS-CoV-2 detection using both nasopharyngeal and anterior nasal samples, especially in symptomatic patients.

Author statemnet

Contributor Yuto Takeuchi drafted the manuscript and performed the statistical analyses. Shigeyuki Notake, Atsuo Ueda and Koji Nakamura collected samples and operated the equipment. Daisuke Kato, Takashi Miyazawa, Shino Muramatsu and Yuki Shinohara interpreted results. Yusaku Akashi, Yoshihiko Kiyasu, Norihiko Terada and Yoko Kurihara revised the manuscript. Hiromichi Suzuki designed the study. All authors contributed to the writing of the final manuscript.

Declaration of competing interest

Denka Co., Ltd., provided fees for research expenses and QuickNavi-Flu+COVID19 Ag, QuickNavi-COVID19 Ag, and QuickNavi-Flu2 without charge. Hiromichi Suzuki received a lecture fee from , regarding this study. Daisuke Kato, Takashi Miyazawa, Shino Muramatsu and Yuki Shinohara belong to Denka Co., Ltd., the developer of the QuickNavi-Flu+COVID19 Ag.