A prospective clinical evaluation of the diagnostic accuracy of the SARS-CoV-2 rapid antigen test using anterior nasal samples.

INTRODUCTION: The diagnostic accuracy of antigen testing of anterior nasal (AN) samples for the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection has not been evaluated in the Japanese population. This study assessed the diagnostic accuracy of the Roche SARS-CoV-2 rapid antigen test (rapid antigen test) using AN samples.
METHODS: Two AN samples and one nasopharyngeal (NP) sample were collected from individuals undergoing screening for SARS-CoV-2 infection. The results of the rapid antigen test and the reverse-transcription polymerase chain reaction (RT-PCR) test using AN samples were compared to those of RT-PCR tests using NP samples.
RESULTS: Samples were collected from 800 participants, 95 and 110 of whom tested positive for SARS-CoV-2 on RT-PCR tests of AN and NP samples, respectively. The overall sensitivity/specificity of the AN rapid antigen test and AN RT-PCR were 72.7%/100% and 86.4%/100%, respectively. In symptomatic cases, the sensitivities of the AN rapid antigen test and AN RT-PCR were 84.7% and 94.9%, respectively. In asymptomatic cases, the sensitivities of the AN rapid antigen test and AN RT-PCR were 58.8% and 76.5%, respectively. The sensitivity of the AN rapid antigen test was over 80% in cases with cycle threshold (Ct) values < 25; it significantly decreased with an increase in the Ct values (p < 0.001).
CONCLUSION: The rapid antigen test with AN samples had a favorable sensitivity, especially in symptomatic cases or in cases with Ct values < 25. It gave no false-positive results. Compared with AN-RT PCR, the AN rapid antigen test had a modestly lower sensitivity in asymptomatic cases.

Introduction

The coronavirus disease 2019 pandemic, caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), remains a significant health concern despite the development of effective vaccines and treatments [[1], [2], [3]]. In addition to universal mask wear and vaccination, testing the broad population is a key infection control strategy [2,[4], [5], [6]].

Antigen testing has been used as an alternative to molecular testing due to an easy specimen-handling procedure, wide availability, and short turnaround times [7]. Antigen testing is especially useful in resource-limited settings, and is now applied in infection control at mass-gatherings and in the general population to reduce the risk of transmission [6,8,9].

Nasopharyngeal (NP) samples are primarily used for antigen testing in medical facilities; however, sample collection requires trained medical staff and causes discomfort to the patients [10]. Thus, anterior nasal (AN) samples may be preferable, especially for mass screening.

The SARS-CoV-2 rapid antigen test (rapid antigen test; Roche Diagnostics GmbH, Mannheim, Germany) is a lateral flow immunochromatography test that is commercially available worldwide. The rapid antigen test showed pooled sensitivities of 88.1% and 69.2% for NP samples obtained from symptomatic and asymptomatic individuals, respectively; it also showed a pooled specificity of 99.1% in both [11]. Its diagnostic performance fulfills the World Health Organization criteria and is one of the highest among those of several products tested [11,12]. Nevertheless, the clinical performance of the rapid antigen test with AN samples has not been well evaluated.

We conducted a prospective study to assess the clinical performance of the rapid antigen test in the Japanese population using AN samples. The results were compared with those of reverse-transcription polymerase chain reaction (RT-PCR) assays.

Methods

This study was performed between July 7 and July 29, 2021, at a PCR center in the Tsukuba Medical Center Hospital (TMCH), which is located in the southern part of the Ibaraki Prefecture, Japan. At the PCR center, NP samples were collected from all patients for clinical purposes, and patient data were recorded as previously reported [13,14]. Two additional AN samples were collected for the rapid antigen test and the RT-PCR test.

The study included individuals who were referred from a local public health center and 51 primary care facilities or were healthcare workers at the TMCH. The participants were those suspected of having contracted a SARS-CoV-2 infection due to their symptoms or a history of close contact. Patients who declined to participate in the study, whose residual samples for RT-PCR were unavailable, and for whom duplicate samples were collected during the same episode were excluded.

Participants were considered “symptomatic” if at least one of the following symptoms existed: fever, cough, nasal discharge and/or congestion, sore throat, loss of taste and/or smell, dyspnea, fatigue, headache, diarrhea, and vomiting.

We obtained verbal informed consent from all participants, and the requirement of a written informed consent was waived due to infection control measures. The study was approved by the ethics committee of the University of Tsukuba (approval number: R03-041).

Sample collection and procedures

Trained medical staff collected both AN and NP samples from all participants. We first obtained an AN sample for the rapid antigen test from both nostrils using the swabs included in the test kits, in accordance with the manufacturer's instructions. Another AN sample was then collected in the same manner for RT-PCR testing, using a FLOQSwab (Copan ItaliaSpA, Brescia, Italy). An NP sample was also collected for RT-PCR testing according to the recommended procedure [15].

Immediately after sample collection, antigen tests were performed in accordance with the manufacturer's instructions, and the results were adjudicated by each examiner. Swab samples collected for RT-PCR testing were suspended in 3 mL of Universal Transport Medium (UTM; Copan Italia S.p.A., Brescia, Italy). NP samples suspended in UTM were used for in-house RT-PCR testing at the microbiology laboratory of the TMCH. AN samples suspended in UTM and the residual NP samples suspended in UTM were cryopreserved at −80 °C and transferred to the Roche Diagnostics technical support laboratory for the RT-PCR on a weekly basis.

Testing for SARS-CoV-2 using RT-PCR

At the Roche Diagnostics technical support laboratory, RNA was extracted and purified from 140 μL aliquots of UTM samples using the MagNA Pure 96 total NA Isolation Kit and the MagNA Pure 96 Instrument (Roche Molecular Systems, NJ), respectively. The real-time RT-PCR was performed using a national standard method developed by the National Institute of Infectious Diseases (NIID), Japan, which targeted the N2 region [16]. The RT-PCR was performed in duplicates using a PCR LightCycler 480 II System (Roche Diagnostics International Ltd., Rotkreuz, Switzerland), the QuantiTect Probe RT-PCR Kit (QIAGEN, Hilden, Germany), and a SARS-CoV-2 positive control (Nihon Gene Research Laboratories, Sendai, Japan). The Ct values of RT-PCR described the average of duplicate.

Statistical analyses

The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated from the results of the NP RT-PCR test. The 95% confidence intervals (CIs) were determined using the Clopper and Pearson method. The degree of agreement between two tests was evaluated by calculating the Cohen's kappa coefficient.

Regarding the clinical data of the participants, the Wilcoxon rank sum test and the Fisher's exact test were used for comparing the continuous and categorical variables, respectively.

The Cochran–Armitage test was used to analyze the trend of sensitivity according to the Ct values of the NP RT-PCR.

Two-sided P-values <0.05 were considered statistically significant. All analyses were conducted using R, version 3.3.1 (The R Foundation for Statistical Computing, Vienna, Austria).

Results

Samples were collected from 800 participants, 333 (41.6%) of whom were symptomatic. Table 1 summarizes the prevalence of each symptom and the days from the symptom onset.

Table 1

Clinical data of the participants.

		SARS-CoV-2 (Nasopharyngeal RT-PCR)
	All	Positive	Negative	p values
N	800	110	690
Age (median [IQR])	36.0 [25.0–50.0]	33.0 [23.0–44.5]	37.0 [25.0–50.8]	0.03
Sex (Female)	341 (42.6)	47 (42.7)	294 (42.6)	1.00
Presence of symptoms	333 (41.6)	59 (53.6)	274 (39.7)	0.01
Days from the onset [IQR]	2.0 [1.0–3.0]	2.0 [1.0–2.3]	2.0 [1.0–3.0]	0.79
Fever	285 (35.6)	51 (46.4)	234 (33.9)	0.01
Cough	118 (14.8)	26 (23.6)	92 (13.3)	0.01
Nasal discharge and/or congestion	57 (7.1)	8 (7.3)	49 (7.1)	1.00
Sore throat	54 (6.8)	8 (7.3)	46 (6.7)	0.84
Loss of taste and/or smell	10 (1.3)	5 (4.5)	5 (0.7)	0.01
Dyspnea	1 (0.1)	0 (0.0)	1 (0.1)	1.00
Fatigue	52 (6.5)	8 (7.3)	44 (6.4)	0.68
Headache	40 (5.0)	5 (4.5)	35 (5.1)	1.00
Diarrhea	11 (1.4)	3 (2.7)	8 (1.2)	0.18
Vomiting	3 (0.4)	2 (1.8)	1 (0.1)	0.05

IQR, interquartile range; RT-PCR, reverse-transcription polymerase chain reaction. Categorical variables are provided with percentages in parentheses. RT-PCR tests were performed using the method developed by the National Institute of Infectious Diseases, Japan.

The SARS-CoV-2 RT-PCR result was positive in 95 and 110 of the AN and NP samples, respectively. There were 15 discordant pairs of samples on RT-PCR testing. All discordant samples were AN-negative/NP-positive, and 11 of them had Ct values > 30 on NP RT-PCR.

Overall diagnostic accuracy of the AN rapid antigen test and AN RT-PCR

The sensitivity, specificity, PPV, and NPV of the AN rapid antigen test were 72.7%, 100%, 100%, and 95.8%, respectively (Table 2 a). The Cohen's kappa coefficient between the AN rapid antigen test and NP RT-PCR was 0.82.

Table 2a

Comparison of the results of the SARS-CoV-2 Rapid Antigen Test (rapid antigen test) using anterior nasal samples and the real-time reverse-transcription polymerase chain reaction using nasopharyngeal samples among all participants.

		NP real-time RT-PCR (NIID)
		Positive	Negative
Rapid antigen test	Positive	80	0
	Negative	30	690
Sensitivity (%)		72.7 (63.4–80.8)
Specificity (%)		100 (99.5–100)
Positive predictive value (%)		100 (95.5–100)
Negative predictive value (%)		95.8 (94.1–97.1)

The Cohen's Kappa coefficient between the two tests was 0.82.

The sensitivity, specificity, positive predictive value, and negative predictive value are presented with their 95% confidence intervals.

CI, confidence intervals; NIID, National Institute of Infectious Diseases, Japan; NP, nasopharyngeal; RT-PCR, reverse-transcription polymerase chain reaction.

The sensitivity and specificity of AN RT-PCR were 86.4% (95% CI: 78.5%–92.2%) and 100% (95% CI: 99.5%–100%), respectively (Table 3 a).

Table 3a

Comparison of the results of the real-time reverse-transcription polymerase chain reaction between AN and NP samples among all participants.

		NP RT-PCR (N2)
		Positive	Negative
AN RT-PCR (N2)	Positive	95	0
	Negative	15	690
Sensitivity (%)		86.4 (78.5–92.2)
Specificity (%)		100 (99.5–100)
Positive predictive value (%)		100 (96.2–100)
Negative predictive value (%)		97.9 (96.5–98.8)

The sensitivity, specificity, positive predictive value, and negative predictive value of AN RT-PCR are presented with their 95% confidence intervals.

AN, anterior nasal; NP, nasopharyngeal; NIID, National Institute of Infectious Diseases, Japan; RT-PCR, reverse-transcription polymerase chain reaction.

Diagnostic accuracies of the AN rapid antigen test and AN RT-PCR in symptomatic and asymptomatic individuals

Among symptomatic cases, 56 AN and 59 NP samples were positive for SARS-CoV-2 on RT-PCR testing. Using NP RT-PCR test results as the reference, the sensitivities of the AN rapid antigen test and AN RT-PCR in symptomatic cases were found to be 84.7% and 94.9%, respectively (Table 2b, Table 3bb).

Table 2b

Comparison of the results of the SARS-CoV-2 Rapid Antigen Test (rapid antigen test) using anterior nasal samples and the real-time reverse-transcription polymerase chain reaction using nasopharyngeal samples among symptomatic participants.

		NP real-time RT-PCR (NIID)
		Positive	Negative
Rapid antigen test	Positive	50	0
	Negative	9	274
Sensitivity (%)		84.7 (73.0–92.8)
Specificity (%)		100 (98.7–100)
Positive predictive value (%)		100 (92.9–100)
Negative predictive value (%)		96.8 (94.0–98.5)

The Cohen's Kappa coefficient between the two tests was 0.90.

The sensitivity, specificity, positive predictive value, and negative predictive value are presented with their 95% confidence intervals.

CI, confidence interval; NIID, National Institute of Infectious Diseases, Japan; NP, nasopharyngeal; RT-PCR, reverse-transcription polymerase chain reaction.

Table 3b

Comparison of the results of the real-time reverse-transcription polymerase chain reaction between AN and NP samples among symptomatic participants.

		NP RT-PCR (N2)
		Positive	Negative
AN RT-PCR (N2)	Positive	56	0
	Negative	3	274
Sensitivity (%)		94.9 (85.9–98.9)
Specificity (%)		100 (98.7–100)
Positive predictive value (%)		100 (93.6–100)
Negative predictive value (%)		98.9 (96.9–99.8)

The sensitivity, specificity, positive predictive value, and negative predictive value of AN RT-PCR are presented with their 95% confidence intervals.

AN, anterior nasal; NP, nasopharyngeal; NIID, National Institute of Infectious Diseases, Japan; RT-PCR, reverse-transcription polymerase chain reaction.

Among asymptomatic cases, 39 AN and 51 NP samples were positive for SARS-CoV-2 on RT-PCR testing. The sensitivities of the AN rapid antigen test and AN RT-PCR were 58.8% and 76.5%, respectively (Table 2c, Table 3cc).

Table 2c

Comparison of the results of the SARS-CoV-2 Rapid Antigen Test (rapid antigen test) using anterior nasal samples and the real-time reverse-transcription polymerase chain reaction using nasopharyngeal samples among asymptomatic participants.

		NP real-time RT-PCR (NIID)
		Positive	Negative
Rapid antigen test	Positive	30	0
	Negative	21	416
Sensitivity (%)		58.8 (44.2–72.4)
Specificity (%)		100 (99.1–100)
Positive predictive value (%)		100 (88.4–100)
Negative predictive value (%)		95.2 (92.7–97.0)

The Cohen's Kappa coefficient between the two tests was 0.72.

The sensitivity, specificity, positive predictive value, and negative predictive value are presented with their 95% confidence intervals (CIs).

NIID, National Institute of Infectious Diseases, Japan; NP, nasopharyngeal; RT-PCR, reverse-transcription polymerase chain reaction.

Table 3c

Comparison of the results of the real-time reverse-transcription polymerase chain reaction between AN and NP samples among asymptomatic participants.

		NP RT-PCR (N2)
		Positive	Negative
AN RT-PCR (N2)	Positive	39	0
	Negative	12	416
Sensitivity (%)		76.5 (62.5–87.2)
Specificity (%)		100 (99.1–100)
Positive predictive value (%)		100 (90.1–100)
Negative predictive value (%)		97.2 (95.2–98.5)

The sensitivity, specificity, positive predictive value, and negative predictive value of AN RT-PCR are provided with their 95% confidence intervals.

AN, anterior nasal; NP, nasopharyngeal; NIID, National Institute of Infectious Diseases, Japan; RT-PCR, reverse-transcription polymerase chain reaction.

Impact of Ct values on the sensitivity of the AN rapid antigen test

Table 4 shows the sensitivities of the AN rapid antigen test according to the Ct values of NP RT-PCR. The AN rapid antigen test had sensitivities of 93.9%, 83.3%, 57.1%, and 5.9% for samples with Ct values of <20, 20–25, 25–30, and >30, respectively. The sensitivity significantly decreased with an increase in the Ct values (p < 0.001).

Table 4

Sensitivity of the SARS-CoV-2 Rapid Antigen Test (rapid antigen test) according to the Ct value.

		Sensitivity (%)
Ct value	N	Rapid antigen test	AN RT-PCR
<20	49	93.9 (83.1–98.7)	95.9 (86.0–99.5)
20–25	30	83.3 (65.3–94.5)	96.7 (82.8–99.9)
25–30	14	57.1 (28.9–82.3)	92.9 (66.1–99.8)
>30	17	5.9 (1.5–28.7)	35.3 (14.2–61.7)
Overall	110	72.7 (63.4–80.8)	86.4 (78.5–92.1)

Sensitivities are provided with their 95% confidence intervals.

The Ct values were determined for NP samples using the RT-PCR developed by the National Institute of Infectious Diseases, Japan.

AN, anterior nasal; Ct, cycle threshold; NP, nasopharyngeal; RT-PCR, reverse-transcription polymerase chain reaction.

Similarly, with NP RT-PCR test results as the reference, the sensitivity of AN RT-PCR was noted to decline with an increase in the Ct value (p < 0.001). The sensitivities of AN RT-PCR were 95.9%, 96.7%, 92.9%, and 35.3% for samples with Ct values of <20, 20–25, 25–30, and >30 respectively.

In cases with Ct values of over 35.9 for AN RT-PCR and 31.7 for NP RT-PCR, no samples tested positive in the AN rapid antigen test.

Ct values in cases with false-negative AN rapid antigen test results

The Ct values of cases with discrepant results between the AN rapid antigen test and the RT-PCR test using either AN or NP samples are shown in Table 5 . The median Ct values of the discordant cases were 35.9 (interquartile range [IQR]: 32.4–37.1) and 31.4 (IQR: 24.3–33.8) for AN RT-PCR and NP RT-PCR, respectively.

Table 5

Detailed results of individual participants with a negative SARS-CoV-2 Rapid Antigen Test result and a positive real-time reverse-transcription polymerase chain reaction test result.

Case IDa	Symptoms	Ct value
		NP RT-PCR	AN RT-PCR
8	–	28.3	36.6
57	+	25.7	34.9
59	+	20.0	31.9
119	+	27.7	37.0
164	–	25.7	33.7
247	+	22.0	32.9
298	–	31.3	39.1
378	+	36.2	35.9
422	–	23.8	37.3
548	–	26.6	36.4
549	–	35.7	38.6
569	–	23.4	26.1
576	–	37.1	37.2
727	+	32.1	31.6
785	–	18.4	23.1
43	+	33.2	ND
48	+	28.8	ND
154	–	31.4	ND
228	+	35.6	ND
264	–	33.8	ND
415	–	18.7	ND
428	–	32.8	ND
520	–	32.4	ND
556	–	34.1	ND
568	–	22.4	ND
570	–	33.7	ND
621	–	33.9	ND
735	–	18.7	ND
741	–	31.7	ND
843	–	34.0	ND

AN, anterior nasal; Ct, cycle threshold; ND, not detected; NP nasopharyngeal; RT-PCR, reverse-transcription polymerase chain reaction.

Data on the entire study population are available in a Supplementary File.

Discussion

This study demonstrated that using AN samples, the AN rapid antigen test had a favorable diagnostic accuracy for SARS-CoV-2. No false-positive results were obtained in this study. The sensitivity of the test varied with the presence or absence of symptoms and with the Ct values. In symptomatic patients, the sensitivity was over 84.7%; however, in asymptomatic individuals, the sensitivity decreased to 58.8%.

The overall sensitivity of the AN rapid antigen test was 72.7%; its specificity was 100% with NP RT-PCR test results as the reference. Although the sensitivity was suboptimal, its concordance rate with AN RT-PCR was 98.1% (Supplementary Table 1). Besides, its diagnostic performance was comparable to that reported previously [17]. The viral load is generally higher in the nasopharynx than in the nostrils [18], which may lower the sensitivity of testing with AN samples. The AN RT-PCR did not detect SARS-CoV-2 in 13.6% of the participants with positive NP RT-PCR samples. Most discordant cases between AN and NP RT-PCR were in asymptomatic individuals and the Ct values were >30 on NP RT-PCR, indicating a low viral load.

In symptomatic cases, the AN rapid antigen test demonstrated a sensitivity of 84.7%. The sensitivity seemed favorable even when compared to that of AN RT-PCR. The higher sensitivity may be due to the higher viral load in symptomatic patients and the shorter duration between symptom onset and examinations. Compared with the asymptomatic individuals, the symptomatic patients in our study had a significantly lower Ct values (median; 19.7 vs. 22.7, p = 0.02). The median duration from symptom onset was 2 days, during which viral shedding generally remained high [19].

Mass screening of asymptomatic individuals with antigen testing has been initiated in some countries [9,20]. A previous study estimated that such screening suppressed 70% of the SARS-CoV-2 transmission in models [9]. AN samples have primarily been used for this purpose, because they can be self-collected; furthermore, AN sample collection is easy and less invasive than NP sample collection [10,17]. Our study indicated that although the specificity was high at 100%, the rapid antigen test using AN samples missed a clinically significant proportion of cases of asymptomatic SARS-CoV-2 infection. Nevertheless, antigen testing seems to effectively identify the majority of transmissible cases wherein viral loads are generally high [21]. The rapid antigen test had a sensitivity of approximately 80% in cases with Ct values < 25 on NP RT-PCR testing, although the sensitivity decreased to under 60% in cases with Ct values of 25–30 (Table 4). A similar trend was observed when we limited the study population to asymptomatic individuals (Supplementary Table 2b). [19,22]. Furthermore, modeling studies suggest that the specificity and frequency of testing are key for a successful mass screening [23,24]. Thus, using the rapid antigen test with AN samples could play a useful role in mass screening.

The study has some limitations. First, the interpretation of the results may have varied between the examiners due to the visual nature of the judgment of antigen testing results [25]. Second, Ct values vary according to the reagents and molecular identification system used, and the choice of positive control, even if same primer targets are used, so the sensitivity of each Ct value range may differ in other settings.

In conclusion, this prospective observational study found that the rapid antigen test with AN samples had a favorable sensitivity in symptomatic patients. However, the sensitivity in asymptomatic individuals was not ideal. Nevertheless, the rapid antigen test with AN samples may be a useful screening tool because of its low invasiveness, high specificity, and the ability to identify individuals with high viral loads.

Funding

This work was supported by .

Declaration of competing interest

Roche Diagnostics Co., Ltd, provided support in the form of salaries to authors M. Horie, Y. Adachi, and K. Togashi and as lecture fees to author H. Suzuki.