Evaluation of false positives in the SARS-CoV-2 quantitative antigen test.

INTRODUCTION: Highly sensitive reagents for detecting SARS-CoV-2 antigens have been developed for accurate and rapid diagnosis till date. In this study, we aim to clarify the frequency of false-positive reactions and reveal their details in SARS-CoV-2 quantitative antigen test using an automated laboratory device.
METHODS: Nasopharyngeal swab samples (n = 4992) and saliva samples (n = 5430) were collected. We measured their SARS-CoV-2 antigen using Lumipulse® Presto SARS-CoV-2 Ag and performed a nucleic acid amplification test (NAAT) using the Ampdirect™ 2019 Novel Coronavirus Detection Kit as needed. The results obtained from each detection test were compared accordingly.
RESULTS: There were 304 nasopharyngeal samples and 114 saliva samples were positive in the Lumipulse® Presto SARS-CoV-2 Ag test. All positive nasopharyngeal samples in the antigen test were also positive for NAAT. In contrast, only three (2.6%) of all the positive saliva samples in the antigen test were negative for NAAT. One showed no linearity with a dilute solution in the dilution test. Additionally, the quantitative antigen levels of all the three samples did not decrease after reaction with the anti-SARS-CoV-2 antibody.
CONCLUSIONS: The judgment difference between the quantitative antigen test and NAAT seemed to be caused by non-specific reactions in the antigen test. Although the high positive and negative predictive value of this quantitative antigen test could be confirmed, we should consider the possibility of false-positives caused by non-specific reactions and understand the characteristics of antigen testing. We recommend that repeating centrifugation before measurement, especially in saliva samples, should be performed appropriately.

Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the virus that causes the atypical pneumonia known as coronavirus disease (COVID-19), emerged in late 2019 in Wuhan, China [1]. The World Health Organization has declared COVID-19 a public health emergency of international concern and a very high-risk assessment at the global level [2,3].

The nucleic acid amplification test (NAAT) is the gold standard and is mostly used to diagnose COVID-19. However, this method requires several hours and a special technique to detect the nucleic acids. Moreover, specialized instruments and expertise are required to perform these tests [4,5]. Other approaches have been developed to diagnose COVID-19 by targeting SARS-CoV-2 antigen. Recently, a new quantifying reagent that can detect SARS-CoV-2 antigens has been developed in order to perform a rapid and accurate detection. The performance of a quantitative antigen test was excellent enough for real-life clinical practice in terms of sensitivity and specificity [6]. In addition, recent studies have revealed the utility of the SARS-CoV-2 quantitative antigen test [7,8]. However, the method for detection was based on chemiluminescent enzyme immunoassay (CLEIA), which may potentially give rise to the rare phenomenon of false-positives [9]. In this study, we aimed to clarify the frequency of false reactions under our unique premeasured management for detecting SARS-CoV-2 antigen.

Materials and methods

Sample collection

We collected 4992 nasopharyngeal samples and 5430 saliva samples from patients suspected of having SARS-CoV-2 infection, including 3483 nasopharyngeal samples and all saliva samples requested by the public health center of Sapporo City from November 2020 to February 2021. Nasopharyngeal swabs from patients with COVID-19 at Sapporo Medical University Hospital were collected during hospitalization using a kit containing a nylon-flocked nasopharyngeal swab and a tube containing universal transport medium (UTM; Copan Diagnostics, Murrieta, CA, United States). Nasopharyngeal swabs from patients at the public health center of Sapporo City were collected using UTM or sodium chloride solution. All saliva samples were collected in sterile tubes. All samples were tested for SARS-CoV-2 antigen as soon as they arrived at our laboratory and were preserved at −80 °C after testing.

Test for SARS-CoV-2 quantitative antigen including premeasured management

We measured the SARS-CoV-2 quantitative antigen using Lumipulse® Presto SARS-CoV-2 Ag (Fujirebio Inc., Tokyo, Japan). The Lumipulse® Presto SARS-CoV-2 Ag was analyzed using a fully automated Lumipulse® L2400 (Fujirebio Inc., Tokyo, Japan). In nasopharyngeal samples, preserved solutions were tested after centrifugation at 2000×g for 5 min. On the other hand, saliva samples were diluted 2-fold with phosphate buffered saline (PBS) and centrifuged at 20,000×g for 10 min to remove cells and debris. Immediately after this procedure, the supernatant was separated into other tubes and centrifuged at 2000×g for 5 min and tested accordingly. All nasopharyngeal swabs were judged as positive or judgment pending or negative by the antigen level (negative: less than 1.34 pg/mL, judgment pending: from 1.34 to less than 10.00 pg/mL, and positive: over 10.00 pg/mL) according to the manufacturer's protocol. Similarly, because saliva samples were diluted 2-fold with PBS, all saliva samples were judged by the antigen level in consideration of dilution factor (negative: less than 0.67 pg/mL, judgment pending: from 0.67 to less than 4.00 pg/mL, and positive: over 4.00 pg/mL). When the result of first test was within the judgment pending level, the sample was tested after re-centrifuged at 2000×g for 5 min regardless of sample species. We judged the sample by the result of retest. On the other hand, we evaluated our unique judgment pending level by lowering the lower limit of judgment pending level to 1.00 pg/mL in nasopharyngeal swab samples and 0.50 pg/mL in saliva samples. In the results, samples with a positive SARS-CoV-2 antigen test and negative NAAT, were defined as false-positives accordingly.

Test for SARS-CoV-2 nucleic acid

The SARS-CoV-2 nucleic acid test was performed on a LightCycler 480 System (Roche, Basel, Switzerland) using the Ampdirect™ 2019 Novel Coronavirus Detection Kit (Shimadzu Corporation, Kyoto, Japan). This test could be performed without extraction of ribonucleic acid. Samples were mixed with an equal amount of the sample treatment reagent and heated at 90 °C for 5 min, and then, 10 μL of them were used. All assays were performed according to the manufacturer's protocol, and the samples were judged as positive or negative based on the amplification curve; when the amplification curve was recognized to rise during the assay, it was judged as positive.

Dilution linearity tests

Samples were diluted with a dedicated dilution solution (Fujirebio Inc., Tokyo, Japan) by a repeated 2-fold dilution.

Absorption tests

Samples were incubated with one-tenth the volume of anti-SARS-CoV-2 monoclonal antibody or protein solution as a negative control for 10 min at room temperature and tested accordingly. The comparison solutions included equal concentrations of protein to the anti-SARS-CoV-2 monoclonal antibody but without the antibody. The results of the absorption test were judged as positive or negative based on the following formula: (1 – (the level of treatment with anti-SARS-CoV2 antibody)/(the level of treatment with comparison solution)) × 100. When it was over 50, it was judged as positive according to the criteria for suppression tests of other Lumipulse® system reagents.

Statistical analysis

Statistical analyses, such as p-value calculation by Mann-Whitney U test, were performed using SAS Platform JMP Pro version 15.1.0. Software (SAS Institute Inc., Cary, NC, USA).

Ethical statement

This study was approved by the Institutional Review Board of Sapporo Medical University Hospital (reference number 322–263).

Results

Of the total 10,422 samples, 418 samples were positive for Lumipulse® Presto SARS-CoV-2 Ag. Positive samples consisted of 304 nasopharyngeal samples (Table 1 a) and 114 saliva samples (Table 1b). First, all positive samples in the antigen test were confirmed using NAAT. All positive nasopharyngeal samples and 111 positive saliva samples confirmed by the antigen test were also positive using the Ampdirect™ 2019 Novel Coronavirus Detection Kit. The positive predictive value of the nasopharyngeal samples and saliva samples were 100.0% (304/304) and 97.4% (111/114), respectively. On the other hand, the negative predictive value of them were 99.9% (4615/4616) and 99.9% (5269/5272), respectively. Three discrepant findings of samples with positive antigen test results and negative NAAT results were obtained from the saliva samples (2.6%; 3/114). The antigen level from the three samples were 12.55, 11.20, and 5.98 pg/mL (cut-off value: 4.0 pg/mL). Next, to confirm non-specific reactions, by repeating 2-fold dilution using dedicated dilute solution, the measured level from two samples decreased linearly; however, that of discrepant sample No.3, shown as a closed square, indicated no linearity (Fig. 1 ). Additionally, the levels of all three samples did not decrease after reaction with anti-SARS-CoV-2 antibody, although positive samples decreased after reaction with the same (Table 2 ). Thus, these three samples were judged as false-positive by quantitative antigen testing according to our definition.

Table 1

The results of antigen test in nasopharyngeal and saliva samples.

an = 4992
Judgment	SARS-CoV-2 antigen level (pg/mL)	Number of samples
Positive	≥10.00	304
Pending	1.34–10.00	72
Negative	<1.34	4616

Fig. 1

Dilution linearity test in 3 discrepant samples.

Closed circle (●): discrepant sample No.1 between the results of antigen test and NAAT, open circle (〇): discrepant sample No.2 between the results of antigen test and NAAT, closed square (■): discrepant sample No.3 between the results of antigen test and NAAT.

Table 2

The results of absorption test in 3 discrepant samples.

Case No.	Sample species	SARS-CoV-2 antigen level		(1-(a)/(b)) × 100 (+≥ 50%)
		(a)	(b)
		Anti-SARS-CoV-2 antibody (pg/mL)	Comparison solution (pg/mL)
1	saliva	7.36	9.58	23.2 (−)
2	saliva	6.33	6.9	8.3 (−)
3	saliva	2.58	2.65	2.6 (−)
Positive control	saliva	23.96	167.18	85.7 (+)

Next, we evaluated the accuracy of the judgment pending level determined by the manufacturer in Lumipulse® Presto SARS-CoV-2 Ag by performing NAAT to compare the samples of judgment pending level and the antigen level between the NAAT-positive and negative groups. In all, there were 124 nasopharyngeal samples and 52 saliva samples within the judgment pending level in first test. Of these, 72 nasopharyngeal samples and 44 saliva samples were finally judged as judgment pending after re-centrifugation. Among them, 27 of 72 nasopharyngeal samples and 30 of 44 saliva samples were positive for NAAT. In nasopharyngeal samples, the median antigen levels in NAAT-positive group and NAAT-negative group were 3.16 pg/mL and 2.02 pg/mL. In the saliva samples, they were 1.80 pg/mL and 0.97 pg/mL. There was a significant difference in the antigen levels between the NAAT-positive and NAAT-negative groups in nasopharyngeal swabs (p = 0.0005) and saliva (p = 0.002) (Fig. 2 ).

Fig. 2

Measurement distribution in the judgment pending group by SARS-CoV-2 quantitative antigen test.

The number of nasopharyngeal swab samples or saliva samples in NAAT-negative and NAAT-positive were 45 and 27, 14 and 30, respectively. The median levels of antigen in NAAT-negative and NAAT-positive were 2.02 pg/mL and 3.16 pg/mL, respectively (a). The median levels of antigen in NAAT-negative and NAAT-positive were 0.97 pg/mL and 1.80 pg/mL, respectively (b). All p-values were calculated using the Mann-Whitney U test. Closed circle (●): positive group tested by NAAT, open circle (〇): negative group tested by NAAT.

Fifty of the 4616 nasopharyngeal negative samples were measured in the range 1.00–1.34 pg/mL, and 20 of the 5272 saliva negative samples in the range 0.50–0.67 pg/mL. One nasopharyngeal sample and three saliva samples were positive for NAAT (Table 3 ). This nasopharyngeal sample was collected 14 days after symptom onset, and the details of the saliva samples were unclear.

Table 3

Discrepant samples of only positive NAAT that newly classified as judgment pending by our unique lower limit.

Case No.	Sample species	SARS-CoV-2 antigen level (pg/mL)	Ampdirect™ 2019 Novel Coronavirus Detection Kit
			N1 Ct value	N2 Ct value
4	Nasopharyngeal swab	1.19	37.70	42.90
5	saliva	0.59	35.08	33.64
6	saliva	0.53	34.50	33.35
7	saliva	0.50	34.76	36.18

Discussion

We evaluated the accuracy of the automated quantitative antigen test, Lumipulse® Presto SARS-CoV-2 Ag. While this reagent is useful for clinical laboratory tests because of its high positive and negative predictive value, false-positive results were potentially caused by non-specific reactions because this reagent is based on the CLEIA method that detects the reactivity of the antigen-antibody. Nevertheless, it was found that the false-positive rate of this test was much lower than that of the qualitative rapid antigen test that had been used worldwide [10]. We observed three false-positive samples due to non-specific reactions among the 114 positive saliva samples (2.6%). In the cases of false-positive samples, dilution linearity and absorption tests are generally useful for confirmation of accurate judgment. It has been reported that non-specific reactions tend to be diminished by dilution [11]. In this study, one of the three samples did not show dilution linearity by dedicated dilution solution, and all the three samples were judged as negative by the absorption test. Quantitative SARS-CoV-2 antigen measurement is an excellent clinical test because the frequency of false-positives is not high compared with other tests based on the CLEIA method [12]. Because 67 of 183 samples within judgment pending level were judged as negative after re-centrifugation, repeating centrifugation would be possible to make the reduction of the false-positive reaction caused by impurities of samples and lead to decrease in performing of unnecessary NAAT [13,14]. In real-life clinical practice, it would be a potentially useful test in a facility that cannot perform NAAT. However, all false-positive reactions were observed in the saliva samples. It was considered that saliva would be more likely to cause a false-positive reaction because saliva contained more impurities than the nasopharyngeal swabs. Therefore, care should be taken when measuring the saliva samples.

Lumipulse® Presto SARS-CoV-2 Ag reagent has the judgment pending level in the range of 1.34–10.00 pg/mL for nasopharyngeal swabs and 0.67–4.00 pg/mL for saliva samples. If the measured level of the sample is within these ranges, re-measurement after centrifugation is recommended. If the measured level after repeat centrifugation is not nearly changed, a confirmation test by NAAT is recommended by the manufacturer. There was a significant difference in antigen levels between the NAAT-positive and NAAT-negative groups in the judgment pending level according to the manufacturer's protocol; however, it seemed to be difficult to judge positive or negative based only on the antigen level. Therefore, we also evaluated the set value of this judgment pending level. The positive rate by NAAT in the judgment pending group was 37.5% (27/72) in nasopharyngeal swabs and 75.0% (30/44) in saliva samples respectively. We further investigated by lowering the lower limit of judgment pending level to 1.00 pg/mL in nasopharyngeal swab samples and 0.50 pg/mL in saliva samples. Fifty nasopharyngeal swab samples and 20 saliva samples were newly classified into our unique judgment pending level, and one nasopharyngeal sample and three saliva samples were positive by NAAT. One nasopharyngeal swab sample was collected 14 days after symptom onset. We reported that the results of NAAT did not match with the results of the antigen test 13 days after symptom onset [6]. We considered that the antigen level would decrease earlier than the RNA load in the nasopharynx. On the other hand, the Ct values by NAAT of the three saliva samples were more than 33.35, in N2, and they might not possibly maintain infectivity [15]. If we obtained positive results in the NAAT and negative results in the antigen test, it was considered that most samples obtained from the patients have passed approximately 2 weeks after onset or have a low viral load [6]. On the other hand, from the viewpoint of diagnostic tests for infectious diseases, it is important to detect positives at an early stage of infection [16]. For the purpose of confirming the spread of infectious disease including the early stage of infection and asymptomatic patients, which would have a low viral load, it might be better to consider the lower limit of judgment pending level in nasopharyngeal swab samples and saliva samples.

This study had some limitations. All samples judged as negative with the lower limit of our unique definition were not tested by NAAT. In saliva samples, we could not obtain the details of suspected patient information because the samples were gathered out of the hospital. Unfortunately, we could not determine the cause of the non-specific reactions. In general, various substances in the body may potentially cause such non-specific reactions.

In conclusion, the quantitative SARS-CoV-2 antigen test reagent, Lumipulse® Presto SARS-CoV-2 Ag reagent, has a highly positive and negative predictive value. We suggest the lower limit of judgment pending level may be reconsidered to detect positives at the early stage of infection. However, we should note that false-positive reactions rarely occur, particularly in saliva samples. Repeating centrifugation is the useful method for reduction of false-positive reactions.

Authorship statement

All authors meet the ICMJE authorship criteria. Contributors R.K., K.A., Y.F., and S.T. were responsible for the organization and coordination of the trial. S.T. was the chief investigator responsible for the data analysis. R.K., R.M., M.M., S.N., H.Y., T.K., M.S., Y.Y., Y.S., Y.K., H.N., and I.K. developed the trial design and conducted an investigation. All authors contributed to the writing of the final manuscript.

Funding source

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declaration of competing interest

The authors declare the following conflict of interests which may be considered as potential competing interests: Satoshi Takahashi received speaker honoraria from MSD K.K. and research grants from Shino-Test Corporation, Roche Diagnostic K.K, Fujirebio Inc, and Abbott Japan Co, Ltd. All other authors declare no conflict of interests.