Accuracy of anterior nasal swab rapid antigen tests compared with RT-PCR for massive SARS-CoV-2 screening in low prevalence population.

The aim was to determine the accuracy of anterior nasal swab in rapid antigen (Ag) tests in a low SARS-CoV-2 prevalence and massive screened community. Individuals, aged 18 years or older, who self-booked an appointment for real-time reverse transcriptase-polymerase chain reaction (RT-PCR) test in March 2021 at a public test center in Copenhagen, Denmark were included. An oropharyngeal swab was collected for RT-PCR testing, followed by a swab from the anterior parts of the nose examined by Ag test (SD Biosensor). Accuracy of the Ag test was calculated with RT-PCR as reference. We included 7074 paired conclusive tests (n = 3461, female: 50.7%). The median age was 48 years (IQR: 36-57 years). The prevalence was 0.9%, that is, 66 tests were positive on RT-PCR. Thirty-two had a paired positive Ag test. The sensitivity was 48.5% and the specificity was 100%. This study conducted in a low prevalence setting in a massive screening set-up showed that the Ag test had a sensitivity of 48.5% and a specificity of 100%, that is, no false positive tests. The lower sensitivity is a challenge especially if Ag testing is not repeated frequently allowing this scalable test to be a robust supplement to RT-PCR testing in an ambitious public SARS-CoV-2 screening.

Severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) and the resulting COVID‐19 pandemic is an ongoing worldwide health emergency [1, 2, 3]. The golden standard for correct detection of SARS‐CoV‐2 is by real‐time reverse transcriptase‐polymerase chain reaction (RT‐PCR). However, this requires laboratory facilities and the test is time consuming. Rapid antigen (Ag) tests offer a scalable and simpler approach, analyses can be performed by non‐specialized personnel, and test results are available within minutes. Especially the latter can potentially lead to a faster containment of infection in the society. Even though the Ag test has a lower sensitivity compared to the RT‐PCR test it has been shown that optimal screening depends highly on the frequency and speed of testing and is only marginally improved by high sensitivity [4, 5, 6].

So far, the specimens for the Ag tests have primarily been collected by nasopharyngeal swabs [7]. Recently swabs from bilateral in the anterior part of the nose have been introduced as a reliable alternative to the nasopharyngeal swabs that are often accompanied with considerable discomfort for the tested person requiring trained healthcare personnel [8, 9]. A swab for an Ag test inserted in the anterior part of the nose would be an easy‐to‐use method of SARS‐CoV‐2 testing and has since the March 8, 2021 been introduced as the standard practice for collecting material for Ag testing in Denmark. The Danish Government has prioritized a national mass COVID‐19 screening strategy with easy and free access to test facilities for all asymptomatic citizens who are still encouraged to be tested twice a week [10]. In April 2021 the COVID passport (Digital Green Certificate) was implemented as part of the reopening of the society and to facilitate free movement in the EU and thus in Denmark. A COVID passport is granted if citizens have a documented negative COVID‐19 test within the last 72 h, are vaccinated, or have recovered from COVID‐19. Both RT‐PCR and Ag tests are valid as documentation [11, 12].

The evidence on the accuracy of the Ag test with swabs from the anterior part of the nose remains sparse. Sensitivity of the Ag test has been shown to be varying and to be lower in asymptomatic individuals; however, studies on the Ag tests as a screening tool in an asymptomatic cohort with a low SARS‐CoV‐19 prevalence is limited and are warranted and furthermore requested also by Cochrane [13, 14].

The aim of this study was to determine the accuracy of the STANDARD Q COVID‐19 Ag test (SD BIOSENSOR) with swabs collected bilateral from the anterior part of the nose by comparison with RT‐PCR in screening of a public setting with a low SARS‐CoV‐19 prevalence.

MATERIALS AND METHODS

Citizens, above 18 years of age, who had self‐booked an appointment for COVID‐19 test at Testcenter Taastrup, Copenhagen, Denmark, in the period from 2 March to 22 March were offered to participate in the project. The infection pressure in the Capital Region of Denmark including Testcenter Taastrup was below 1.0%, throughout the study period [15, 16].

Citizens were tested without a referral from a medical professional, and thus participants represented the general population with no or with non‐characteristic COVID‐19 symptoms and were not evaluated by a doctor prior to testing. Patients referred to COVID‐19 testing by a doctor with symptoms of COVID‐19 were tested in a separate section of the test centers and were not included in this study [17]. In Denmark, citizens are encouraged by the government during the present re‐opening of the society phase to be tested frequently, that is, every 72 h hence the implementation of the Ag test as part of the screening program to enhance test capacity.

Participants were, in addition to the oropharyngeal swab they already had booked an appointment for, subsequently examined by the Ag test.

RT‐PCR

The oropharyngeal sample for RT‐PCR was collected using a fiber swab touching the palatine tonsils and the posterior wall of the oropharynx as recommended by CDC. The results of the RT‐PCR tests are continuously recorded when available. Detection of SARS‐CoV‐2 was performed by single‐target RT‐PCR at TestCenter Danmark, Statens Serum Institute. Oropharyngeal swabs were collected by the personnel at Testcenter Taastrup and eluted in PBS and RNA was extracted using RNAdvance Blood (Beckman). One‐step RT‐PCR to detect SARS‐CoV‐2 was performed using Luna Universal Probe One‐step RT‐qPCR kit (New England Biolab) [18]. The following primers and probe binding to the E‐gene were used: E_Sarbeco_F (ACAGGTACGTTAATAGTTAATAGCGT), E_Sarbeco_R (ATATTGCAGCAGTACGCACACA), E_Sarbeco_P1 (FAM‐ACACTAGCCATCCTTACTGCGCTTCG‐BHQ1.). Samples with viral cycle threshold (Ct) values between 10 and 38 were considered positive. The results of the RT‐PCR test were considered the golden standard.

Rapid Ag test

The specimens for Ag tests were collected by swabs bilateral from the anterior part of the nose [19]. The same swab was inserted approximately 2–3 cm in each nostril and was examined immediately after collection [20]. The sample collection and use of antigen assay were performed by trained non‐healthcare personnel. The results of the swabs were filed/reported within 15 min electronically via an app.

The STANDARD Q COVID‐19 Ag test produced by SD BIOSENSOR was performed by personnel from the private company Copenhagen Medical A/S and conducted according to SD BIOSENSOR’s instructions (IFU), that is, immediately after the oropharyngeal swab for RT‐PCR testing. Participants received the result of the Ag test by individual links received on their mobile phones.

RT‐PCR of leftover material of Ag test

The leftover material of the positive Ag tests was analyzed by RT‐PCR in order to identify possibly false positive Ag test results. After application of the Ag test material a cassette with the residual test material (approx. 50 µL) was transferred to a sample tube containing guanidinium thiocyanate preservation fluid and analyzed by RT‐PCR.

Statistical analysis

Sensitivity, specificity, positive, and negative predictive values of the Ag test were calculated with test results from RT‐PCR as reference for both Ct values of 30, 33, and 38. Cases deemed positive by RT‐PCR with a Ct value above the defined Ct level of the sub analysis of 33 or 30 were excluded from these analysis. Analysis were performed in R statistics (version 3.6.1).

RESULTS

We included 7074 paired and conclusive tests corresponding to a total of 6824 participants as 240 participants were tested twice or more with a minimum of 1 day between tests. The gender distribution was approximately equal (female: 50.7%, n = 3461 participants). The median age was 48 years (IQR: 36–57 years). About 1% (n = 70) of the RT‐PCR results were missing and 0.4% (n = 30) of the RT‐PCR test results were inconclusive (Ct values: 38–40). Participants with missing data or inconclusive RT‐PCR results were excluded from the analysis, and as we wanted to calculate the accuracy of the rapid antigen test, we only included patients with both a RT‐PCR test and a rapid antigen test done at the same time, thus if the result was missing from the RT‐PCR test patients were excluded even if they had a new RT‐PCR test done another day. No inconclusive Ag test results were found (see Fig. 1).

Fig. 1

Flow diagram of included participants.

One participant with an inconclusive RT‐PCR test result was found positive in the Ag test. The leftover nasal material revealed that the participant was found positive by RT‐PCR. The rest of the participants with inconclusive and missing RT‐PCR results (n = 99) were negative in the Ag test.

A total of 66 RT‐PCR tests were positive, corresponding to a prevalence of SARS‐CoV‐2 0.9%. We found 32 participants with a positive Ag test, all with a corresponding positive RT‐PCR test, equivalent to a sensitivity of 48.5%. No false positive results were found; thus, we found 7008 negative RT‐PCR tests, all with a paired negative rapid Ag test leading to a specificity of 100%. The positive predictive value was also 100% and the negative predictive value was 99.5% (Table 1).

Table 1

Agreement between RT‐PCR test results and antigen test results with Ct < 38

Overall
	RT‐PCR positive (%)	RT‐PCR negative (%)	Total (%)
Antigen test positive (%)	32 (0.45)	0 (0)	32 (0.45)	Positive predictive value: 100%
Antigen test negative (%)	34 (0.48)	7008 (99.07)	7042 (99.55)	Negative predictive value: 99.5%
Total (%)	66 (0.93)	7008 (99.07)	7074 (100)
	Sensitivity: 48.5%	Specificity: 100%

Changing the Ct value to <33 resulted in a sensitivity of 56.2% (Table 2). Sensitivity and specificity with Ct values <30 resulted in a sensitivity of 63.9% (Table 3).

Table 2

Agreement between RT‐PCR test results and antigen test results with Ct<33

Ct < 33
	RT‐PCR positive (%)	RT‐PCR negative (%)	Total (%)
Antigen test positive (%)	27 (0.38)	0 (0.00)	27 (0.38)	Positive predictive value: 100%
Antigen test negative (%)	21 (0.30)	7008 (99.32)	7029 (99.62)	Negative predictive value: 99.7%
Total (%)	48 (0.68)	7008 (99.32)	7056 (100)
	Sensitivity: 56.2%	Specificity: 100%

Table 3

Agreement between RT‐PCR test results and antigen test results with Ct<30

Ct < 30
	RT‐PCR positive (%)	RT‐PCR negative (%)	Total (%)
Antigen test positive (%)	23 (0.33)	0 (0.0)	23 (0.33)	Positive predictive value: 100
Antigen test negative (%)	13 (0.18)	7008 (99.49)	7042 (99.97)	Negative predictive value: 99.5
Total (%)	36 (0.51)	7008 (99.49)	7044 (100)
	Sensitivity: 63.9	Specificity: 100

Out of the 32 positive Ag tests 25 of these had an additional PCR test performed of the leftover nasal material. Seven of the positive Ag tests’ leftover material were not analyzed due to failure in collection method. All the positive Ag tests were also positive by RT‐PCR of the leftover nasal material as well, thus, revealing that no false positive test results were found.

DISCUSSION

This study aimed to investigate the accuracy of the rapid Ag test in a cohort with a low SARS‐CoV‐19 prevalence as requested by Cochrane [13]. To our knowledge this is the first study to investigate the accuracy of the rapid Ag test with swabs collected from the anterior part of the nose in a low prevalence setting and with the Ag test as a screening tool. The study was performed in a public setting and we found a low prevalence of SARS‐CoV‐2 of 0.9%. Participants were not referred neither examined by a doctor and both the Ag test and the RT‐PCR test were performed as a screening in the general population, which enables the results of this study to be generalized. A negative Ag test performed within the last 72 h grants today a COVID passport (Digital Green Certificate) [11, 12].

We found a sensitivity of 48.5% and a specificity of the Ag test based on analysis of anterior nasal swabs of 100%. The Ag test sensitivity shall be interpreted in light of the low prevalence in the investigated population, as low prevalence in general challenges sensitivity of diagnostics test [13]. Furthermore, the lower sensitivity of the Ag test, as described in the introduction, does not affect the important value of the Ag test in screening as long as frequency of the testing is high [4, 5, 6]. Recommendations suggest that the rapid Ag test is useful as an epidemiologic and scalable screening and results in a valid COVID passport when testing is frequent, that is, every third day [4, 5, 6], thus, the Ag test even with a lower sensitivity is an acceptable screening tool taking into account the minimal invasive method of collecting the sample for analysis, the fast result time, and the lower costs. It has further been shown that individuals with a lower Ct value, and thus a high viral load, have been associated with a higher transmissibility of SARS‐CoV‐2. However, it is worth noticing that no obvious cut‐off Ct value for eliminating transmission exists and a substantial amount of household transmission occurred in households where the primary cases had high sample Ct values (low viral load) [21].

No COVID test, that is, neither the RT‐PCR, is 100 reliable as we demonstrated that one inclusive RT‐PCR test (the person did not show up for a re‐RT‐PCR test) showed to be positive when tested with Ag test and was subsequently confirmed by RT‐PCR on the nasal material. It is noteworthy that we did not find any false positive test and the Ag test thus did not lead to unnecessary isolation and quarantine. This might be a problem with infrequent testing with a more sensitive test as individuals in the recovery period, who have a virus load below the infectious threshold, can test positive even though they are not at risk of infecting [4, 5]. This has an important impact for the tested individuals personally and on a larger scale for the society and economic situation in general.

These findings contribute to the understanding that the rapid Ag test is a relevant supplement to RT‐PCR tests with the RT‐PCR remaining the golden standard. Even though RT‐PCR is considered the gold standard for detection of SARS‐CoV‐2 infection, no diagnostic test is flawless. Error in registration and the need of transporting the swab material to laboratories for RT‐PCR testing leads to missing test result, in this study 1.4% of the RT‐PCR tests. This is opposed to the Ag test where all participants had received a test result before leaving to the test facility. As the sensitivity of the Ag test is not as high as with RT‐PCR, this result should, however, be interpreted with caution. The choice of RT‐PCR as reference and the defined criteria for positive results highly affects the sensitivity and specificity of the investigated test [22, 23]. In this study, changing the criteria for positive RT‐PCR from Ct ≤ 38 to Ct ≤ 33 and Ct ≤ 30 increased the sensitivity of the Ag test from 48.5% to 56.2% and 63.9%, respectively. A lower Ct value is equal to a higher viral load and this is associated with a greater risk of transmission of disease and greater risk of symptoms [24]. The difference in sensitivity with regard to the Ct value highlights that the sensitivity is much dependent on the viral load.

Sample collection from the anterior part of the nose opens the doors for the possibility of self‐collected sampling and home testing. However, it is worth noticing that the technique of collecting the sampling affects the sensitivity, thus good collection instructions is most needed [25].

It has been proposed that swabs from the anterior part of the nose is equivalent to swabs collected from the nasopharynx in high prevalence settings [26] and the difference in the sensitivity between this study and our study is probably due to the difference in SARS‐CoV‐2 prevalence.

In December 2020, under the second wave of SARS‐CoV‐2, we performed an almost identical test set‐up, with the same test kit (SD Biosensor) where we investigated the accuracy of the rapid Ag test with nasopharyngeal swabs [14]. The prevalence of SARS‐CoV‐2 infections in the population was at this time considerably higher (4.6%), and the sensitivity of the investigated test method was found to be 69.7%, that is, 20 percentage point higher than found in this study. The difference in the prevalence of SARS‐CoV‐2 infections between the two studies might be the reason for the difference in the sensitivity found between the Ag test with swabs from the nasopharynx and swabs from the anterior part of the nose. Nevertheless, comparing the sensitivity of the Ag test in asymptomatic individuals from our previous study where swabs were collected in the nasopharynx to our current study with swabs from the anterior part of the nose, the sensitivity was equal (sensitivity of 49.2% when samples were collected from the nasopharynx and 48.5% with samples from the anterior part of the nose) [7]. It thus seems sufficient to continue with the swabs from the anterior part of the nose.

A limitation to the study is the comparison of test results from oropharyngeal and nose swabs. Oropharyngeal swabs are the standard in public RT‐PCR test facilities in Denmark and the study is a reflection on the normal test settings. Diagnostic results from RT‐PCR of oropharyngeal and nasopharyngeal swabs are comparable [27] and both methods are in accordance with CDC recommendations [28].

CONCLUSION

In conclusion, this study investigating the accuracy of the rapid Ag test with swabs collected from the anterior part of the nose in a low prevalence (0.9%) massive screening community setting, shows that the rapid Ag test had a sensitivity of 48.5%. Sensitivity of the Ag test increased when changing the criteria for positive RT‐PCR from Ct ≤ 38 to Ct ≤ 33 and Ct ≤ 30. The specificity was 100%, thus we found no risk of a false positive test and no risk of falsely quarantining citizens. This study demonstrates that the capable Ag test, despite the lower sensitivity, is an important tool in SARS‐CoV‐2 screening as long as testing is frequent and that the Ag test is a good supplement to the RT‐PCR in an ambitious public SARS‐CoV‐2 screening.

CONFLICTS OF INTEREST

The authors declare no conflicts of interest.

Funding information

This research received no external funding. Copenhagen Medical A/S delivered the rapid antigen tests and provided test personnel for performing the tests, but were not involved in the study preparation or analysis.

INFORMED CONSENT STATEMENT

Informed consent was obtained from all subjects involved in this study.