Self-testing for the detection of SARS-CoV-2 infection with rapid antigen tests for people with suspected COVID-19 in the community.

OBJECTIVES: To evaluate the performance of nasal mid-turbinate self-testing using rapid antigen detection tests (RDT) for persons with suspected coronavirus disease 2019 (COVID-19) in the community. Self-testing for COVID-19 infection with lateral flow assay severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) RDT, provides rapid results and could enable frequent and extensive testing in the community, thereby improving the control of SARS-CoV-2.
METHODS: Participants visiting a municipal SARS-CoV-2 testing centre, received self-testing kits containing either the BD Veritor System (BD-RDT) or Roche SARS-CoV-2 antigen detection test (Roche-RDT). Oro-nasopharyngeal swabs were collected from the participants for quantitative RT-PCR (qRT-PCR) testing. As a proxy for contagiousness, viral culture was performed on a selection of qRT-PCR positive samples to determine the Ct-value at which the chance of a positive culture dropped below 0.5 (Ct-value cut-off). Sensitivity and specificity of self-testing were compared to qRT-PCR with a Ct-value below the Ct value cut-off. Determinants independently associated with a false-negative self-test result were determined.
RESULTS: A total of 3201 participants were included (BD-RDT n = 1595; Roche-RDT n = 1606). Sensitivity and specificity of self-testing compared with the qRT-PCR results with a Ct-value below the Ct-value cut-off were 78.4% (95% CI 73.2%-83.5%) and 99.4% (95% CI 99.1%-99.7%), respectively. A higher age was independently associated with a false-negative self-testing result with an odds ratio of 1.024 (95% CI 1.003-1.044).
CONCLUSIONS: Self-testing using currently available RDT has a high specificity and relatively high sensitivity to identify individuals with a high probability of contagiousness.

Introduction

Extensive testing of individuals who are potentially infected with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has a central role in efforts to mitigate the spread of SARS-CoV-2 [1,2]. Self-testing for SARS-CoV-2 infection could enable massive testing in the community, thereby improving the control of SARS-CoV-2 [3,4]. Lateral flow assay SARS-CoV-2 antigen tests (rapid antigen detection tests; RDT) could be suitable candidates for self-testing for SARS-CoV-2 infection [[5], [6], [7]]. Studies on the performance of these RDT have shown promising results when samples were collected and the tests were performed by qualified personnel [[5], [6], [7]]. Recent reports have established the achievability of nasal mid-turbinate self-sampling under supervision [[8], [9], [10]]. Data on the performance of self-testing with RDT is limited to comparisons with quantitative real-time RT-PCR (qRT-PCR) detection of SARS-CoV-2 RNA. The qRT-PCR detects not only intact virus but also non-transmittable SARS-CoV-2 RNA [11,12], and could overestimate the number of contagious patients. Other reports have tried to overcome this limitation by stratifying the results for the cycle threshold value (Ct-value) of the qRT-PCR [5,6]. However, the Ct-value at which patients are expected to be no longer contagious is unknown for most qRT-PCR assays and patient populations. The objective of this study is to evaluate the performance of self-testing for persons with suspected coronavirus disease 2019 (COVID-19) (e.g. patients with COVID-19 related symptoms or close contacts of patients with COVID-19) using two commercially available RDT: BD Veritor System for Rapid Detection of SARS-CoV-2 (Becton Dickinson, Franklin Lakes, NJ, USA) (BD-RDT) and Roche SARS-CoV-2 antigen detection test (Roche, Basel, Switzerland) (Roche-RDT) for the detection of contagious COVID-19 patients in the community.

Materials and methods

Study design and participants

This manufacturer-independent cross-sectional study was conducted from 23 December 2020, to 17 January 2021, in the test centre of the Municipal Health Services (MHS) in Tilburg, Noord-Brabant, the Netherlands. In the Netherlands, community testing for SARS-CoV-2 is coordinated by the MHS. Persons with COVID-19-related symptoms or persons with close contact with a confirmed COVID-19 patient, can make a free-of-charge appointment at these test centres for a SARS-CoV-2 test, without consulting a health-care professional. Consecutive persons aged 18 years or older who presented at the test centre, were able to understand the written instructions in Dutch and provided verbal informed consent and contact information (e-mail address and telephone number) were deemed eligible for inclusion.

Study procedure

Eligible participants were randomly allocated by a traffic controller to either a test lane distributing the BD-RDT self-testing kit or a test lane distributing the Roche-RDT self-testing kit. Participants received a small bag with the self-testing kit, were instructed to perform the SARS-CoV-2 self-test immediately after arrival at home and were asked to provide their e-mail address and telephone number. At the test centre, the by that time prevailing method for SARS-CoV-2 testing was carried out. Both an oropharyngeal and a nasopharyngeal swab were collected from the participants by a trained member of the MHS and both swabs were suspended in a single container with 3 mL gelatin–lactalbumin–yeast virus transport medium (Mediaproducts, Groningen, the Netherlands). The suspended swabs were sent to Microvida Laboratory for Medical Microbiology and Immunology, Tilburg, the Netherlands, for qRT-PCR testing within 4 hours after sample collection. Participants received an e-mail with a link to a digital survey form. The survey content is presented in the Supplementary material (Methods S1). Results of the RDT had to be available to the study team before qRT-PCR results were communicated with the participant. When the participant did not complete the survey within 2 hours following inclusion, the participant was telephoned by a member of the research team to fill in the survey form jointly. If the participant had not yet performed the self-test when being telephoned, the member of the research team asked the participant to perform the test and fill in the survey form sent via e-mail. Participants were not assisted during the self-testing procedure.

Self-testing

Participants received a self-testing package containing either a BD-RDT or a Roche-RDT, a flocked swab, a foldable cardboard test frame, and a written and illustrated booklet including general information on the study and an instruction on how to collect a mid-turbinate nasal sample, how to perform the test and how to interpret the test result. This instruction included a QR-code link to a 2-minute online video illustrating mid-turbinate self-sampling and self-testing using the BD-RDT (http://www.corona-test-instructies.nl/) and Roche-RDT (http://www.coronatest-instructies.nl/) (see Supplementary material, Methods S2 and S3). Participants performed the BD-RDT and the Roche-RDT on self-collected mid-turbinate nasal samples. Only the BD-RDT product information lists nasal mid-turbinate samples as suitable samples.

Laboratory methods

The qRT-PCR was performed using either the Alinity assay (AA) or a laboratory-developed assay (LDA). Samples with a positive qRT-PCR result that were collected before 12 January 2021 were sent for viral culture. An extensive description of virus culture and qRT-PCR protocol is provided in the Supplementary material (Method S4) [[13], [14], [15]].

Sample size

At the start of the study, the diagnostic accuracy of self-performed RDT was unknown. We assumed the diagnostic accuracy to be lower than when performed by professionals, and based the sample size calculation on an expected sensitivity of 80% for infectious individuals, with a margin of error of 7%, type I error of 5% and power of 90%. Hence, the minimum number of participants with a positive qRT-PCR test was 140 per RDT arm. The qRT-PCR test positivity percentage was monitored over time and recruitment was adjusted if needed.

Statistical analysis

Statistical analyses were performed using Statsmodels v0.12.2 in python v3.8. The primary outcome was the sensitivity and specificity of RDT compared with qRT-PCR with a Ct-value lower than or equal to a Ct-value cut-off that correlated with at least a 50% chance of recovering a viable virus using viral culture. A detailed description of the various statistical analyses performed is provided in the Supplementary material (Method S5).

Ethics

The study protocol was reviewed by the Dutch ‘Medical research Ethics Committees United’ (MEC-U). The study was judged to be beyond the scope of the Dutch medical scientific research act (WMO) (MEC-U subject: W20.302). A waiver of written informed consent was granted to enable the required high flow of individuals in the test centre and prevent any safety hazard associated with the handling of documents obtained from possibly infectious participants.

Results

In total, 3529 eligible participants were included in the study, of whom 3201 (90.7%) were willing and able to complete and share the survey results (Fig. 1 ). Of the 3201 respondents, 1595 (49.8%) received a self-testing package containing the BD-RDT and 1606 (50.2%) received the self-testing package containing the Roche-RDT (Fig. 1). No difference in baseline characteristics of the participants who completed and shared their survey result were detected between the two groups of respondents (Table 1 ). Because of sample loss, the qRT-PCR was not performed for 11 (0.3%) of the 3201 samples. The AA and LDA were performed in 990 (31.0%) and 2200 (69.0%) samples, respectively. Results of the qRT-PCR were positive in 376 (11.8%) samples, negative in 2811 (88.1%), and inconclusive in 3 (0.1%). Out of the 376 samples with a positive qRT-PCR result, 288 were sent for viral culture (AA: n = 85 samples; LDA: n = 203 samples). Of those, 177 (61.5%) had a positive culture result (see Supplementary material, Table S1). For both the AA and the LDA the Ct-values in the samples with a positive viral culture (median Ct-value AA: 17.9, interquartile range (IQR) 16.1–19.8; median Ct-value LDA: 18.1 IQR 16.4–20.5) were lower than in the samples with a negative viral culture (median Ct-value AA: 28.3, IQR 25.1–33.1; median Ct-value LDA: 31.0, IQR 25.0–34.8) (Fig. 2 ). Based on a univariate logistic regression model, the Ct-value cut-off, where the chance for a positive viral culture was 0.5 or lower, was 23.0 (95% CI 16.0–43.0) for the AA (pseudo-R 2 0.64) and 24.5 (95% CI 20.4–33.6) for the LDA (pseudo-R 2 0.57) (see Supplementary material, Figs S1 and S2).

Fig. 1

Participant flow diagram. ∗Samples of participants were included in the analysis determining the Ct-value cut-off where the p for a positive viral culture was smaller than 0.5 but not in the analysis determining the sensitivity and specificity of self-testing.

Table 1

Baseline characteristics of study participants

	Overall	Test type
		BD-RDT	Roche-RDT
	(n = 3201)	(n = 1595)	(n = 1606)
Age (years), median (IQR)	41 (29–54)	41 (29–54)	41 (29–54)
Gender, n (%)
Male	1406 (43.9)	718 (45.0)	688 (42.8)
Female	1806 (56.5)	877 (55.0)	918 (57.2)
Highest level of education, n (%)
Elementary school	61 (1.9)	33 (2.1)	28 (1.7)
High school	1607 (50.2)	828 (51.9)	779 (48.5)
Bachelor degree	1075 (33.6)	516 (32.5)	559 (34.8)
Masters degree or higher	458 (14.3)	218 (13.7)	240 (14.9)
Currently symptoms of COVID-19 infection, n (%)
Yes	2216 (69.2)	1114 (69.8)	1102 (68.6)
No	985 (30.8)	481 (30.2)	504 (31.4)
Symptoms of COVID-19 infection in past 3 weeks, n (%)
Yes	200 (6.2)	98 (6.1)	102 (6.4)
No	3001 (93.7)	1497 (93.9)	1504 (93.6)
No symptoms of COVID-19 infection, n (%)
Yes	785 (24.5)	383 (24.0)	402 (25.0)
No	2416 (75.5)	1212 (76.0)	1204 (75.0)

Abbreviations: BD-RDT, BD Veritor System for Rapid Detection of SARS-CoV-2 (Becton Dickinson, Franklin Lakes, NJ, USA); COVID-19, coronavirus disease 2019; IQR, interquartile range; RDT, rapid antigen detection test; Roche-RDT, Roche SARS-CoV-2 antigen detection test (Roche, Basel, Switzerland).

Fig. 2

Ct-value distribution per quantitative RT-PCR platform for viral culture positive and viral culture negative samples and for rapid antigen detection test (RDT) positive and RDT negative samples.

Out of the total of 3201 RDT self-tests performed, 210 (6.6%) tests were positive (BD-RDT n = 88; Roche-RDT n = 122), 2943 (91.9%) tests were negative (BD-RDT n = 1476; Roche-RDT n = 1467), and 48 (1.5%) tests yielded an inconclusive result (BD-RDT = 31; Roche-RDT n = 17) (Fig. 1). The proportion of inconclusive results differed significantly between the BD-RDT and Roche-RDT (p 0.039). No difference in responses to the survey statements between participants receiving a BD-RDT or Roche-RDT was detected (see Supplementary material, Table S2). The sensitivity and specificity of self-testing compared with qRT-PCR were 55.6% (95% CI 50.5%–60.7%) and 99.8% (95% CI 99.6%–99.9%), respectively (Table 2 ; see Supplementary material, Table S3). For both the Roche-RDT and BD-RDT the specificity was high, being 99.7% (95% CI 99.4%–99.9%) and 99.9% (95% CI 99.7%–100%), respectively. The sensitivity of self-testing using the Roche-RDT (61.5%, 95% CI 54.6%–68.3%) was higher than when using the BD-RDT (49.1%, 95% CI 41.7%–56.5%) (Table 2; see Supplementary material, Table S3). When only samples with a Ct-value in the qRT-PCR below the previously defined Ct-value cut-off were considered positive, the sensitivity of self-testing using both the Roche-RDT and BD-RDT increased to respectively 80.1% (95% CI 73.4%–86.9%) and 76.1% (95% CI 68.1%–84.1%), with a relatively small decrease in specificity (BD-RDT: 99.7%, 95% CI 99.4%–99.9%; Roche-RDT: 99.1%, 95% CI 98.6%–99.6%) (Table 2; see Supplementary material, Table S3). When comparing the result of the self-test to the composite reference standard, the sensitivity was 77.2% (95% CI 71.3%–83.1%) (Table 2; see Supplementary material, Table S4). This sensitivity was similar to that comparing self-testing results to qRT-PCR positive samples with a Ct-value below the Ct-value cut-off. The association between a false-negative test result in the self-test compared with the qRT-PCR with a Ct-value below the Ct-value cut-off was investigated for five variables using a univariate logistic regression model (Table 3 ). Out of these five variables, two were associated with the occurrence of a false-negative self-test result with a p value < 0.2 and were included in the multivariate analysis (Table 3). Higher age was independently associated with the occurrence of a false-negative self-test result (Table 3).

Table 2

Sensitivity and specificity of the self-testing compared with quantitative RT-PCR, quantitative RT-PCR with a Ct-value below the cut-off and to the composite reference standard

	Sensitivity (95% CI)			Specificity (95% CI)
	Overall	BD-RDT	Roche-RDT	Overall	BD-RDT	Roche-RDT
qRT-PCR pos. or neg.	55.6% (50.5%–60.7%)	49.1% (41.7%–56.5%)	61.5% (54.6%–68.3%)	99.8% (99.6%–99.9%)	99.9% (99.7%–100%)	99.7% (99.4%–99.9%)
qRT-PCR, Ct-value < cutoffa	78.4% (73.2%–83.5%)	76.1% (68.1%–84.1%)	80.1% (73.4%–86.9%)	99.4% (99.1%–99.7%)	99.7% (99.4%–99.9%)	99.1% (98.6%–99.6%)
Composite reference standardb	77.2% (71.3%–83.1%)	75.9% (66.7%–85.1%)	78.8% (70.5%–85.8%)	99.8% (99.6%–99.9%)	99.9% (99.7%–100.0%)	99.7% (99.4%–99.9%)

Abbreviations: BD-RDT, BD Veritor System for Rapid Detection of SARS-CoV-2 (Becton Dickinson, Franklin Lakes, NJ, USA); COVID-19, coronavirus disease 2019; IQR, interquartile range; neg.: negative; pos., positive (Ct-value <45.0); qRT-PCR, quantitative RT-PCR; RDT, rapid antigen detection test; Roche-RDT, Roche SARS-CoV-2 antigen detection test (Roche, Basel, Switzerland).

Ct-value below which the chance of a positive culture was larger than 0.5 (Alinity assay: 23.0; Laboratory-developed assay 24.5).

Composite reference standard was defined as positive when having a positive result in at least two out of the following three tests: viral culture, qRT-PCR and RDT.

Table 3

Univariate and multivariate logistic regression analysis for the occurrence of false-negative self-test compared with quantitative RT-PCR with a Ct-value below the Ct-value cut-off

Variable	Univariate analysis		Multivariate analysis
	Odds ratio (95% CI)	p-value	Odds ratio (95% CI)	p-value
Age (years)	1.024 (1.003–1.044)	0.023	1.022 (1.001–1.042)	0.038
Current COVID-19 related symptomsa	0.543 (0.260–1.134)	0.104	0.607 (0.286–1.288)	0.193
Roche-RDTa	0.833 (0.454–1.527)	0.554
Malea	0.921 (0.502–1.688)	0.790
Highest level of education (1–4)	1.326 (0.857–2.053)	0.205

Abbreviations: COVID-19, coronavirus disease 2019; RDT, rapid antigen detection test; Roche-RDT, Roche SARS-CoV-2 antigen detection test (Roche, Basel, Switzerland).

Yes: 1; No: 0.

Discussion

Self-testing using commercially available RDT proved to be feasible and delivered reliable results. Specificity was extremely high (>99%) whereas sensitivities were 76.1% (BD-RDT) and 80.1% (Roche-RDT). In addition, the sensitivity was higher in younger participants. In a previous study, viral loads of SARS-CoV-2 in patients older than 20 years did not differ between different age groups [16]. Suggesting that the observed higher sensitivity of self-testing in younger participants could be due to better performance of the self-test by young participants. Future studies could target specific age groups and see if this improves performance.

To assess the performance of the RDT we determined the Ct-value below which viable virus could readily be detected. The two qRT-PCR platforms showed similar Ct-value patterns for samples with positive and negative viral cultures with minor differences in the threshold for positive viral culture. We used viral culture as a reference to determine the cut off as we aimed to determine the sensitivity of the RDT to detect infectious individuals. Quantitative RT-PCR is a highly sensitive method that can detect viral RNA for prolonged periods after the initial infection. Most of these individuals however, are no longer infectious [12]. There is an ongoing discussion about the value of qRT-PCR with high Ct-values for this kind of evaluation as it is clear that most high Ct-values do not represent infectious cases. However, the exact threshold is unclear and varies between qRT-PCR platforms [17]. For the two qRT-PCR platforms used in this study the thresholds for the presence of viable virus were determined at Ct 23.0 for the AA and Ct 24.5 for the LDA. These values cannot be extrapolated to other platforms.

There are limited published data on the reliability of self-testing. A recent study conducted in Germany found that a layperson can be trained to administer a rapid self-test properly [9]. The study involved 146 individuals showing symptoms, of whom 40 tested positive for SARS-CoV-2 using a qRT-PCR test. All participants then conducted additional self-tests using nasal swabs. Of all those who tested positive, 91.4% were able to confirm their result via rapid self-test. Practically all of those who tested negative were able to confirm their result with a self-test. This shows the potential of self-testing. However, it was not a real-life evaluation. As far as we are aware, we present the first large-scale clinical evaluation of self-testing for SARS-CoV-2. The results show that self-testing is possible and delivers useful results. As sensitivity was not optimal, self-testing should not be used as a diagnostic tool for severely ill patients to guide possible therapies in a hospital setting. The poorer sensitivity of self-testing using RDT compared with qRT-PCR on nasopharyngeal swabs taken by health-care professionals is probably both due to the intrinsic lesser sensitivity of RDT when compared with qRT-PCR and poorer self-sampling and self-test performance [18]. However, the test characteristics are suitable for large-scale preventive testing programmes to open up specific activities in society. A modelling study showed that for preventive testing the frequency of testing is much more important than the sensitivity of the test [3]. For example, a test with 80% sensitivity performed by at least 70% of the population once weekly was estimated to reduce the effective reproduction number estimate (Rt) from 1.5 to below 1.0. This may facilitate opening up activities like education, contact professions, high-risk jobs (e.g. slaughterhouses) and so on. These theoretical considerations should be confirmed in real-life settings.

The present study has some limitations. Participants under the age of 18 years were not included. Therefore, no information is available on the performance of self-testing in children. Moreover, the current study only included participants who arrived at the test centre of the MHS. Possibly excluding those individuals who are less mobile, higher age groups and severely ill patients who could not travel to a nearby test centre. Additionally, positive and negative predictive values of self-testing could be different for other geographical regions at different phases of the pandemic due to changing prevalence of COVID-19 in the community. In the current study, a model was used to determine the Ct-value at which the chance of a positive viral culture was 50% or lower. Using a model rather than culture positivity could eliminate possible circumstances that would have resulted in false-negative culture results in specific samples (e.g. long time before freezing a specific sample). However, viral culture positivity is only a surrogate marker for infectiousness. Therefore, infectious COVID-19 patients could have had a Ct-value in the SARS-CoV-2 qRT-PCR higher than the Ct-value cut-off.

In conclusion, we showed that self-testing using currently available RDT has a high specificity and relatively high sensitivity (75%–80%) to identify individuals with a high probability of contagiousness. This application has the potential for frequent and extensive testing, which may be an aid to lift current restrictions to society.

Author contributions

JS supported in the conceptualization and methodology of the study, performed the data curation and formal analysis of the results and has written the original draft. VF and WB supported in the conceptualization and methodology of the study, performed the data curation and formal analysis of the results and have reviewed and edited the manuscript. GG, AM, SP, JV, JLM performed the experiments and have reviewed and edited the manuscript. CN, FO and MH performed the project administration and data acquisition of the study and have reviewed and edited the manuscript. MK supported the methodology of the study, performed the data curation and formal analysis of the results and has reviewed and edited the manuscript. KG and AR performed the data acquisition of the study, and reviewed and edited the manuscript. JK led the conceptualization of the study, supervised the methodology, data curation and formal analysis of the results, and wrote the original draft.

Transparency declaration

All of the authors have declared that there are no conflicts of interest to declare. This research was funded by the .