Urgent need of rapid tests for SARS CoV-2 antigen detection: Evaluation of the SD-Biosensor antigen test for SARS-CoV-2.

At the time of writing, FIND has listed four CE-marked SARSCoV-2 antigen tests. We evaluated the recently CE-approved rapid POCT SD-Biosensor for SARS-CoV-2 nucleoprotein detection in nasopharyngeal secretions from 330 patients admitted to the Emergency Room for a suspect of COVID-19 and travelers returning home from high risk countries. Sensitivity, specificity, accuracy, negative and predictive values were consistent with the use of the test to mass-screening for SARS-CoV-2 surveillance.

Background

In the context of COVID-19 pandemic, the development of rapid and easy-to-perform diagnostic methods is of high priority, to shorten the time of result-reporting, but this is a condition that demands rapid and cost-efficient approaches. The currently gold standard for the detection of SARS-CoV-2 relies on viral RNA amplification by real-time RT-PCR (RT-PCR) and requires few hours before results release [1]. The pandemic highlighted the limits of production and trade for molecular tests as we are facing a worldwide shortage of reagents. Point-of-care diagnostic tests (POCTs) for detecting viral antigens in clinical samples would be very helpful for the diagnosis of COVID-19 [2] either as mass-screening or first aid tests at the emergency room. At the time of writing, the Foundation for Innovative New Diagnostics (https://www.finddx.org/) has listed four CE-marked rapid SARS-CoV-2 antigen tests, which are primarily lateral flow immunochromatographic assays from nasopharyngeal swab (NP) with result reporting in less than 30 min. The evaluation of these tests by the scientific community, even if limited, outlines the major concern of false-negative results due to low viral loads. In fact, recently published works from different countries evaluating commercial and in-house POCTs for SARS-COV-2 showed a concordant high level of specificity (from 99.5 to 100 %), but a wide range of sensitivity (30 %–93.9 % [3,4]. Therefore, great efforts for the implementation of antigen test performance are currently on-going.

Objective

To evaluate a recently CE-approved POCT, the STANDARD Q COVID-19 Ag (SD-Biosensor, RELAB, I), for the detection of SARS CoV-2 nucleoprotein in NP swabs in comparison with the gold standard RT-PCR. POCT performances were studied in terms of sensitivity, specificity, and negative and predictive values to assess the contribution of this test to mass-screening for COVID-19.

Study design

The STANDARD Q COVID-19 Ag (R-Ag) was applied to 330 patients of two different populations from the phase-1 and -3 of the pandemic according to the Italian Government measures (http://www.governo.it/it/coronavirus-misure-del-governo): 185 randomly selected patients (mean age 44.6, 95 %CI: 40.7–48.6) referring at the Emergency Rooms of two Infectious Disease reference centers in North-Italy (ASL Città di Torino, Turin and San Martino University Hospital, Genoa) with symptoms and sign consistent with of COVID-19 from March 3rd to May 1st, 2020 and 145 travelers (mean age 35.9, 95 % CI: 32.7–39.1) returning home from European high risk countries (Croatia, Spain and Malta) in August, 2020.

Samples were collected in COPAN UTM medium (COPAN, I) and processed for SARS CoV-2 by RT-PCR using different methods: Seegene Allplex® 2019 n-CoV Assay (N = 159), DiaSorin Simplexa® (n = 28), and Cobas 6800 Roche® (N = 118). Cycle threshold (Ct) values were recorded and the mean Ct through different reactive genes was used as a proxy for the viral load. In a minority of samples, COVID-19 Ag antigen test (R-Ag) was run on left-over of diagnostic samples from March and April stored at −20 °C (13/185, 7%), then in parallel with RT-PCR in all the others. Three hundred ul of UTM were mixed to the R-Ag extraction buffer provided by the kit, three drops were applied to the solid device and covered with the proper film. Results were manually read after 15−30 min.

Statistical analysis was performed with R statistical framework: Shapiro-Wilk normality test on Ct values, Student’s T test for comparison between the mean Ct values from R-Ag-positive and R-Ag-negative, ROC curve (pROC package), and Cohen's kappa coefficient (psych package).

Results

Detection rates of SARS CoV-2 by R-Ag and RT-PCR were 23.3 % (77/330) and 33 % (109/330), respectively; no false positive with R-Ag were observed (Table 1 ). R-Ag sensitivity, specificity, negative and positive predictive values were 70.6 %, 100 %, 87.4 % and 100 %, respectively, compared with RT-PCR. Concordance between the two techniques was 90.3 % (Cohen’s k = 0.76, 95 % CI: 0.69−0.84).

Table 1

Results of the SD-Biosensor antigen (R-Ag) test compared to RT-PCR in the samples tested. R-Ag sensitivity, specificity, negative and positive predictive values were 70.6 %, 100 %, 87.4 % and 100 %, respectively, compared with RT-PCR. Concordance between the two techniques was.90.3 %.

	R-Ag +N.	R-Ag –N.	TotalN.	Positive%	Negative%
RT-PCR+	77	32	109	70.6
RT-PCR-	0	221	221		100
Total	77	253	330	23.3

In diagnostic samples (N = 185) R-Ag was positive in 75/104 (72.1 %) RT-PCR positive swabs and negative in 81/81 (100 %) with a sensitivity, specificity, negative and positive predictive values of 72.1 %, 100 %, 73.6 % and 100 %, respectively. Concordance between the two techniques was 84.3 % (Cohen’s k = 0.7, 95 % CI: 0.61−0.8). In screening samples, (N = 145) R-Ag was positive in 2/5 (40 %) RT-PCR positive swabs (asymptomatic patients) and negative in 140/140 (100 %) RT-PCR negative samples (overall sensitivity, specificity, negative and positive predictive values: 40 %, 100 %, 100 % and 97.9 %, respectively, compared to RT-PCR). Concordance between the two techniques was 97.9 % (Cohen’s k = 0.56, 95 % CI: 0.12–1).

The mean Ct values across viral genes by RT-PCR was used as a proxy for viral load; we observed a significantly lower value for concordant RT-PCR-positive/R-Ag-positive samples (22.3, IC95 % 22.26–22.34, range 12.3–36), than that in discordant RT-PCR-positive/R-Ag-negative samples (32.1, IC95 % 32.05–32.15; range 23.7–38.1) (p-value <<0.0001) (Fig. 1 ). Ct values for the two R-Ag groups showed a normal distribution (p-value = 0.39, R-Ag-positive; p = 0.36, R-Ag-negative). According to different mean Ct classes (≤25, 25–28, 28–30, 30–35, >35) the detection rate of R-Ag was 100 % for samples with a Ct <28 and decreased to 38.5 %, 26.7 %, and 9.1 % in the other ranks. Fig. 2 shows the ROC curve for R-Ag to estimate the Ct threshold value for R-Ag to detect a positive swab (equal to 27.7). In Table 2 a simulation of negative and positive predictive (NPV, PPV) values according to disease prevalence is shown.

Fig. 1

Mean Ct across viral genes values for SD-Biosensor antigen (R-Ag) positive and negative tests in studied samples. A subset of R-Ag negative/RT-PCR positive samples were subjected to viral cell culture with negative results after 21-days of incubation.

Fig. 2

ROC curve of sensitivity and specificity of the SD-Biosensor antigen (R-Ag) test in studied samples.

Table 2

Simulation of Negative and Positive Predictive (NPV, PPV) values according to disease prevalence.

	PPV	NPV
Prevalence of COVID-19	Diagnostic samples (N = 185)
0.5 %	100 %	99.86 % (95 % CI 99.81−99.90)
1 %	100 %	99.72 % (95 %CI 99.62−99.79)
2 %	100 %	99.43 % (95 % CI 99.23−99.58)
5 %	100 %	98.55 % (95 % CI 98.04−98.93)
10 %	100 %	96.99 % (95 % CI 95.95−97.78)
Prevalence of COVId- 19	Screening samples (N = 145)
0.5 %	100 %	99.70 % (95 % CI 99.39−99.85)
1 %	100 %	99.4 % (95% CI 98.78−99.70)
2 %	100 %	98.79 % (95 % CI 97.56−99.4)
5 %	100 %	96.94 % (95% CI 93.93−98.48)
10 %	100 %	93.75 % (95 % CI 88.0−96.84)

In a small subset of R-Ag negative/RT-PCR positive samples (15/32) with Ct > 30 we inoculated NP swab in Vero cells. After subculturing with sequential blind passages, in the absence of any cytopathic effect, all supernatants tested negative for SARS CoV-2 by RT-PCR [1] after 21-day incubation.

Discussion

To control COVID-19 pandemic, improvement of SARS CoV-2 diagnosis with easy, rapid and cost-efficient approaches is urgently required. POCTs for the detection of SARS-CoV-2 antigens are quite promising; however, the principal concerns are the false-negative rate due to low viral loads [[3], [4], [5], [6], [7], [8]].

The STANDARD Q COVID-19 Ag identified 70.6 % of RT-PCR positive sample, respectively, with no false positive results (100 % of specificity). Using the Ct value by RT-PCR as a proxy for viral load, R-Ag-positive samples had a significantly lower Ct than that of R-Ag-negative samples; the majority of discordant RT-PCR-positive/R-Ag-negative samples reported negative results when cell-cultured. Therefore, R-Ag false negative results were found in samples with a low viral load, consistent with low viable virus and low infectiousness [9,10]. A major limit of our study was that the test was assessed in suboptimal conditions using UTM samples instead of on-site NP swabs.

The clinical performance of POCTs largely depends on the circumstances in which they are used, and the appropriate setting should be identified. In agreement with recently published works, our data confirm that this POCT is effective during the acute/recent phase of the disease within a few days after symptoms onset when the viral load in the upper respiratory tract is at its peak [[3], [4], [5],11]. The adoption of POCT for SARS CoV-2 testing is certainly more suitable in point of care centers for mass screening where the prevalence of COVID-19 is much lower and the pre-test probability of not having the disease is higher than that in the patients admitted to the emergency room were the pre-test probability of having COVID-19 is significantly higher and false negative results are relevant for the correct management of patients.

The main advantages of POCTs for antigen testing are rapidity, easy of interpretation, limited technical skill and infrastructure required, and this continues to make them worth pursuing. Lastly, the adoption of POCTs in mass screening testing could decrease the burden on virology laboratories that have been overwhelmed during the last COVID-19 pandemics, and the shortage of reagent they are facing.

Funding

We thank RELAb for the donation of the STANDARD Q COVID-19 SD-Biosensor kits to pursue the study. No other specific grant from public funding agencies was received.

CRediT authorship contribution statement

Francesco Cerutti: Writing - original draft, Formal analysis, Conceptualization. Elisa Burdino: Investigation, Validation. Maria Grazia Milia: Investigation. Tiziano Allice: Investigation, Validation. Gabriella Gregori: Investigation, Validation. Bianca Bruzzone: Data curation. Valeria Ghisetti: Conceptualization, Writing - review & editing, Supervision.

Declaration of Competing Interest

The authors report no declarations of interest.