Evaluation of the rapid antigen detection test STANDARD F COVID-19 Ag FIA for diagnosing SARS-CoV-2: experience from an Emergency Department.

The purpose of this study was to assess the clinical performance of STANDARD F COVID-19 Ag FIA (SD Biosensor Inc., Gyeonggi-do, Republic of Korea), a rapid antigen detection test (RADT) for diagnosing SARS-CoV-2, in patients attended at the Emergency Department with signs or symptoms compatible with COVID-19 that had started in the last 5 days. The clinical performance of the antigen test was compared with RT-PCR, the reference standard. We included 663 specimens from non-repetitive patients. Clinical sensitivity and specificity were 84.0% (95% CI 76.1-89.7) and 99.6% (95% CI 98.5-99.9), respectively. The positive and negative predictive values were 98.1% (95% CI 92.7-99.7) and 96.4% (95% CI 94.4-97.7), respectively. The kappa index agreement between RT-PCR and the RADT was 0.89 (95% CI 0.84-0.93). We concluded that STANDARD F COVID-19 Ag FIA is an excellent first-line RADT method to diagnose symptomatic patients in the emergency department.

Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has rapidly spread worldwide becoming a great diagnostic challenge for microbiology laboratories [1]. An early and rapid SARS-CoV-2 detection has become mandatory for preventing the virus spread [2]. Although the RT-PCR is the gold-standard method to diagnose the infection, point of care (POC) methods remain a highly useful tool to speed up the diagnostic process [3,4].

Although there are POC systems based on RT-PCR with final results in less than 1 hour [5,6], its use to perform large screenings represent huge economical costs. On the contrary, rapid antigen detection tests (RADT) represent both, economic and fast POC methods to accelerate diagnosis. Nevertheless, these RADT have lower sensitivity and specificity than molecular-based diagnostics [7,8]. This loss in accuracy forces an evaluation in different real-life scenarios in order to assess their accuracy and utility.

The purpose of this study was to assess the clinical performance of STANDARD F COVID-19 Ag FIA (SD Biosensor Inc., Gyeonggi-do, Republic of Korea), an automated RADT based on immunofluorescence detection for diagnosing SARS-CoV-2. The evaluation was performed using nasopharyngeal swabs from patients attended in the Emergency Department with signs or symptoms compatible with COVID-19 that had started in the last 5 days.

Materials and methods

This prospective study was conducted from October 14, 2020 to November 18, 2020 in a single tertiary university hospital, Hospital Universitario Marqués de Valdecilla (HUMV), in Santander (Spain). Patients attended in the Emergency Department of HUMV with signs or symptoms compatible with COVID-19 that had started in the last 5 days were recruited. Demographic data were no collected.

Left-over nasopharyngeal swab specimens used firstly in routine RT-PCR-based diagnosis in the hospital were used in antigen test evaluation within 24 hours after collection. Specimens were kept at 4°C until testing. Virus transport and preservation media used for nasopharyngeal testing included Biocomma (Biocomma Limited, ShenZen, China) and DeltaSwab (Deltalab, Rubí, Spain). Both systems are nasopharyngeal flocked swabs with non-inactivated preservation virus transport medium and were employed interchangeably.

The clinical performance of the RADT STANDARD F COVID-19 Ag FIA was compared with RT-PCR, the reference method for SARS-CoV-2 diagnosis. The RT-PCR reagent used was VIASURE SARS-CoV-2 Real Time PCR Detection Kit (Certest Biotec, Zaragoza, Spain), based on amplification and detection of ORF1ab and N genes. The limit of detection of the RT-PCR was ≥10 copies of viral RNA for both genes.

Nucleic acid extraction was performed using automated systems as Biocomma Nucleic Acid Purification Kit (Biocomma Limited, ShenZen, China) and IndiMag Pathogen IM48 Cartridge (Indical Bioscience, Leipzig, Germany). Amplification was performed on a BioRad CFX96 thermocycler (BioRad Laboratories, the Netherlands). The cycle threshold (Ct) was automatically determined by the manufacturer's software. A positive RT-PCR result was defined as amplification of any of the 2 SARS-CoV-2 target genes, following manufacturer's instructions.

RADT was performed following manufacturer's instructions. Reading of the results was performed using a F200 or F2400 analyser (SD Biosensor Inc., Gyeonggi-do, Republic of Korea), which automatically reads the intensity of fluorescence following antibody–antigen complex formation. A sample is considered positive if cutoff index (COI) value is ≥1. The results obtained with the antigen test in this study did not impacted patient management and informed consent was not needed.

Results

A total of 663 specimens from non-repetitive patients were included in the study. In 125 specimens (18.9%) the RT-PCR result was positive, while the RADT detected 105 (15.8%). A negative result was obtained in 538 specimens (81.1%) by RT-PCR and in 558 (84.2%) by the RADT.

Clinical sensitivity and specificity of the RADT compared with RT-PCR were 84.0% (95% CI 76.1−89.7) and 99.6% (95% CI 98.5−99.9), respectively. The positive and negative predictive values were 98.1% (95% CI 92.7−99.7) and 96.4% (95% CI 94.4−97.7), respectively. The kappa index agreement between RT-PCR and the RADT was 0.89 (95% CI 0.84−0.93), which means an almost perfect agreement (Table 1 ).

Table 1

Clinical performance of antigen test compared with RT-PCR.

Sensitivity	84.0 %	95% CI (76.1−89.7)
Specificity	99.6 %	95% CI (98.5−99.9)
Positive predictive value	98.1 %	95% CI (92.7−99.7)
Negative predictive value	96.4%	95% CI (94.4−97.7)
Kappa index	0.89	95% CI (0.84−0.93)

CI = confidence interval.

Two false positive results (FP, 0.3%) were obtained with the RADT. Cutoff index values (COI) of FP were 1.22 and 1.69. Twenty false negatives results (FN, 3.0%) were found with the antigen test. In 16 out of 20 of FN the Ct obtained with the RT-PCR was ≥ 30 (Table 2 ).

Table 2

Antigen test false negative results compared with RT-PCR (n = 20).

Ct, Gene N	Ct, Gene ORF1a	COI
32	34	0,07
36	35	0,59
25	23	0,87
29	30	0,08
37	36	0,24
27	24	0,33
30	29	0,66
35	34	0,26
34	34	0,17
33	34	0,06
36	40	0,19
28	26	0,57
36	35	0,02
34	34	0,23
27	24	0,34
36	38	0,25
32	29	0,08
31	30	0,12
38	37	0,31
33	31	0,08

Ct = cycle threshold. Target genes: N and ORF1a.

COI = cutoff index value.

Sensitivity of RADT according to Ct results of RT-PCR has also been analysed (Table 3 ). Sensitivity of the RADT when testing positive samples for SARS-CoV-2 with Ct ≤30, was 92.9%. In samples with Ct ≥31, sensitivity of the RADT was 7.7%.

Table 3

Sensitivity of antigen test according to Ct results of RT-PCR.

	Patients, n	Sensitivity
Ct ≤15	6	100%
Ct 16−20	47	100%
Ct 21−25	41	92.7%
Ct 26−30	18	72.2%
Ct ≥31	13	7.7%
Total	125	84.0%

Ct = cycle threshold.

For comparison purposes the lowest Ct value obtained between N or ORF1a genes was considered.

Discussion

Rapid detection of infected patients with SARS-CoV-2 is mandatory to interrupt transmission of the virus [2]. Rapid molecular methods based on RT-PCR offer results in less than 1 hour, however the price is not feasible for large-scale screening. For this reason, RADT represent an efficient alternative as first-line diagnostic method combining both advantages: low price and fast results. Notwithstanding, performance of RADT is highly dependent on the time elapsed since the onset of symptoms, since this accuracy is closely linked to the viral load [9]. Clinical evaluations in different scenarios are imperative in order to use the correct diagnostics in the correct time.

In this evaluation the antigen test STANDARD F COVID-19 Ag FIA presented an excellent clinical performance (84.0% sensitivity and 99.6% specificity) and an almost perfect agreement compared with the reference method, the RT-PCR, when used in patients with signs or symptoms compatible with COVID-19 that had started in the last 5 days. This finding is consistent with other RADT evaluations performed in symptomatic patients with similar clinical courses (onset of symptoms ≤5 days) [7,10].

The antigen test STANDARD F COVID-19 Ag FIA shows good clinical sensitivity (92.9%) in symptomatic patients with high viral loads (Ct ≤30). Meaning that this RADT is effective in cases of active viral replication, i.e., in the early acute phases of infection. As expected, sensitivity of the RADT decreases as the viral load in patients becomes lower (7.7% sensitivity; Ct ≥31). Similar findings have been reported previously [11,12]. The main reason of FN of RADT is a lack of sensitivity in cases with high Ct. In this evaluation; in the 80% of FN the Ct was ≥30.

Comparing our findings with other evaluations of the STANDARD F COVID-19 Ag FIA, there are disparities of results. Baccani et al, showed an overall sensitivity of 35.7%, while Orsi et al, of 86.7%. Both evaluations showed 100% sensitivity in samples with Ct <26 [13,14]. As seen in our study, time from the onset of symptoms is determinant for a correct use of RADT.

This evaluation has been done in a high pre-test probability scenario. During the study period (October 14 – November 18, 2020) the cumulative incidence rate per 100,000 inhabitants in the last 14 days, rise from 118 to 547 in the region of Cantabria. Together with a final high positive predictive value of the test, no confirmation of positive samples with RT-PCR was needed. However, in our experience, because of the 2 false positive results with antigen test (COI: 1.22 and 1.69), we decided to include in our internal diagnostic algorithm a confirmatory RT-PCR in samples with antigen test result with COI values between 1 and 2 in order to obtain 100% of specificity. This is consistent with a previous work showing a median COI value of 1.4 in samples performed with the same RADT and not confirmed to be positive by RT-PCR [15].

A further advantage of the STANDARD F COVID-19 Ag FIA is the automatic test interpretation of results, thus avoiding possible misinterpretations of RADT based on visual reading. Additionally, the analyser could automatically transmit the results to the laboratory information system, increasing the efficiency of the RADT. It is a rapid, inexpensive and reliable assay with a great value as a point-of-care-technique especially in settings as an Emergency department, where a short time of results is crucial for a better patient management.

An actual matter of concern is how SARS-CoV-2 new variants and mutations could affect the performance of COVID assays. As most RADT capture the viral nucleocapsid protein, and typically mutations affect in the Spike protein, mutations are not likely to limit the performance of RADT [16]. Nevertheless, since mutations in the nucleocapsid protein affecting performance of RADT have already been reported [17], continuous control of accuracy of diagnostic tests is mandatory.

A limitation of the study is that not information about cellularity of nasopharyngeal swabs was provided. We aware some of discrepant results could be explain with this data.

Based on these data, the antigen test STANDARD F COVID-19 Ag FIA is an excellent first-line method to diagnose symptomatic patients in both emergency departments and primary healthcare centres.

Author contributions

Sergio Garcia-Fernandez: Supervision, Conceptualization, Investigation, data curation, validation, writing original draft. Daniel Pablo-Marcos: Investigation, data curation, validation, review and editing of draft. Silvia Velasco de la Fuente: Investigation, data curation, validation, review and editing of draft. María José Reina Rodríguez: Investigation, data curation, validation, review and editing of draft. Mónica Gozalo: Investigation, data curation, validation, review and editing of draft. Jesús Rodríguez-Lozano: Investigation, data curation, validation, review and editing of draft. José Manuel Méndez-Legaza: Investigation, data curation, validation, review and editing of draft. Jorge Calvo: Supervision, conceptualization, Investigation, data curation, validation, review and editing of draft.

Declaration of competing interest

The authors report no conflicts of interest relevant to this article