Comparative evaluation of Panbio and SD Biosensor antigen rapid diagnostic tests for COVID-19 diagnosis.

The aim of our study was to evaluate the diagnostic performance of two antigen rapid diagnostic tests (Ag-RDTs) to diagnose severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection. We evaluated Panbio and SD-Biosensor Ag-RDTs. We employed 186 polymerase chain reaction (PCR) negative samples to evaluate the specificity and 170 PCR positive samples to assess the sensitivity. We evaluated their sensitivity according to Cycle threshold (C t ) values and days post onset of symptoms (d.p.o.). Tests were compared using the McNemar's test. Agreement was evaluated using the kappa score. Specificity was 100% for Panbio and 97.3% for SD-Biosensor. Sensitivity for samples with C t  ≤ 20 was 100% for both assays and for samples with C t  = 20-25 was 93.0% (Panbio) and 95.3% (SD-Biosensor) (p =  1.000). Sensitivity decreased for samples wit C t  = 25-30 (Panbio: 41.3%, SD-Biosensor: 52.2%, p = 0.125) and samples with C t  ≥ 30 (Panbio: 5.0%, SD-Biosensor: 17.5%, p = 0.063). Sensitivity within seven d.p.o. was 87.7% for Panbio and 90.4% for SD-Biosensor and notably decreased after seven d.p.o. Agreement with PCR was excellent for high viral load samples (C t  ≤ 25): Panbio, 98.9%, kappa = 0.974; SD-Biosensor, 97.4%, kappa = 0.940. Agreement between Ag-RDTs was excellent (94.9%, kappa = 0.882). Panbio and SD-Biosensor Ag-RDTs showed excellent agreement and diagnostic performance results for samples with high viral loads (C t  ≤ 25) or samples within seven d.p.o.

INTRODUCTION

The pandemic due to severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) has become one of the main global economic and health priorities. Viral RNA detection in respiratory samples using polymerase chain reaction (PCR) is the current reference method for coronavirus disease 2019 (COVID‐19) diagnosis. However, they are not useful as point of care (POC) tests due to an excessive turnaround time for results.1, 2 Antigen rapid diagnostic tests (Ag‐RDTs) have been developed as alternative tests to PCR for COVID‐19 symptomatic patients, as they could be employed as POC tests, presented a lower cost than PCR assays and could improve the turnaround time for results. Moreover, some authors have shown that these advantages could overcome the sensitivity limitation, especially where PCR is unavailable or where prolonged turnaround times preclude clinical utility.3, 4, 5 Our objective was to evaluate the diagnostic performance of two of these commercialized Ag‐RTDs.

METHODS

Population and study period

The study was performed between 1 and 30 December 2020. We included 356 nasopharyngeal samples, which were submitted in 3 ml of universal transport medium (UTM). Each sample corresponded to one single patient and we included 186 PCR negative samples and another 170 PCR positive samples.

Diagnostic techniques

RNA amplification was performed using three Real‐Time PCR platforms: Allplex SARS‐CoV‐2 assay (Seegene, which detected SARS‐CoV‐2 E, N and RdRP genes), Viasure SARS‐CoV‐2 Real Time PCR Detection Kit (Certest Biotech S.L.; detected genes: ORF1ab and N) and GeneFinder COVID‐19 Plus RealAmp Kit (Osang Healthcare Co.; detected genes: E, N and RdRP). Nasopharyngeal samples were tested using one or another PCR platform indistinctly, according to the usual laboratory workflow. Samples were considered as positive when amplification was detected for all genes included inf each RT‐PCR assay. Reliability of these three platforms had been previously evaluated in our laboratory, showing 100% agreement and quite similar C t values in the results for those samples that amplified all the targets covered by each technique (see Table S1).

Regarding Ag‐RDTs, we evaluated two Ag‐RDTs that detected SARS‐CoV‐2 nucleoprotein antigens: Panbio COVID‐19 Ag Rapid Test Device (Abbot Rapid Diagnostics GmbH, Jena) and SD‐Biosensor STANDARD F COVID‐19 Ag FIA (SD Biosensor, Inc.). Lecture of the results was optical for Panbio and by immunofluorescence using a STANDARD F2400 analyzer (SD Biosensor, Inc.) for SD‐Biosensor Ag‐RDT.

All equipments were used according to the manufacturer's instructions for both the handling and the interpretation of the results.

Clinical data

Clinical variables of the study population (time from the onset of symptoms) were obtained from the medical records. We assessed SARS‐CoV‐2 viral load using the Cycle threshold (C t) value corresponding to N gene for all PCR positive samples.

Statistical analysis

Categorical variables were expressed as proportions and continuous variables as median and interquartile range (IQR) and. Sensitivity and specificity with 95% confidence intervals (95% CI) were calculated using RT‐PCR as gold standard. Sensitivity was evaluated globally and also according to the C t value for N gene using different cutoffs (high viral load samples: C t    20 and C t = 20–25; low viral load samples: C t = 25–30 and C t>30) and the days post onset of sypmtoms (d.p.o.), using a cutoff of 7 days ( 7 days, 7–14 days, >14 days). Agreement between techniques was evaluated using the Cohen's kappa score and the McNemar's test. For these comparisons, a p value less than or equal to 0.05 was considered significant. Statistical analysis was performed using Stata/IC 13.1 (StataCorp).

RESULTS

Diagnostic performance results according C t values are summarized in Table 1 and Figure S1. Specificity was 100% for Panbio and 97.3% for SD‐Biosensor, due to five PCR negative samples that were positive for this assay. Overall sensitivity was 60.0% for Panbio and 66.5% for SD‐Biosensor and this difference was statistically significant (p =  0.003). Sensitivity was higher for samples with high viral loads, reaching 100.0% for samples with C t   20 for both tests. For samples with C t = 20–25, sensitivity was 93.0% for Panbio and 95.3% for SD‐Biosensor (p = 1.000). Sensitivity significantly decreased for low viral load samples: for C t values between 25 and 30, sensitivity was 41.3% for Panbio and 52.2% for SD‐Biosensor (p = 0.125), and for C t values over 30, sensitivity was 5.0% for Panbio and 17.5% for SD‐Biosensor (p = 0.063). There were no significant differences on sensitivity results between Ag‐RDTs according C t values (Table 1).

Table 1

Diagnostic performance of the evaluated Ag‐RDTs according to viral load

Type of sample	C t values	Panbio		SD‐Biosensor		p value
PCR negative	N/A	Positive samples	0/186	Positive samples	5/186	0.063
		Specificity	100.0	Specificity	97.3
(n = 186)
					(93.8–99.1)
			(98.0 – 100.0)
PCR positive	25.2 (20.2– 29.7)	Positive samples	102/170	Positive samples	113/170	0.003
		Sensitivity	60.0	Sensitivity	66.5
(n = 170)			(52.2 – 67.4)		(58.8–73.5)
High viral load samples
C t ≤20	17.8 (16.8– 18.5)	Positive samples	41/41	Positive samples	41/41	1.000
		Sensitivity	100.0	Sensitivity	100.0
(n = 41)			(91.4 – 100.0)
					(91.4–100.0)
C t = 20–25	22.1 (20.8– 23.6)	Positive samples	40/43	Positive samples	41/43	1.000
		Sensitivity	93.0	Sensitivity	95.3
(n = 43)	(80.9 – 98.5)		(84.2–99.4)
Low viral load samples
C t = 25–30	27.2 (26.0– 28.7)	Positive samples	19/46	Positive samples	24/46	0.125
		Sensitivity	41.3	Sensitivity	52.2
(n = 46)			(27.0 – 56.8)		(36.9–67.1)
C t > 30	31.3 (30.6 – 33.3)	Positive samples	2/40	Positive samples	7/40	0.063
(n = 40)		Sensitivity	5.0	Sensitivity	17.5
					(7.3–32.8)
			(0.6 – 16.9)

Note: Statistics: values are expressed as absolute count (percentage) and median (interquartile range). Sensitivity and specificity results are expressed as percentage with 95% CI. P‐values were calculated by the McNemar's test. Significant differences are shown in bold.

Abbreviations: Ag‐RDT, antigen rapid diagnostic test; C t, cycle threshold; N/A, not applicable; 95% CI, 95% confidence interval; p‐value, level of significance.

Regarding symptoms of those 170 PCR positive patients, information regarding symptoms was unavailable for 10 patients. For the remaining 160 cases, 134 (83.7%) patients presented COVID‐19 symptoms and 26 (16.3%) were asymptomatic. Information about time from the onset of symptoms to sample obtention was unavailable for another six symptomatic patients. Table 2 and Figure S2 summarize the diagnostic performance results according to the time from the onset of symptoms. Both Ag‐RDTs showed high values of sensitivity in samples taken within the first seven d.p.o. (87.7% for Panbio, and 90.4% for SD‐Biosensor, p = 0.625). Sensitivity decreased significantly from the eighth d.p.o., reaching a sensitivity of 31.0% for Panbio and 38.1% for SD‐Biosensor for day 8%–14% and 30.8% for Panbio and 38.5% for SD‐Biosensor from 14 days. There were no significant differences on sensitivity results between Ag‐RDTs according d.p.o. (Table 2).

Table 2

Diagnostic performance of the evaluated Ag‐RDTs according to time form the onset of symptoms

Time from the onset of symptoms	Days	Panbio		SD‐Biosensor		p value
≤7 days	4 (2 – 6)	Positive samples	64/73	Positive samples	66/73	0.625
		Sensitivity	87.7	Sensitivity	90.4
(n = 73)
					(81.2 – 96.1)
			(77.9– 94.2)
8–14 days	9 (8 – 11)	Positive samples	13/42	Positive samples	16/42	0.250
		Sensitivity	31.0	Sensitivity	38.1
(n = 42)			(17.6– 47.1)
					(23.6 – 54.4)
>14 days	16 (15 – 18)	Positive samples	4/13	Positive samples	5/13	1.000
		Sensitivity	30.8	Sensitivity	38.5
(n = 13)
					(13.9 – 68.4)
			(9.1 – 61.4)

Note: Statistics: sensitivity results are expressed as percentage with 95% CI. P‐values were calculated by the McNemar's test. No p‐value was statistically significant (p < 0.05).

Abbreviations: Ag‐RDT, antigen rapid diagnostic test; 95% CI, 95% confidence interval; p‐value, level of significance.

Finally, regarding agreement results, agreement of Ag‐RDTs with PCR was moderate in overall samples (agreement = 80.9%, k = 0.596 for Panbio; 82.6%, k = 0.646 for SD‐Biosensor) but it was excellent for high viral load samples (C t   25) for both Ag‐RDTs (Panbio: 98.9%, k = 0.974, SD‐Biosensor: 97.4%, k = 0.940). Moreover, agreement between Ag‐RDTs was excellent for overall samples (94.9%, k = 0.882) being even better when we focused in high viral load samples (97.8%, k = 0.948).

DISCUSSION

Ag‐RDTs have demonstrated their reliability as diagnostic tools to aid in the control of SARS‐CoV‐2 pandemic. Although Panbio Ag‐RDT has been the most frequently evaluated test,7, 9, 10, 11, 12 the number of commercialized assays is growing exponentially.10, 13, 14, 15, 16, 17 Our results show that Panbio and SD‐Biosensor Ag‐RDTs are reliable to diagnose SARS‐CoV‐2 infection, as they fulfilled the general recommendations for the use of these tests that are recommended by the WHO ≥ 80% sensitivity and ≥97% specificity compared with PCR.5, 18 Moreover, they showed excellent performance within the first seven d.p.o. or when they are performed in samples with high viral load as well as excellent levels of agreement between them. Diagnostic performance of Ag‐RDTs within the first 7 d.p.o. could be attributed to higher viral loads that are observed in these samples, as compared with samples obtained several days later, which is evidenced by observing the increasing trend of the median Cts values according to d.p.o. (see Figure S3).

Some authors have shown that, besides the lower sensitivity of Ag‐RDTs compared with PCR, they improve the turnaround time for results, which is key to interrupt transmission chains to control the spread of this pandemic.3, 4, 18, 19 As a consequence, several diagnostic algorithms already recommend the use of these tests as first step for symptomatic patients within the first 57 days after the onset of symptoms18, 19, 20 and our results support the use of Panbio and SD‐Biosensor tests for that purpose. Some authors have pointed out that Ag‐RDTs could also be reliable for detecting asymptomatic patients with high infectious capacity. These findings would support the use of Ag‐RDTs as screening test for massive population testing. However, more studies are needed to ensure the effectiveness of these tools for that purpose.

Our study presents some limitations: it is a retrospective study that has been conducted in a single institution and we have analyzed the results of two among all commercialized Ag‐RDTs. Consequently, our conclusions should not be extrapolated to other available Ag‐RDTs and more prospective multicenter studies and meta‐analysis are needed to establish the usefulness of other Ag‐RDTs. However, to the best of our knowledge, our work constitutes the first comparative evaluation of Panbio and SD‐Biosensor Ag‐RDTs and our findings indicate that these assays could be reliable tools for the early diagnosis of symptomatic COVID‐19 cases and the control of this pandemic.

CONFLICT OF INTERESTS

The authors declare that there are no conflict of interests.

ETHICS STATEMENT

The study was conducted according to the ethical requirements established by the Declaration of Helsinki. The Ethics Committee of Hospital Universitario Príncipe de Asturias (Madrid) approved the study (Protocol n°: Antigeno‐COVID). Since the present study is retrospective, informed consent was not required.

AUTHOR CONTRIBUTIONS

Study concept and design: Felipe Pérez‐García and Juan Cuadros‐González. Clinical data acquisition and interpretation of data: Juan Romanyk, Inés Pérez Ranz, Andrea Labrador Ballestero, Helena Moya Gutiérrez, Javier Conzález Arroyo, Victoria González Ventosa and Concepción Domingo Cruz. Statistical analysis: Felipe Pérez‐García. Drafting of the manuscript: Felipe Pérez‐García and Juan Cuadros‐González. Supervision, critical revision of the manuscript for relevant intellectual content: Juan Cuadros‐González. All authors read and approved the final manuscript.

Supporting information.

Click here for additional data file.

Supporting information.

Click here for additional data file.

Supporting information.

Click here for additional data file.

Supporting information.

Click here for additional data file.

Supporting information.

Click here for additional data file.