Field evaluation of a rapid antigen test (Panbio™ COVID-19 Ag Rapid Test Device) for COVID-19 diagnosis in primary healthcare centres.

OBJECTIVES: To our knowledge no previous study has assessed the performance of a rapid antigen diagnostic immunoassay (RAD) conducted at the point of care (POC). We evaluated the Panbio™ COVID-19 Ag Rapid Test Device for diagnosis of coronavirus 2019 disease (COVID-19) in symptomatic patients (n = 412) attending primary healthcare centres.
METHODS: RAD was performed immediately after sampling following the manufacturer's instructions (reading at 15 min). RT-PCRs were carried out within 24 h of specimen collection. Samples displaying discordant results were processed for culture in Vero E6 cells. Presence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in cell cultures was confirmed by RT-PCR.
RESULTS: Out of 412 patients, 43 (10.4%) tested positive by RT-PCR and RAD, and 358 (86.9%) tested negative by both methods; discordant results (RT-PCR+/RAD-) were obtained in 11 patients (2.7%). Overall specificity and sensitivity of rapid antigen detection (RAD) was 100% (95%CI 98.7-100%) and 79.6% (95%CI 67.0-88.8%), respectively, taking RT-PCR as the reference. Overall RAD negative predictive value for an estimated prevalence of 5% and 10% was 99% (95%CI 97.4-99.6%) and 97.9% (95%CI 95.9-98.9), respectively. SARS-CoV-2 could not be cultured from specimens yielding RT-PCR+/RAD- results (n = 11).
CONCLUSION: The Panbio™ COVID-19 Ag Rapid Test Device performed well as a POC test for early diagnosis of COVID-19 in primary healthcare centres. More crucially, the data suggested that patients with RT-PCR-proven COVID-19 testing negative by RAD are unlikely to be infectious.

Introduction

Rapid detection, effective isolation of symptomatic cases, and systematic tracing of close contacts are paramount to blunt the community spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection. Nowadays, reverse-transcriptase polymerase chain reaction (RT-PCR) is the diagnostic reference standard for coronavirus disease 2019 (COVID-19) [1]; however, specialized instruments and expertise are required to conduct RT-PCR assays. In addition, many countries have encountered supply shortages of RT-PCR reagents. Rapid antigen detection immunoassays (RAD) are particularly suited for point-of-care testing (POCT), as they can easily be performed and interpreted without equipment, are inexpensive, and improve turnaround times. Moreover, results returned by a recently launched antigen assay appeared to correlate better with patient infectiousness than RT-PCR results [2]. In this field study, we evaluated the Panbio™ COVID-19 Ag Rapid Test Device (Abbott Diagnostic GmbH, Jena, Germany), a lateral flow immunochromatographic assay targeting SARS-CoV-2 nucleoprotein in nasopharyngeal specimens (NPs) for the diagnosis of COVID-19 in symptomatic patients attending primary healthcare centres.

Material and methods

Patients

Between 2nd September and 7th October 2020 this prospective study enrolled 412 patients with clinical suspicion of COVID-19 (median age 31 years, range 1–91 years, 58% female), of whom 327 were adults (median age 36 years, range 17–91 years) and 85 children (≤16 years old, median 11 years, range 1–16 years), attending primary care centres of the Clínico-Malvarrosa Health Department in Valencia (Spain). Only patients with compatible signs or symptoms appearing within the prior week were recruited. The study was approved by the Hospital Clínico de Valencia (HCU) INCLIVA Research Ethics Committee.

SARS-CoV-2 testing

Using flocked swabs, and following appropriate safety precautions, trained nurses at each participating centre collected two NPs per patient, one of which (provided by the manufacturer) was used for RAD while the other was placed in 3 mL of universal transport medium (UTM, Becton Dickinson, Sparks, MD, USA) and delivered to the HCU Microbiology Service for RT-PCR testing. RAD assay was performed immediately after sampling following the manufacturer's instructions (reading at 15 min). RT-PCRs were carried out within 24 h of specimen collection with the TaqPath COVID-19 Combo Kit (Thermo Fisher Scientific, MS, USA) which targets SARS-CoV-2 ORF1ab, N and S genes. RNA was extracted using the Applied Biosystems™ MagMAX™ Viral/Pathogen II Nucleic Acid Isolation Kits coupled with Thermo Scientific™ KingFisher Flex automated instrument. The AMPLIRUN® TOTAL SARS-CoV-2 Control (Vircell SA, Granada, Spain) was used as the reference material for SARS-CoV-2 RNA load quantification.

SARS-CoV-2 cell culture

Samples collected in UTM were stored at –80°C for up to 2 weeks before being processed for culture in Vero E6 cells. Presence of SARS-CoV-2 was confirmed by RT-PCR (see footnote to Supplementary Material Table S1).

Statistical analyses

Agreement between RAD assay and RT-PCR was assessed using Cohen's κ statistics. Differences between medians were compared using the Mann–Whitney U-test. Receiver operating characteristic (ROC) curves were built to determine SARS-CoV-2 RT-PCR cycle threshold (CT) and RNA loads best discriminating between RT-PCR+/RAD+ and RAD– specimens. Two-sided p values < 0.05 were considered significant. Statistical analyses were performed using SPSS version 25.0 (SPSS, Chicago, IL, USA).

Results

Out of 412 patients, 43 (10.4%) tested positive by RT-PCR and RAD, and 358 (86.9%) tested negative by both methods, showing discordant results (RT-PCR+/RAD–) in 11 patients (2.7%) (Supplementary Material Table S1). Concordance between the two methods was good (κ 0.87, 95%CI 0.79–0.94). Overall specificity and sensitivity of RAD was 100% (95%CI 98.7–100%) and 79.6% (95%CI 67.0–88.8%) respectively. Sensitivity slightly increased (80.4%, 95%CI 66.8–89.3%) in patients with clinical courses of <5 days (Fig. 1 A). Sensitivity was higher in adults (82.6%, 95%CI 69.3–90.9%) than in paediatric patients (62.5%, 95%CI 30.6–86.3%) (Supplementary Material Table S2).

Fig. 1

(A) Field performance of the Panbio™ COVID-19 Ag Rapid Test Device (RAD) according to time between symptoms onset and testing (

(A) Field performance of the Panbio™ COVID-19 Ag Rapid Test Device (RAD) according to time between symptoms onset and testing (patients with clinical suspicion of COVID-19 attending primary healthcare centres. (B) RT-PCR CT values in specimens testing either RAD + or RAD–. (C) Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) RNA loads in specimens testing either RAD + or RAD–. The AMPLIRUN® TOTAL SARS-CoV-2 Control (Vircell SA, Granada, Spain) was used as the reference material for SARS-CoV-2 RNA load quantification (in copies/mL, considering RT-PCR CTs for the N gene; the linear regression equation was: Y = –0.31 × X + 13.77; R2 = 9.89). Median and p values are shown.

Overall RAD negative predictive value for an estimated prevalence of 5% and 10% (the incidence of COVID-19 in our Health Department during the study period was within that range) was 99% (95%CI 97.4–99.6%) and 97.9% (95%CI 95.9–98.9), respectively.

RT-PCR CT values were significantly higher and SARS-COV-2 RNA loads significantly lower (p < 0.001) in RT-PCR+/RAD– than in RT-PCR+/RAD + specimens (Fig. 1B,C). ROC curve analyses indicated that RT-PCR CT < 25 and SARS-CoV-2 RNA loads >5.9 log10 copies/mL thresholds best discriminated between RT-PCR+/RAD+ and RT-PCR+/RAD– specimens, with a sensitivity and specificity of 100%. As expected, the overall RAD sensitivity was directly dependent upon the RT-PCR CT values (SARS-CoV-2 RNA loads) (Supplementary Material Table S3).

The time from symptoms onset to sampling did not differ (p 0.86) between RT-PCR+/RAD + patients (median 3 days, range 1–7 days) and RT-PCR+/RAD– patients (median 2 days, range 1–6 days).

All 11 specimens yielding discordant RT-PCR/RAD results tested negative by culture, whereas SARS-CoV-2 could be recovered from all three specimens returning RT-PCR+/RAD + results (CT 4, 14 and 16).

Discussion

Previous studies evaluating SARS-CoV-2 RAD tests used either retrieved specimens, which had been cryopreserved a varying number of times, or ones conducted at central laboratories, or both [[3], [4], [5], [6], [7]]. To our knowledge this is the first report on the performance of a RAD assay conducted at POC. Compared to RT-PCR, the Panbio™ COVID-19 Ag Rapid Test Device assay yielded an excellent specificity and a fairly good overall sensitivity (79.6%, 95%CI 67.0–88.8%), the latter slightly improved when time to testing was less than 5 days from the onset of symptoms (80.6%, 95%CI 66.8–89.3%). This figure is less impressive than is claimed by the manufacturer (93%). Based upon our findings (Supplementary Material Table S2), the sample panel evaluated by the manufacturer might have included a large fraction of specimens displaying high viral loads. Nevertheless, the overall sensitivity for the RAD assay reported herein was much closer to that found by Linares and colleagues (86.5%) [3]. Sensitivity of SARS-CoV-2 RAD assays has been reported to vary between 45% and 97% [[3], [4], [5], [6], [7]], yet direct comparison between studies is hampered by marked dissimilarities in patient clinical characteristics and age, testing sites, type of specimen processed, and time to testing, among others.

Interestingly, sensitivity was higher in adults than in paediatric patients. Previous studies found no age-related differences in SARS-CoV-2 RNA load in the upper respiratory tract [8]. Although speculative, dating of symptoms onset could have been more inaccurate in children than in adults.

In a setting like ours, with an incidence of COVID-19 ranging between 5% and 10% at the time of study, the RAD negative predictive value was 99% (95%CI 97.4–99.6%) and 97.9% (95%CI 95.9–98.9), respectively.

Out of 54 RT-PCR-positive specimens, 11 tested negative by RAD. In line with previous reports [[2], [3], [4]], SARS-CoV-2 RNA load was significantly higher in RT-PCR+/RAD + specimens than in RT-PCR+/RAD– samples. In our setting, specimens with RT-PCR CT > 25 (equivalent to SARS-CoV-2 RNA loads <5.9 log10 copies/mL) returned discordant RAD/RT-PCR results.

An important observation of our study was that SARS-CoV-2 could not be cultured from RT-PCR+ (CT > 25)/RAD– specimens. Along these lines, Pekosz and colleagues [2] found one out of 27 RAD–/culture-positive specimens, using a highly sensitive cell culture system (VeroE6 TMPRSS2). The SARS-CoV-2 RNA load threshold associated with culture positivity herein (>5.9 log10 copies/mL) was remarkably close to other previously published results (around 106 copies/mL) [2,[9], [10], [11]]. Other studies have reported positive cultures from upper respiratory tract specimens displaying RT-PCR CT values > 30 or even higher [10]. In this sense, it should be stressed that CT values returned by different RT-PCR assays for a given specimen, even among those targeting the same gene region, may vary substantially [2]. In line with this, the lowest genome copy number that allowed SARS-CoV-2 recovery in cell culture in Huang's study [10] was 5.7 log10/mL, compared with 5.9 log10/mL in ours, in both the N gene being taken as the reference for SARS-CoV-2 RNA quantification.

The main limitation of the current study is the relatively low number of cases in the series (13%); however, this could be viewed as a strength, as this figure likely represents the reality in many community settings worldwide where RAD testing is increasingly being used.

In summary, we found the Panbio™ COVID-19 Ag Rapid Test Device to perform well as a POCT for early diagnosis of COVID-19 in primary healthcare centres. More crucially, our data suggest that patients with RT-PCR-proven COVID-19 testing negative by RAD are unlikely to be infectious. Given that false-negative RAD results may be inconsequential from a public health perspective [12], a laboratory diagnostic approach that skipped RT-PCR confirmation of negative RAD tests in non-hospitalized patients would certainly alleviate laboratory workloads while RT-PCR tests are in short supply.

Author contributions

EA, IT, FB, DH, MM, SP, LF, AV, CSdA, JP and JC: methodology (RT-PCR and RAD) and data validation. EM and MA F–F: methodology (cell culture) and data validation. EA, IT: formal analysis. DN: conceptualization, supervision, writing the original draft. All authors reviewed the original draft.

Transparency declaration

The authors declare that they have no conflicts of interest. This work received no public or private funds. Our group has received private funds for research projects from (Spain), Biomerieux Inc. (France), (Spain) and (Spain) within the last 3 years.