Multicenter evaluation of the Panbio™ COVID-19 rapid antigen-detection test for the diagnosis of SARS-CoV-2 infection.

OBJECTIVES: The standard RT-PCR assay for coronavirus disease 2019 (COVID-19) is laborious and time-consuming, limiting testing availability. Rapid antigen-detection tests are faster and less expensive; however, the reliability of these tests must be validated before they can be used widely. The objective of this study was to determine the performance of the Panbio™ COVID-19 Ag Rapid Test Device (PanbioRT) (Abbott) in detecting severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in nasopharyngeal swab specimens.
METHODS: This prospective multicentre study was carried out in ten Spanish university hospitals and included individuals with clinical symptoms or epidemiological criteria of COVID-19. Only individuals with ≤7 days from the onset of symptoms or from exposure to a confirmed case of COVID-19 were included. Two nasopharyngeal samples were taken to perform the PanbioRT as a point-of-care test and a diagnostic RT-PCR test.
RESULTS: Among the 958 patients studied, 325 (90.5%) had true-positive results. The overall sensitivity and specificity for the PanbioRT were 90.5% (95%CI 87.5-93.6) and 98.8% (95%CI 98-99.7), respectively. Sensitivity in participants who had a threshold cycle (CT) < 25 for the RT-PCR test was 99.5% (95%CI 98.4-100), and in participants with ≤5 days of the clinical course it was 91.8% (95%CI 88.8-94.8). Agreement between techniques was 95.7% (κ score 0.90; 95%CI 0.88-0.93).
CONCLUSIONS: The PanbioRT performs well clinically, with even more reliable results for patients with a shorter clinical course of the disease or a higher viral load. The results must be interpreted based on the local epidemiological context.

Introduction

There are ongoing efforts to develop fast, reliable, inexpensive diagnostic tests specific for the detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) antigens. Rapid antigen-detection tests (RADTs), for both laboratory and near-patient use, detect SARS-CoV-2 proteins produced by replicating viruses in respiratory secretions [1]. Currently, there are already multiple CE-marked commercial RADTs, but very few have been independently evaluated and compared with RT-PCR using different swabs and strictly following manufacturers' instructions [2].

The Panbio™ coronavirus disease 2019 (COVID-19) Ag Rapid Test Device (PanbioRT) (Abbott Diagnostic GmbH, Jena, Germany) is a rapid in vitro test for the qualitative detection of the viral nucleocapsid protein in nasopharyngeal swab specimens from individuals with clinical or epidemiological suspicion of COVID-19. This is a lateral-flow-format test that uses immunochromatography with colloidal gold, and it is designed to be performed at the patient care site by trained healthcare personnel, yielding results in 15–20 min. The objective of this study is to determine the performance of the PanbioRT test with CE marking.

Materials and methods

Participating hospitals and individuals

Between September and October 2020 we carried out an independent, prospective, multicentre diagnostic evaluation study across ten independent university hospitals in two Spanish autonomous communities (Madrid and the Basque Country). The hospitals participating in the study from Madrid were Hospital Clínico Universitario San Carlos, Hospital Universitario Ramón y Cajal, Hospital Universitario La Paz, Hospital Universitario Doce de Octubre, and Hospital Universitario Gregorio Marañón. Hospitals from the Basque Country were Hospital Universitario Araba, Hospital Universitario Cruces, Hospital Universitario Basurto, Hospital Universitario Donostia, and Hospital Universitario Galdakao-Usansolo.

Individuals with clinical symptoms or epidemiological criteria (asymptomatic close contacts) of COVID-19 in whom a diagnostic RT-PCR test was indicated were offered participation in this study. All participants were reported as part of the study and verbal informed consent was obtained prior to their inclusion; in the case of children, parents were informed and their permission was requested. Twelve individuals refused to participate. The result of the PanbioRT did not influence the clinical management of the patients, which was decided based on the RT-PCR result. The symptoms, number of days since the onset of symptoms or exposure, threshold cycle (C ) values for PCR, and demographic data were collected for all participants. Participants' data were coded, and no samples were stored after the PanbioRT was performed. Only individuals with ≤7 days from the onset of symptoms or from exposure to a confirmed case of COVID-19 were included, according to recommendations of the WHO [1]. The study was presented to the Research Ethics Committee of the Hospital Clínico Universitario San Carlos, which responded favourably (internal code 20-017).

SARS-CoV-2 testing

Two nasopharyngeal samples were taken per patient, one in each choana. One of them was taken with the swab provided by PanbioRT, and the other one with a swab suitable for taking a virus sample including a universal transport medium for RT-PCR (Copan flocked swabs with UTM™, Universal Transport Medium). PanbioRT was performed immediately—under point-of-care conditions (regulations of quality systems ISO 15189) [3,4] and according to the manufacturer's instructions (lot numbers 41ADFO11A, 41ADFO12A and 0255648)—by physicians and nurses from emergency services trained by microbiology specialists. The second swab was used for a molecular diagnostic (RT-PCR) by each hospital according to its standard procedures for COVID-19 diagnosis. The commercial RT-PCR methods used for the participating hospitals in this study were the TaqManTM 2019-nCoV assay (Applied Biosystems, Pleasanton, CA, USA), Allplex™ 2019-nCoV Assay (Seegene, Seoul, South Korea), GENOMICA S.A.U. (Madrid, Spain), SARS-COV-2 Real Time PCR KIT (Vircell, Granada, Spain), TaqPath COVID-19 Combo Kit (Thermo Fisher, Waltham, MA, USA), Viasure Real Time PCR (CerTest Biotec, Zaragoza, Spain) and GeneXpert (Cepheid, Sunnyvale, CA, USA). Due to the high diagnostic demand, in some hospitals different PCR techniques were used.

Statistical analyses

Specificity and sensitivity, with 95% confidence intervals (95%CIs), of PanbioRT were calculated using the RT-PCR results as the standard, or which is the same as the proportion of negative and positive agreement, respectively. Sensitivity was calculated for all patients and for specific groups of patients according to the time of onset of symptoms or exposure, RT-PCR C values and symptoms, and age. The level of agreement between the tests was evaluated using Cohen's κ score [5]. Statistical analyses were performed using GraphPad Prism software v.7.02 (GraphPad Software Inc., San Diego, CA, USA).

Results

A total of 958 individuals who had at least one symptom compatible with COVID-19 (n = 830) or who had been in close contact with a diagnosed COVID-19 patient (n = 128) were included in this study. There were between 8 and 245 individuals from each participating hospital with a median age of 40 years (interquartile range 32); 61.3% were women (Table 1 ) and 58 cases were paediatric patients (≤14 years old).

Table 1

Study cohort included in the validation study of the Panbio™ COVID-19 Ag Rapid Test Device

Total N (valid PCR results)	958
Positive PCR (% (n))	37.5% (359)
Age (median (interquartile range))	40 (32)
Gender (% F, (n/N))	61.3% (587/958)
Symptoms present (% yes, (n/N))	86.6% (830/958)
Days from symptom onset or from exposure (mean (N))	2.8 (958)
Days ≤5 (n/N (%))	854/958 (89.1%)
Days 6–7 (n/N (%))	104/958 (10.9%)
PCR CT (n)	297
CT ≥ 25 (n (%))	112 (37.7%)
CT < 25 (n (%))	185 (62.3%)

C, threshold cycle.

Among these 958 patients, RT-PCR was positive in 359 (37.5%) and negative in 599 (62.5%). PanbioRT was positive in 332 (34.7%) and negative in 626 (65.3%) (Table 2 ).

Table 2

Summary of the results of the Panbio™ COVID-19 Ag Rapid Test Device compared to RT-PCR

RT-PCR	PanbioTM COVID-19 Ag Rapid Test Device
	Positive	Negative	TOTAL
Positive	325	34	359
Negative	7	592	599
TOTAL	332	626	958

The agreement between the two methods was 95.7% (κ score 0.90; 95%CI 0.88–0.93). In 41 patients the results differed between the two tests; 34 of them were positive with the RT-PCR test but negative with the PanbioRT (false negatives, 3.5% of the total cases); the remaining seven discrepancies were false positives of PanbioRT (Table 2, Supplementary Material Table S1). All 34 false negatives were in symptomatic participants, eight (23.5%) of these at 6–7 days since the onset symptoms, and 33 (97.1%) had C  ≥ 25 values for RT-PCR (Supplementary Material Table S1). The seven false positives were also in symptomatic patients at 1–7 days from the onset of symptoms (Supplementary Material Table S1).

Based on these data, the overall sensitivity and specificity of the PanbioRT were 90.5% (95%CI 87.5–93.6) and 98.8% (95%CI 98–99.7), respectively (Table 3 ). Sensitivity ranged between hospitals from 67% to 100%, and was >80% in nine of the ten participating hospitals. The hospital with sensitivity of 67% had only six positive cases by RT-PCR, three of them with CT ≥ 25, of which two were negative by PanbioRT. Sensitivity was higher in patients who had a C  < 25 for the RT-PCR test (99.5%; 95%CI 98.4–100) (Table 3) than in those with C  ≥ 25 (70.3%; 95%CI 61.7–78-7). Sensitivity was also higher in patients with ≤5 days of the clinical course of the disease (91.8%; 95%CI 88.8–94.8) (Table 3) than in those with 6–7 days of clinical evolution (80.9%; 95%CI 0.69–0.93).

Table 3

Estimation of clinical performance of the Panbio™ COVID-19 Ag Rapid Test Device compared to RT-PCR

Relative sensitivity (95%CI)	90.5% (87.5–93.6)
Sensitivity days ≤5 (95%CI)	91.8% (88.8–94.8)
Sensitivity CT < 25 (95%CI)	99.5% (98.4–100%)
Relative specificity (95%CI)	98.8% (98.0–99.7)
Agreement (κ index; 95%CI)	95.7% (0.9; 0.88–0.93)
Positive predictive value (95%CI)	97.8% (96.3–99.4)
Negative predictive value (95%CI)	94.6% (92.8–96.3)

C, threshold cycle.

Among the 128 asymptomatic participants who had close contact with a COVID-19 patient, there was full concordance in the 31 (24.2%) who were positive by RT-PCR and in the 97 that were negative. Six (10.3%) of the 58 paediatric patients included in the study were positive by RT-PCR and also by the PanbioRT.

The negative predictive value (NPV) and positive predictive value (PPV) in the study cohort, with a high prevalence (37.5%), were 94.6% and 97.8%, respectively (Table 3). Because PPV and NPV can vary depending on prevalence data, both indicators were estimated in scenarios with lower prevalences of 5% and 10%; the results obtained were 79.8% and 89.3%, respectively, for PPV, and 99.5% and 98.9%, respectively, for NPV.

Discussion

In this study, the PanbioRT gave very good clinical performance values, with 90.5% sensitivity and 98.8% specificity; moreover, sensitivity was even improved in patients with ≤5 days of clinical progression of the disease. Sensitivity reached 99.5% in samples with CT < 25, which is probably closer to the limit of infectivity, as previously reported [6].

The strengths of this study include the large study size, the high percentage of positive cases, the inclusion of multiple centres, the prospective nature of the study, and use of the test according to the manufacturer's instructions under point-of-care conditions. A limitation of the study may be the use of different RT-PCR protocols, because the C values can vary slightly between techniques; however, all of them are used for routine diagnosis in participating hospitals, and all of them are validated and widely used worldwide.

As the COVID-19 pandemic continues unabated, the gap between the number of tests that are needed and the testing capacity of laboratories or in primary-care settings increases [7]. RADT tests are simple to perform and interpret by minimally trained health workers at the point of care, they do not require specific equipment, they are less expensive than RT-PCR, and they provide quick results. Reported performance results of early RADTs for COVID-19 diagnosis were poor and precluded their general use [[8], [9], [10], [11]]; however, some new RADTs appear to have substantially improved reliability [6,[12], [13], [14]]. Although these rapid tests show promise for use as part of a larger strategy for COVID-19 diagnosis and control [1], there are insufficient validation studies to support their use in varied patient environments.

In two recent Spanish studies with 412 patients (54 positive by RT-PCR) [6] and 255 patients (60 positive by RT-PCR) [12], the overall sensitivities were 79.3% and 76.3%, respectively; however, in the second study sensitivity was 86.5% in symptomatic patients with ≤7 days from the onset of symptoms [12]. WHO guidelines require that SARS-CoV-2 RADTs demonstrate ≥80% sensitivity and ≥97% specificity compared to the RT-PCR reference assay [1]. Thus, our data support the clinical use of the PanbioRT instead of the RT-PCR test in patients with symptoms of COVID-19 with a short clinical course (≤5–7 days) of the disease. Although the results obtained in asymptomatic patients and children under 14 years of age were good, the number of cases included for these subpopulations was small (128 and 58, respectively), making it inadvisable to conclude general results in that respect. In a pre-published study with frozen samples [15], sensitivity was significantly higher among samples collected in the setting of case identification (92.6%) and contact tracing (94.2%) than in asymptomatic screening (79.5%). Another recent study showed higher sensitivity in symptomatic individuals (85.3%) than in asymptomatic ones (54.5%) [12]. This is consistent with the advice from the WHO against using RADTs for screening asymptomatic individuals in populations with low COVID-19 prevalence [1] due to the potential for higher incidence of false positives.

The performance of an RADT may depend on the epidemiological situation of the population being tested; therefore, how the test is used and how the results are interpreted will depend on local epidemiological factors [1]. In populations with a high prevalence and a high frequency of symptomatic patients, a positive rapid test would be considered confirmatory for infection. However, a negative result would lead to further testing for respiratory pathogens, including an RT-PCR test for COVID-19 if the symptoms were consistent with this disease. In populations with a low prevalence of COVID-19 and more asymptomatic patients, a negative test would be accepted, but a positive test, which is more likely to be false, could require a confirmatory RT-PCR test.

The use of RADT as a diagnostic tool can greatly reduce the testing burden on microbiology laboratories. However, in the primary-care setting, which has also reached saturation in the testing and diagnosis of COVID-19, changes would be required to enable them to perform the rapid test on-site. The ability to perform this test in patient care centres would simplify the process of testing, and provide rapid results to the doctor and the patient, thus improving the decision-making process and reducing pressure on the healthcare providers. However, it is essential to carry out RADTs strictly following the appropriate biosecurity measures.

This study has had an immediate clinical impact, having been used to include the RADT as a valid diagnostic test in the Spanish Strategy for Early Detection, Surveillance, and Control, of COVID-19 (update 25th September 2020) [16].

In conclusion, this study showed that the PanbioRT provides very good clinical performance as a point-of-care test, with even better results for patients with a shorter clinical course of the disease or higher viral load. While this study has had a direct impact on the national diagnostic strategy for COVID-19 in Spain, the results must be interpreted based on the local epidemiological context. The ease and speed of RADT with good clinical performance could help to prevent an overload on healthcare services, as laboratories will have to cope with an increase in respiratory infections during winter.

Author contributions

JO and MC conceived and designed the study. JG, RC, AD, PM, MS, M-DF, GC, J-LD and JO coordinated the study. JG, IM, PM, J-LT, PG, J-CG, NA, SH, AG, FG, PE, M-AS and Spanish PanbioTM COVID-19 validation group performed the experiments. PM and JO wrote the manuscript. All authors have read, edited and approved the final manuscript.

Transparency declaration

RC has participated in educational programmes organized by Abbott. The other authors declare that they have no conflicts of interest. No external funding was received for this work.