Clinical performance of the Panbio assay for the detection of SARS-CoV-2 IgM and IgG in COVID-19 patients.

Following the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic, numerous serological tests have been developed, including rapid diagnostic tests. This study aims at assessing the clinical performance of the Panbio immunoglobulin G (IgG)/IgM coronavirus disease 2019 (COVID-19) test (Abbott), a rapid lateral flow assay for the qualitative detection of IgG and IgM against SARS-CoV-2. One hundred and thirty-eight samples from 95 COVID-19 patients with a positive SARS-CoV-2 reverse-transcriptase polymerase chain reaction were analyzed to assess the clinical sensitivity. Seventy-six pre-COVID-19 samples were used to evaluate the clinical specificity. Two independent and blinded raters determined visually the presence or absence of the IgG, IgM, and control lines for each test after 10 and 20 min. The sensitivity obtained from collected samples more than 14 days after the onset of symptoms was 95.2% for IgG. IgM was less frequently detected (highest sensitivity of 20.5%). The specificities obtained were 98.7% and 100% for IgG and IgM, respectively. In addition, the sensitivity of the assay was better when the reading was performed at 20 min than at 10 min, whereas the specificity was unchanged. The Panbio COVID-19 IgG/IgM rapid test detects IgG with high sensitivity 14 days since symptom onset but presents a low sensitivity for IgM. The specificity was excellent for both IgG and IgM.

INTRODUCTION

Rapid tests are designed for use where a preliminary screening test result is required and are especially useful in resource‐limited countries or for broad screening campaigns, where access to blood sampling may be difficult or not obligatory. However, these tests have to be of high quality, user‐friendly, quick, and easy to perform, and they have to require little or no additional equipment. In the context of COVID‐19, all the above‐mentioned criteria are of importance as serological tests may be useful for the diagnosis, for the characterization of the course of the disease, for identifying convalescent plasma donors directly on site, for lockdown exit programs, for epidemiological study, and for the assessment of COVID‐19 vaccine response. Due to their widespread dissemination and limited experience with these assays, it is crucial for laboratories to rigorously validate these methods before a broad introduction into routine clinical practice. This study aims at evaluating the clinical performances of the Panbio COVID‐19 IgM/IgG rapid test (Abbott) in a population of COVID‐19 patients.

MATERIALS AND METHODS

Sample collection

This study was conducted from June 16, 2020 to June 24, 2020. Blood samples were collected from patients into serum‐gel tubes (BD Vacutainer® 8.5 ml tubes, Becton Dickinson) or lithium heparin plasma tubes (BD Vacutainer® 4.0 ml tubes) according to the standardized operating procedure and manufacturer's recommendations. Samples were centrifuged for 10 min at 1885 to 2500g (ACU Modular® Pre Analytics, Roche Diagnostics®). A total of 214 samples were collected from April, 2019 to May 25, 2020, and stored in the laboratory biobank at −20°C. Pre‐COVID‐19 samples (n = 76) were all collected before March 2020, the start of the pandemic in Belgium. One hundred and thirty‐eight samples from 95 COVID‐19 patients were collected between March 21, 2020 and May 25, 2020. Frozen samples were thawed at room temperature. The study fulfilled the ethical principles of the Declaration of Helsinki.

Analytical procedures

The Panbio IgG/IgM COVID‐19 rapid test (Abbott) is a rapid lateral flow assay (LFA) for the qualitative detection of IgG and IgM directed against SARS‐CoV‐2 in human whole blood, serum, or plasma specimens. The Panbio test was performed according to the manufacturer's instructions for use. Briefly, 10 µl of the sample was applied into the specimen well, and then two drops of buffer were applied. Raters determined visually the presence or absence of the IgG, IgM, and control lines for each test 10 and 20 min after the addition of the buffer. As recommended by the manufacturer, even a slightly colored strip was considered positive.

The reverse‐transcriptase polymerase chain reaction (RT‐PCR) for SARS‐CoV‐2 determination in respiratory samples (nasopharyngeal swab samples) was performed on the LightCycler® 480 Instrument II (Roche Diagnostics®) using the LightMix® Modular SARS‐CoV E‐gene set.

Assessment of the clinical sensitivity

Samples (n = 138) obtained from 95 patients with a confirmed RT‐PCR SARS‐CoV‐2 diagnosis were assessed to determine the clinical sensitivity of the assay. Sensitivity was defined as the proportion of correctly identified COVID‐19‐positive patients since symptom onset. Antibody kinetics was evaluated using all the samples divided into different categories based on the number of days after the symptom onset, as follows: 0–2 days (n = 15), 3–5 days (n = 6), 6–8 days (n = 14), 9–11 days (n = 9), 12–14 days (n = 11), 15–17 days (n = 13), 18–21 days (n = 13), 22–25 days (n = 15), 26–31 days (n = 13), 32–40 days (n = 12), and more than 40 days (n = 17).

Assessment of the clinical specificity

Non‐SARS‐CoV‐2 samples (n = 76) collected before the COVID‐19 pandemic (between April and June 2019) with potential cross‐reactions (n = 38) were also analyzed to assess the specificity. Samples included positive antinuclear antibodies (n = 4), anti‐thyroglobulin antibody (n = 1), anti‐Treponema pallidum antibodies (n = 1), anti‐thyroid peroxidase antibodies (n = 3), direct coombs (n = 1), hepatitis B Ag (n = 3), IgA Chlamydia pneumoniae (n = 1), IgG Chlamydia trachomatis (n = 1), IgM Borrelia burgdorferi (n = 1), IgM Cytomegalovirus (n = 4), IgM Mycoplasma pneumoniae (n = 1), IgM Parvovirus B19 (n = 1), IgM Toxoplasma gondii (n = 6), IgG polyclonal activation (n = 1), IgM and IgG polyclonal activation (n = 1), search for irregular agglutinins (n = 5), rheumatoid factor (n = 1), urinary tract infection with Escherichia coli (n = 1), urinary tract infection with Klebsiella oxytoca (n = 1), and samples from 38 healthy volunteers were included for the specificity calculation. Specificity was defined as the proportion of naïve patients classified as negative.

Evaluation of reading conditions

Two independent and blinded raters determined visually the presence or absence of the IgG, IgM, and control lines for each test after 10 and 20 min. In case of discrepancies, a third blinded and independent rater checked the presence or absence of the lines. Consensus results between all raters were used. The intrarater (10 min vs. 20 min) and the interrater (Rater 1 vs. Rater 2) concordances were determined.

Statistical analysis

Data analysis was performed using GraphPad Prism® software (version 8.2.1) and MedCalc® software (version 14.8.1). Confidence intervals for sensitivity and specificity were "exact" Clopper–Pearson confidence intervals. The Cohen's κ coefficient was used to assess the intra‐ and interrater concordance.

RESULTS

Clinical performances

All the tests (n = 214) were valid (i.e., the control line was visible). Kinetics of the sensitivity of the Panbio assay to detect IgG and IgM since the onset of the first symptoms is described in the Figure 1. After 14 days since symptom onset, the Panbio assay detected IgG in 95.2% (95% confidence interval [CI]: 88.1%–98.7%). Before 14 days since the first symptoms, sensitivities were not high enough to be reliably used in clinical practice (50.9%, 95% confidence interval [CI]: 37.1%–64.7%).

Figure 1

Kinetics of the sensitivity of the Panbio assay since the onset of symptoms. (A) Kinetics of the sensitivity of the Panbio assay since the onset of first symptoms detects IgG (blue dots), IgM (green triangle), and IgG and/or IgM (yellow squares). The result of each test was determined visually after 20 min by two independent and blinded operators. (B) Sensitivities of the Panbio assay for IgG, IgM, and IgG and/or IgM since the onset of first symptoms. IgG, immunoglobulin G; IgM, immunoglobulin M

Immunoglobulin M was less frequently detected by the Panbio assay, with sensitivities of 7.3% (95% CI: 2.0%–17.6%) and 20.5% (95% CI: 12.4%–30.8%) for samples the first 14 days and for those obtained more than 14 days since symptom onset, respectively. The highest sensitivity for IgM obtained in a particular category based on the number of days after the symptom onset was 30.8% (95% CI: 9.1%–61.4%) (Figure 1).

Only one sample was positive for IgM and negative for IgG. This sample was collected 22 days after the first symptoms. The sensitivity of the Panbio assay to detect IgM and/or IgG within the first 14 days since symptom onset was unchanged compared with the sensitivity to detect IgG (50.9%; 95% CI: 37.1%–64.7%). After 14 days since symptom onset, the Panbio assay detected IgG and/or IgM in 96.4% (95% CI: 89.8%–99.3%) of samples.

Among the 76 samples collected before the COVID‐19 pandemic, only one sample from a healthy volunteer gave a false positive result with IgG. Samples with potential cross‐reaction gave no false‐positive result. The specificity was 98.7% (95% CI: 92.9%–100.0%) and 100% for IgG and IgM, respectively.

The inter‐rater variability was excellent when the tests were read at 10 min and 20 min for both IgG (Cohen's κ coefficient at 10 and 20 min were 0.972 and 0.991, respectively) and IgM (Cohen's κ coefficient at 10 and 20 min were 0.945 and 0.974). In addition, the sensitivity of the assay was better when the reading was performed at 20 min than at 10 min (Table 1), whereas the specificity was unchanged. Cohen's κ coefficients for the different time of reading were lower for IgM than IgG, indicating that the time of reading influence more IgM results than IgG (Table 1). The positive lines (IgM and IgG) read at 10 min were always positive at 20 min.

Table 1

Evaluation of the impact of the rater and the time of reading on the IgG (A) and IgM (B) test results

	A. Number of samples read positive for IgG/total number of samples		κ coefficient between reading time
	Reading after 10 min	Reading after 20 min
Rater 1	105/138 (76.1%)	106/138 (76.8%)	0.991
Rater 2	106/138 (76.8%)	107/138 (77.5%)	0.991
κ coefficient between raters	0.972	0.991

Abbreviations: IgG, immunoglobulin G; IgM immunoglobulin M.

DISCUSSION

The detection of anti‐SARS‐CoV‐2 antibodies represents an additional method for the diagnosis of COVID‐19, which may significantly improve the sensitivity of pathogenic diagnosis for COVID‐19 when combined with RT‐PCR. A wide range of assays has been developed, including ELISA, chemiluminescent immunoassay (CLIA), electrochemiluminescence immunoassay (ECLIA), and rapid tests. , , , , , The main advantage of rapid diagnostic tests is that they do not require specific equipment and are easy to use. Furthermore, these tests are rapid, and they can be easily implemented in a low‐resource laboratory.

The World Health Organization (WHO) encourages laboratories to perform independent assay validation, in particular regarding the clinical utilization of rapid device. Based on the conclusions of the study of the Frederick National Laboratory for Cancer Research (FNLCR), a Federally Funded Research and Development Center (FFRDC) sponsored by the National Cancer Institute (NCI), the FDA concluded that a list of 65 serological assays should not be distributed. External validations of these tests are therefore paramount, and plenty of data are arriving in the literature. , , , , , , , , , , Given the leading position of Abbott for COVID‐19 testing, independent external validation of their assays is mandatory to ensure that the performance is in line with their claims.

In our evaluation, the sensitivity obtained for all samples collected more than 14 days after the onset of symptoms was 95.2% for IgG. The Panbio assay showed weak sensitivity for IgM (Figure 1). The specificities obtained were 98.7% and 100% and for IgG and IgM, respectively. In the instructions for use, Abbott Diagnostics mentioned a sensitivity and a specificity of 95.8% and 94.0%, respectively. In the manufacturer's study, 48 samples of PCR‐confirmed patients and 50 pre‐COVID‐19 samples were analyzed. Taken apart, IgG had a sensitivity and a specificity of 95.8% and 100%, and IgM a sensitivity and a specificity of 56.3% and 94%. Our results are in agreement with these claims and we even obtained a better specificity for IgM, although the sensitivity was lower than claimed. However, in the information provided by the manufacturer, the details of the studied populations were lacking, that is, timing between symptom onset or since PCR positivity and the blood sampling, as well as the characteristics of samples included for specificity calculation.

As observed on other assays and platforms, that is, LFA, ELISA, CLIA, ECLIA, , , , we found that sensitivities before 14 days since symptom onset were not sufficient to be reliably used in clinical practice. We, therefore, recommend obtaining a control or confirmatory sample after 14 days to increase the detection rate.

Comparing the clinical performance of these rapid tests is hazardous. Indeed, the design of studies varies widely across studies, that is, number of positive and negative samples, the definition of negative samples, number of days since symptoms or since PCR positivity, and comparison to a neutralization test. Some studies included only a very limited number of patients, included control samples collected during the pandemic period, , defined different categories since symptom onset (i.e., < or > 7 days, 0–6, 7–13, 14–25 days, or 5–9, 10–18 days ), or different categories since RT‐PCR positivity. Moreover, as with other rapid LFA, we showed that the result may depend on the reader and on the timing of reading (20 min better than 10 min). The utilization of an automated reader may be useful to decrease the interindividual variation, especially when the colored stripe appears very thin.

CONCLUSIONS

The Panbio COVID‐19 IgM/IgG rapid test presents high sensitivities for IgG 14 days since symptom onset but very low sensitivity for IgM. The specificity was excellent for both IgG and IgM. Further investigations designed to evaluate the clinical performances of Panbio over a longer period of time are needed to further consider its use in seroprevalence studies.

CONFLICT OF INTERESTS

Among the authors, Jonathan Douxfils is the chief executive officer and founder of Qualiblood sa, and reports personal fees from Diagnostica Stago, Roche, Roche Diagnostics, Daiichi‐Sankyo, and Portola, outside the submitted work. The remaining authors declare that there are no conflict of interests.

AUTHOR CONTRIBUTIONS

Jonathan Douxfils, Julien Favresse, and Hélène Haguet and were responsible for the conception and design of the study. Jonathan Douxfils, Julien Favresse, and Hélène Haguet were responsible for the acquisition, analysis, and interpretation of data. Jonathan Douxfils, Julien Favresse, and Hélène Haguet were responsible for drafting the manuscript. Christine Eucher, Marc Elsen, Julie Cadrobbi, Marie Tré‐Hardy, and Jean‐Michel Dogné contributed to the final draft of the manuscript. All authors agree to be accountable for the content of the work.

PEER REVIEW

The peer review history for this article is available at https://publons.com/publon/10.1002/jmv.26884