A comparative evaluation between the Abbott Panbio™ COVID-19 IgG/IgM rapid test device and Abbott Architect™ SARS CoV-2 IgG assay.

INTRODUCTION: Antibodies to SARS-CoV-2 serve as critical diagnostic markers for determining how broadly the COVID-19 pandemic has spread, confirming patient recovery, monitoring potential long-term effects of infection, and evaluating potential protection from reinfection. As new antibody tests become available, it is important to evaluate their performance and utility. The aim of this study was to compare the performance of the Abbott PanbioTM COVID-19 IgG/IgM Rapid Test Device against the Abbott ArchitectTM SARS CoV-2 IgG Assay for the detection of the COVID-19 IgG antibody.
METHODS: Two panels of specimens were utilized to challenge both antibody tests: (1) a set of 150 prepandemic negative specimens collected in 2014, and (2) a set of 122 specimens from 87 hospitalized COVID-19 patients in the US and UK that were confirmed with a positive SARS-CoV-2 RNA test result.
RESULTS: The ArchitectTM test had a specificity of 100 % and sensitivity of 99.1 % and 93.9 % when excluding or including immunocompromised patients, respectively for specimens collected >14 days post symptom onset or >5 days post-RNA testing. The PanbioTM test had 99.3 % agreement to ArchitectTM. Notably, N = 6 immune-compromised individuals were identified that did not develop detectable antibodies by day 30.
CONCLUSION: There is good concordance between the ArchitectTM SARS CoV-2 IgG Assay and PanbioTM COVID-19 IgG/IgM Rapid Test Device for the detection of SARS CoV-2 IgG.

Introduction

Highly accurate antibody tests are urgently needed to combat the SARS-CoV-2 pandemic that has already claimed >900,000 lives worldwide [1]. Antibody tests are valuable tools that can be applied to advance our understanding of the immune response to SARS-CoV-2, estimate the scope of the COVID-19 pandemic, and appropriately manage patient care as we continue to learn more about the long-term impact of COVID-19 on recovered patients. For antibody tests to effectively meet these needs, they must be highly accurate. Therefore, it is important to challenge antibody tests with clinical specimens to evaluate their performance.

Initial studies have utilized antibody tests to monitor the immune response to SARS-CoV-2 in hospitalized patients. IgM and IgG seroconversion occurred within 10–12 days and 12–14 days respectively after the onset of symptoms [2,[4], [5], [6]]. IgM levels begin to decline by week 5 and almost disappear after week 7 whereas IgG levels persist beyond week 7 [7]. When serology markers were compared to RNA detection, a higher positivity rate was observed using IgM ELISA compared to quantitative PCR after 5.5 days of illness onset [3]. With little to no gap between initial detection of IgG and IgM antibodies and a clear loss of IgM within approximately 5 weeks, IgG may serve as the primary and most persistent serological marker with the longest duration and may offer some immunity [18].

IgG testing needs are unique to each setting. In places where high throughput and detection of multiple diagnostic markers are important, core laboratory instruments will be suitable. The EUA-approved and CE-marked Abbott Architect™ SARS-CoV-2 IgG assay (Architect™) is the gold standard for COVID-19 antibody testing and has been used to monitor seroprevalence in several recent studies [[9], [10], [11]]. However, worldwide access to core lab diagnostic tests has not been equal and has been especially lacking in low- and middle-income countries (LMICs) [8] where there are few laboratory facilities to perform large scale molecular and serological testing. Lateral-flow point-of-care tests that can provide results within 30 min with a minimal need for additional supplies or instrumentation would be ideal to confirm COVID-19 infections in these countries. The Abbott Panbio™ COVID-19 IgG/IgM Rapid Test Device (Panbio™) is ideally suited to meet this unmet global need because it is a rapid antibody detection lateral flow test that is simple to use, requires a small drop of whole blood or serum and produces a result in 10−20 min. However, the accuracy of Panbio™ must be evaluated to determine whether it is a suitable alternative to core laboratory tests. In this study, the performance of the Abbott Panbio™ was compared to the Abbott Architect™ SARS-CoV-2 IgG EUA approved assay to address this question.

Materials and methods

Specimens

Two panels of specimens were utilized to challenge both antibody tests: (1) a set of 150 pre-pandemic negative specimens collected in 2014, and (2) a set of 122 specimens from 87 hospitalized COVID-19 patients in the US and UK that were confirmed with a positive SARS-CoV-2 RNA test result. The samples were collected under informed consent and were obtained from three sources as shown in Table 1 .

Table 1

Sample Information.

Cohort	Collection Date	Donor Description	RT-PCR Results (Yes/No)	Sample Type and Number Tested
				Serum	Plasma
Guys' and St. Thomas' Hospital, London, UK	April 2020	COVID-19 positive patients	Yes	82	0
Discovery Life Sciences, Huntsville, Alabama	March and April, 2020	COVID-19 positive patientsa	Yes	5	35
Gulf Coast Regional Blood Center, Houston, Texas	2014	Presumed negative donors	No	50	100

40 samples collected from 5 patients on various days after RT-PCR testing.

The samples from Guys' and St. Thomas' Hospital, London, UK (hereafter referred to as the UK cohort) were from hospitalized patients >14-days post-onset of symptoms and were confirmed positive for SARS-CoV-2 RNA with the AusDiagnostics SARS-CoV-2 test (175 copies/mL limit of detection for SARS CoV-2b) [21]. The samples from Discovery Life Sciences, Huntsville, Alabama (hereafter referred to as the US cohort) were from 5 hospitalized patients 66–77 years of age who tested positive with the Abbott RealTime SARS-CoV-2 EUA approved RNA test (100 copies/mL limit of detection for SARS CoV-2) [22]. The samples from the Gulf Coast Regional Blood Center, Houston, Texas (hereafter referred to as the Gulf Coast cohort) were collected in 2014 and were presumed negative for SARS CoV-2.

Serological testing

The specimens were tested on the Panbio™ and Architect™ according to the package inserts. Both the Panbio™ and Architect™ assays detect IgG against the SARS-CoV-2 nucleocapsid (N) protein.

For Panbio™ testing, the serum and plasma samples were mixed by low speed vortexing after which 10 μL was applied to the specimen well of the test device. Then, 2 drops (approximately 60 μL) of buffer was added to the specimen well and a timer was started. The test device was read between 10 and 20 min after the start of the test. A valid result consisted of the appearance of a red line in the C (Control) area of the reading window. A negative result consisted of only the red line in the C area of the reading window. The presence of a red line in the C and G (IgG) areas of the reading window indicated a valid result for the presence of IgG. Since this study was performed to determine the presence of IgG, results appearing in the M (IgM) area of the reading window were not evaluated.

Results

A comparative evaluation between the Panbio™ and the Architect™ was conducted to determine specificity, sensitivity, and concordance with the same panels of samples. First, specificity was evaluated with 150 pre-pandemic specimens collected from healthy donors in the US in 2014. None of the specimens were positive on the Architect™ test, whereas 1 false positive was detected with the Panbio™ test, resulting in specificity rates of 100 % and 99.4 %, respectively.

Second, sensitivity was evaluated with panels of specimens collected from hospitalized patients in the US and UK. The US panel consisted of 40 serial collections from 5 individual donors during which seroconversion occurred between 1–17 days post positive RNA test results. IgG was detected in these patients by both the Architect™ and Panbio™ on the same timepoints between 5–8 days after the positive RNA result (Table 2 ). The complete concordance between the two tests with these samples indicates similar performance for detecting seroconversion for both Architect™ and Panbio™.

Table 2

US Cohort: Time Course for IgG Testing and Detection.

Patient	Days from RT-PCR	Days since 1stBleed	ArchitectTMS/Ca	PanbioTM
#1	5	0	3.60	Positive
	7	2	4.14	Positive
	10	5	5.27	Positive
	11	6	5.46	Positive
	13	8	5.80	Positive
#2	1	0	0.02	Negative
	3	2	0.02	Negative
	4	3	0.03	Negative
	5	4	0.04	Negative
	6	5	0.14	Negative
	8	7	2.51	Positive
	9	8	3.96	Positive
	10	9	4.48	Positive
	11	10	4.77	Positive
	14	13	5.12	Positive
	16	15	5.47	Positive
	17	16	5.45	Positive
#3	4	0	0.10	Negative
	8	4	1.47	Positive
	11	7	2.67	Positive
	12	8	3.05	Positive
	14	10	3.54	Positive
	15	11	3.51	Positive
	16	12	3.74	Positive
#4	6	0	5.71	Positive
	7	1	5.74	Positive
	8	2	6.35	Positive
	9	3	6.43	Positive
	10	4	6.55	Positive
	12	6	6.98	Positive
	13	7	6.99	Positive
	14	8	6.84	Positive
	15	9	7.10	Positive
#5	1	0	0.09	Negative
	2	1	0.37	Negative
	6	5	4.79	Positive
	7	6	5.50	Positive
	8	7	5.50	Positive
	9	8	5.58	Positive
	11	10	5.37	Positive

Architect S/C of 1.4 or higher is a positive result.

Sensitivity was further evaluated with the UK panel of single timepoint collections from 82 hospitalized patients between 14–56 days post symptom onset. Testing of these samples on the Architect™ and Panbio™ was performed on the same day. Amongst these, six of the samples tested negative by both the Architect™ and Panbio™ collected on days 15–38 post onset of symptoms. A review of the medical histories for these patients revealed that 5 of 6 suffered from immune disorders or were taking immune-suppressive medications (Table 4). Thus, the low IgG levels in these patients can be attributed to underlying conditions affecting the immune system. The Architect™ assay had a sensitivity of 100 % (76/76) and 92.6 % (76/82) when the immunosuppressed samples were excluded and included in the analysis, respectively. The equivalent sensitivities for the Panbio™ assay were 98.7 % (75/76) and 91.5 % (75/82).

Table 4

UK Cohort: Medical History of SARS-CoV-2 IgG Negative Patients.

Patient	ArchitectTMSARS CoV-2 IgG Assay (S/C)	PanbioTMCOVID-19 IgG/IgM Rapid Test Device	Medical History
#1	0.02	Negative	Celiac disease, kidney transplant recipient on immunosuppressants. Other: COPD, sarcoidosis
#2	0.87	Negative	Rheumatoid arthritis, on immunosuppressant. Other: IHD, heart failure
#3	0.96	Negative	Type 2 diabetes, dialysis
#4	0.02	Negative	Adult acute myeloid leukemia, on immunosuppressant
#5	0.65	Negative	Adult acute myeloid leukemia, on immunosuppressant
#6	0.03	Negative	X-linked hypogammaglobulinaemia, receiving intravenous IgG

When both the UK and US specimens from hospitalized COVID-19 patients were combined, the overall sensitivity for patients >14 days post symptom onset or >5 days post RNA-positive results for the Architect™ assay were 99.1 % (108/109) and 93.9 % (108/115) when immunosuppressed patients were excluded and included, respectively. For the Panbio™ assay, the sensitivities were 98.2 % (107/109) and 93.0 % (107/115) when immunosuppressed patients were excluded and included, respectively, as shown in Table 3 .

Table 3

Specificity and Sensitivity of Architect™ and Panbio™ assays.

Assay	Specificity(%)	Sensitivity
		Excluding Immunosuppressed Patients (%)	Including Immunosuppressed Patients (%)
ArchitectTM	100.0	99.1	93.9
PanbioTM	99.4	98.2	93.0

The medical histories of 6 the immunosuppressed patients from the UK cohort are shown in Table 4. Lastly, the overall concordance between the two tests for all specimens in the study was determined to be 99.3 % (270/272), with one Architect™ negative and one Architect™ positive sample having discordant results with Panbio™, as shown in Table 5 .

Table 5

Concordance between Architect™ SARS-CoV-2 IgG Assay and Panbio™ COVID-19 IgG/IgM Rapid Test Device.

		ArchitectTMSARS-CoV-2 IgG Assay
		Positive	Negative	Total
PanbioTM COVID-19 IgG/IgM Rapid Test Device	Positive	107	1	108
	Negative	1	163	164
	Total	108	164	272

Discussion

This study was performed to compare the performance of the Abbott Panbio™ COVID-19 IgG/IgM Rapid Test Device against the gold-standard Abbott Architect™ SARS CoV-2 IgG Assay for the detection of the COVID-19 IgG antibody. We report an overall percent agreement of 99.3 % between Architect™ and Panbio™. In patients that were >14 days post symptom onset or >5 days post RNA-positive, the sensitivity was 100 % (76/76) and 92.6 % (76/82) when the immunosuppressed patients were excluded and included in the analysis, respectively, for Architect™. For the Panbio™ assay, the equivalent sensitivities were 98.7 % (75/76) and 91.5 % (75/82). The specificity was 100 % (150/150) for Architect™ and 99.3 % (149/150) for Panbio™. These sensitivity and specificity rates for Architect™ are similar to those reported in other recent studies [9,11]. The observed sensitivity for Panbio™ (92.7 %) was higher compared to other lateral flow tests (55–70 %), whereas the specificity for Panbio™ (99.3 %) was similar to those evaluated in a recent study (95–100) [12]. The sensitivity and specificity of serology tests are critical to enable proper patient management. A recent validation study determined the Architect™ assay to have 100.0 % sensitivity and 93.9 % specificity [26]. The same study determined the positive percent agreement and negative percent agreement to be 100.0 % and 99.6 %, respectively.

The discordant results are shown in Table 6 . For both results, the Architect™ test result is considered correct.

Table 6

Discordant Results.

Discordant Sample	Cohort	Result		Discordant Result
		ArchitectTMSARS CoV-2 IgG Assay (S/C)	PanbioTMCOVID-19 IgG/IgM Rapid Test Device
#1	Guys' and St Thomas' Hospital, London, UK	4.51	Negative	False Negative
#2	Gulf Coast Regional Blood Center, Houston, Texas	0.07	Positive	False Positive

Discordant sample #1 was positive by Architect™ (S/C = 4.51) which is well above the threshold S/C value of 1.40. The false negative rate for the Panbio™ is 0.9 % (1/108). Discordant sample #2 was negative by Architect™ (S/C = 0.07) but was determined to be positive by the Panbio™. Since the sample was collected in 2014 several years before the emergence of COVID-19, the positive result is likely due to cross-reactivity between the sample and the components of the Panbio™ assay. The false positive rate for the Panbio™ COVID-19 IgG/IgM Rapid Test Device is 0.6 % (1/164). A limitation of the present study was that only serum and plasma samples were tested, whereas whole blood samples were not. While it has already been established that the Panbio™ test is compatible with fingerstick whole blood, venous whole blood, serum and plasma [27], and the Architect™ test is also compatible with venous whole blood, serum and plasma [28], future studies will be necessary to compare the concordance between the two tests for fingerstick whole blood against venous whole blood.

In the UK cohort, 6 of the 82 patient samples that were RT-PCR positive were IgG-negative when tested with both the Architect™ and Panbio™assays. A review of their medical histories revealed that 5 of these 6 patients were either immunocompromised, had auto-immune disease, or were on immunosuppressants. Delayed or completely absent seroconversion in immunocompromised COVID-19 patients has been previously reported [11]. Delayed antibody responses to other virus-borne diseases such as Hepatitis B and West Nile has been shown in immunosuppressed patients [13,14]. Samples from these patients who were RT-PCR positive but IgG negative can be attributed to low levels of antibody production that were below the detection limits of the Architect™ and Panbio™ assays.

The seroconversion profiles of the UK and US cohorts is consistent with other reports. For the UK cohort, all 76 patients who were not amongst the immunocompromised group seroconverted after day 14 of symptom onset which is consistent with other recent reports of IgG seroconversion seen in the second week after symptom onset [2,5]. For the US cohort, IgG-seroconversion was seen in all patients >5 days after being determined to be RT-PCR positive, consistent with other reports of the number of days from a positive RT-PCR result to IgG presentation [16].

While RNA testing is important for the initial detection of infection, RT-PCR testing has limitations including difficulties in sampling, technique-dependent variability of results, and decline in RNA levels within days after the initial infection [15,17]. The variability in symptoms can also lead to delays in RT-PCR testing especially in asymptomatic patients [20]. Since IgG levels have been shown to persist beyond 7 weeks after symptom onset, a positive antibody test is a longer-term marker of infection [7]. Antibody testing offers an alternative for patient management when RT-PCR results are not available and can serve as a secondary confirmation of infection for patients with a positive or inconclusive RT-PCR result [8]. In particular, the Panbio™ test can play a critical role in monitoring long-term antibody response to SARS-CoV-2 especially in settings that lack laboratory facilities to perform large scale molecular and serological testing, whereas the Architect™ test is a high-throughput assay that is more suitable for larger diagnostic laboratory testing. However, the utility of antibody testing is limited by the timing of each individual’s immune response, and would be impacted by the potential loss of antibodies over time (i.e. seroreversion).

In addition to diagnostic uses, antibody testing is suitable for public health surveillance and vaccine development [19]. Indeed, the neutralizing antibody (nAb) response appears to be proportional to the severity of the disease [25] and the spike (S) protein is the main inducer of neutralizing antibodies [24]. A recent study showed that N is more sensitive than S for detection by antibodies: 100 % for N compared to 91 % for S at >14 days post symptom onset [23]. The N protein is therefore predicted to serve as a surrogate marker for the S protein-nAb complex. While all viral proteins are vulnerable to mutations that could impact diagnostic test performance, the S protein is under more selective pressure, ostensibly due to its immunodominant role [29]. As a result, continued vigilance in monitoring SARS-CoV-2 mutations will be critical for ensuring that diagnostic tests keep pace with emerging strains.

CRediT authorship contribution statement

Rahul Batra: Writing - original draft. Luis Gonzalez Olivieri: Conceptualization, Methodology, Writing - original draft, Data curation. Delfin Rubin: Investigation. Ana Vallari: Investigation. Sandra Pearce: Investigation. Ana Olivo: Investigation. John Prostko: Investigation. Gaia Nebbia: Investigation. Sam Douthwaite: Investigation. Mary Rodgers: Writing - original draft. Gavin Cloherty: Writing - original draft.

Declaration of Competing Interest

Luis Gonzalez Olivieri, Mary Rodgers, Gavin Cloherty, Delfin Rubin, Ana Vallari, Sandra Pearce, Ana Olivo and John Prostko are employees and shareholders of Abbott Laboratories.