Evaluation of Abbott anti-SARS-CoV-2 CMIA IgG and Euroimmun ELISA IgG/IgA assays in a clinical lab.

BACKGROUND: We report our findings of test performance especially specificity of a fully automated Abbott Architect anti-SARS-CoV-2 CMIA IgG and Euroimmun anti-SARS-CoV-2 ELISA IgA/IgG in human plasma.
METHODS: We used positive cohort of 97 samples from Covid-19 patients or healthcare workers, collected at late time points from symptom onsets. We also included another cohort of 215 samples as negative controls, 78 of which had positive serology test results of other infectious diseases or autoimmunity. Assay specificity was assessed by using a total of 847 anonymized samples which were collected before the Covid-19 pandemic from local patient populations seeking clinical care for rheumatoid diseases, thyroid cancer, and therapeutic drug monitoring.
RESULTS: Abbott IgG, Euroimmun IgG/IgA had high precision, demonstrated by both intra- and inter-day CVs of <2%. There was no Abbott or Euroimmun IgG assay cross reactivity in the 78 samples with positive serology of non-SARS-CoV-2 infectious diseases and positive autoimmune antibodies. The Abbott IgG has specificity of 99.6%, while Euroimmun IgG and IgA were as high as 91.5% and 71.5%, respectively.
CONCLUSIONS: Our evaluation confirmed high specificity of the Abbott IgG assay, while it was lower for Euroimmun IgG. Euroimmun IgA has suboptimal specificity which may limit its clinical use. Assay sensitivity was high for both Abbott and Euroimmun IgG assays.

Introduction

There is continuing interest and demand for serologic tests for the novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the pathogen which causes the coronavirus disease 2019 (COVID-19). However, the clinical utilities of serologic tests (e.g. anti-SARS-CoV-2 IgG) remain uncertain at present. For example, it is unclear if a positive anti-SARS-CoV-2 IgG result may indicate immunity, or if it may indicate past or current infection, the diagnosis of which is and should be made by molecular tests (e.g. RT-PCR) which detect viral RNA [1]. Currently, serologic test results are largely confined to epidemiologic analysis and depending on the prevalence of infection in the population, may require tests with exceptionally high specificity in order to make accurate assessments and conclusions of disease spread and infection rates [2]. There are currently many commercial tests available due to the relaxation of regulatory oversight. However, there has been a critical lack of data of rigorous validation to verify assay sensitivity and specificity. Sharing and publicizing SARS-CoV-2 serologic assay performance data by users, such as the clinical laboratories, is critical to health care providers and the general public. As of this writing, over 20 commercial companies and clinical laboratories have received emergency use authorization (EUA) by the FDA for their serology assays [3]. Most of these companies claim high assay sensitivity and specificity, based on abbreviated evaluation which may be limited in scope and sample size. Early studies demonstrated high sensitivity and specificity of both Abbott and Euroimmun (EI) IgG assays [4], [5].

Materials and methods

Abbott anti-SARS-CoV-2 IgG assay

The chemiluminescent microparticle immunoassay (CMIA) from Abbott Diagnostics is a 2-step qualitative assay detecting in serum or plasma IgG antibodies against the SARS-CoV-2 nucleocapsid protein, which was performed on an Architect i2000 instrument as random access and by following manufacturer’s instructions. Briefly, in step one, patient sample, assay diluent, and SARS-CoV-2 antigen coated paramagnetic microparticles are combined and incubated. If anti-SARS-CoV-2 IgG is present in the sample, it will bind to the SARS-CoV-2 antigen coated microparticles. In step two, the reaction mixture is washed and acridinium-labeled anti-human IgG conjugate is added. Following incubation and another wash cycle, pre-trigger and trigger solutions are added. The pre-trigger solution (hydrogen peroxide) splits the acridinium dye off the IgG conjugate that is bound to the microparticle complex. When the trigger solution (sodium hydroxide) is added, the acridinium undergoes an oxidative reaction. This produces the chemiluminescent reaction which is measured as relative light units (RLUs). There is a direct relationship between the RLUs measured and the amount of anti-SARS-CoV-2 IgG antibodies in the sample.

The assay relies on an assay-specific calibrator to report a ratio of specimen absorbance to calibrator absorbance. The interpretation of result is determined by an index (S/C) value, which is a ratio over the threshold value. The Abbott IgG assay result is positive (index ≥1.4) or negative (index <1.4) and depending on the laboratory information system (LIS) setup, the index value could be displayed along with a positive result to mark the relative strength of signal.

EI anti-SARS-CoV-2 IgG and IgA assays

The EI ELISAs are separate qualitative assays detecting in serum or plasma either IgG or IgA antibodies against the S1 domain of viral spike protein in microplate strips each with 8 break-off reagent wells coated with recombinant structural protein of SARS-CoV-2. Evaluation of the two assays were performed by the EUROLab Workstation by following manufacturer’s instructions and program. Briefly, in the first reaction step, diluted patient samples are incubated in the wells. If antibodies against the SARS-CoV-2 spike protein are present in the samples, they will bind to the antigen, forming antigen-antibody complexes. Residual sample and non-specific reactants are eliminated by washing. Conjugate (horseradish peroxidase-labeled anti-human IgG or IgA) is added and will bind to these complexes. Unbound conjugate is removed by washing. Enzyme substrate/chromogen (TMB/H2O2) is then added and incubated. In the presence of bound enzyme, the substrate is converted to a product. After adding stop solution (0.5 mol/l sulfuric acid), the optical density (OD) of the end reaction is measured spectrophotometrically at 450 nm.

The OD of each sample is directly proportional to the concentration of anti-SARS-CoV-2 IgG or IgA. Results are normalized against the calibrator OD from each plate and expressed as a ratio (sample OD/calibrator OD). All steps for this assay have been automated on the EUROLab Workstation. The assays rely on an assay-specific calibrator to report a ratio of specimen absorbance to calibrator absorbance. The interpretation of result is determined by an index (S/C) value, which is a ratio over the threshold value. The EI IgG or IgA assay result is positive (index ≥1.1), borderline (index ≥0.8 but <1.1), or negative (index <0.8). Depending on the laboratory information system (LIS) setup, the index value could be displayed along with a positive result to mark the relative strength of signal.

Assay precision, carryover, and interference evaluation

We used positive and negative controls supplied in the test sets and additional 6 known positive patient samples for inter-day and intra-day assay precision evaluation. Potential assay interference was evaluated by spiking pooled patient samples of low, medium, and high level of anti-SARS-CoV-2 IgG concentrations with (final concentration of 0.2–660 mg/dl) each of 16 compounds (biotin, acetylcysteine, ampicillin, cefoxitin, doxycycline, theophylline, levodopa, methyldopa, metronidazole, acetylsalicylic acid, ibuprofen, phenylbutazone, rifampicin, cyclosporine, acetaminophen, and heparin), ethyl alcohol (5%), bilirubin, hemoglobin, protein, and triglyceride-rich lipoprotein.

Assay sensitivity and cross reactivity evaluation

For clinical sensitivity evaluation, we analyzed 97 specimens from 97 patients or healthcare workers with RT-PCR confirmed and/or clinical assessment indicated SARS-CoV-2 infections. Duration from symptom onset and/or from first or earliest positive nasopharyngeal swab RT-PCR result was determined by medical chart review (approved by the medical school and hospital IRB). Assay cross reactivity was assessed by using a collection of 78 samples with positive ANA (by ELISA), dsDNA, RF, cyclic-citrullinated peptide IgG, RPR, and positive serology for HAV (IgG), HBV (HBV surface Ab, HBV core Ab), HCV, CMV, VZV, EBV, rubella, rubeola, mumps, HSV, and treponema pallidum, all of which were collected during the current COVID-19 pandemic. An additional 137 control samples were included for method comparison, of which there were no RT-PCR results nor clinical assessment indicating SARS-CoV-2 infections.

Assay specificity evaluation

Specificity was determined by using 847 de-identified remnant serum samples from rheumatoid disease screening (n = 643; 2011–2013), therapeutic drug monitoring (TDM) of lamotrigine, levetiracetam, testing for thyroglobulin (Tg), CA125, CA19-9, CEA, AFP, and CA15-3 (n = 94; before October 2019), and serum protein electrophoresis test (n = 110; 2012). Samples were from patients ranged in age from 1 to 95 y with 67% female and 33% male. Samples were previously tested for ANA by ELISA, specific autoantibodies against SSA, SSB, centromere, Sm, scl-70, Jo-1, RNP, and for lamotrigine, levetiracetam, Tg, as well as anti-Tg antibody. A total of 165 samples were positive for one or more of ANA screening by ELISA or specific autoantibody results, with a positive rate of 25%. The samples with Tg results had 23% positive rate for the concurrent anti-Tg autoantibodies. Remnant specimens were obtained and stored frozen at −30 °C or at −80 °C until analysis. Each specimen was thawed, and aliquots made within 24–48 h of analysis with same aliquot used for all 3 assays.

Results

For precision studies, QC materials (positive and negative) as supplied by Abbott and EI were analyzed. Inter-day precision (CV) of QC samples (n = 10) and 6 known positive patient samples (n = 5) were all <2%. Intra-day precision (CV) of QC samples (n = 20) and 6 known positive samples (n = 20) were all <2%. No evident interference was observed for hemolysis, icterus, and lipemia, nor for samples spiked with ethyl alcohol, protein or each of the 16 compounds. There was no carryover greater than 1%.

The diagnostic sensitivity of Abbott IgG at 14–21, >21 days post symptom onset was 96% and 100%, respectively. The overall sensitivity (including samples of unknown days since symptom onsets) was 97.9%. The diagnostic sensitivity for EI IgG at 14–21, >21 days post symptom onset was both 100%, and there was no borderline result. The overall sensitivity (including samples of unknown days since symptom onsets) remained 100%. Using time from the first positive SARS-CoV-2 RT-PCR result, the sensitivity of Abbott IgG at <10 days and >10 days was 87.5% and 100%, respectively. The overall sensitivity (including samples of unknown days since first positive RT-PCR result) was 97.9%. And sensitivity of EI IgG at <10 days and >10 days was both 100%, and there was no borderline result. The overall sensitivity (including samples of unknown days since first positive RT-PCR result) remained 100% (Table 1 ).

Table 1

Assay Sensitivity of Abbott CMIA IgG and EI ELISA IgG (borderline as negative).

Assay sensitivity		Abbott (CMIA) IgG	EI (ELISA) IgG
	days since symptom onset
	14–21	4/4 (100%)	4/4 (100%)
	>21	26/27 (96.0%)	27/27 (100%)
	unknown	65/66 (98.5%)	66/66 (100%)
	overall	95/97 (97.9%)	97/97 (100%)
Assay sensitivity	days since positive RT-PCR
	≤10	7/8 (87.5%)	8/8 (100%)
	>10	48/48 (100%)	48/48 (100%)
	unknown	40/41 (97.5%)	41/41 (100%)
	overall	95/97 (97.9%)	97/97 (100%)

The diagnostic specificity of the Abbott IgG was 99.6%, with 3 positive results (index of 1.4, 1.8, and 4.2) out of a total of 847 pre-pandemic samples tested. The diagnostic specificity of the EI SARS-CoV-2 IgG assay was 91.5% if borderline results were considered negative, and 87.4% if borderline results were considered positive. The EI IgA assay specificity was 71.5% and 60.0% when borderline results were included as negative or positive, respectively (Table 2 ).

Table 2

Assay Specificity of Abbott CMIA IgG and EI ELISA IgG/IgA.

Immunoassay			Abbott (CMIA)	EI (ELISA)
	group		IgG	IgG	IgA
	negative		844	740	508
	positive		3	72	241
		borderline		35	98
Assay specificity			99.6%
		borderline as (−)		91.5%	71.5%
		borderline as (+)		87.4%	60.0%

Abbott IgG and EI IgG assays cross reactivity results were all negative using 78 samples with positive ANA by ELISA (n = 5), dsDNA (n = 5), RF (n = 3), cyclic-citrullinated peptide IgG (n = 2), and positive serology for HAV (n = 6), HBV (n = 11), HCV (n = 3), CMV (n = 2), VZV (n = 7), EBV (n = 6), rubella (n = 5), rubeola (n = 4), mumps (n = 2), HSV (n = 7), RPR (n = 5), and treponema pallidum (n = 5).

Combining method comparison results of 847 pre-pandemic, 97 positive COVID-19, 78 samples for cross reactivity, as well as additional 137 COVID-19 undefined samples, the overall agreement between the Abbott IgG and EI SARS-CoV-2 IgG assays was 91.5% and 88.4% (not shown) if EI IgG borderline results were considered negative or positive, respectively (Table 3 ).

Table 3

Overall agreement between Abbott CMIA IgG and EI ELISA IgG (borderline as negative).

		EI IgG
		negative	positive	total
Abbott IgG	negative	962	96	1058
	positive	3	98	101
	total	965	194	1159*

Total number of samples includes 97 clinically confirmed positive samples, 78 positive serology (other infectious diseases or autoimmunity), 137 clinically unconfirmed samples during collected during the pandemic, as well as 847 samples collected before the pandemic.

Discussion

Our study confirmed the high specificity of Abbott IgG assay by using a larger cohort of samples from local patient population which were collected in the pre-pandemic era. The EI IgG assay had a much lower specificity than its claim in IFU at the time of FDA EUA, regardless of whether borderline results were considered positive or negative. It was evident that EI IgG assay was associated with many more false positive results relative to the Abbott IgG assay. The EI IgA assay demonstrated low specificity when borderline results were considered negative, and much worse when borderline results were considered positive. Because of the low specificity of EI IgA assay, its clinical sensitivity was not assessed.

The 3 positive samples by Abbott IgG were from a 7 y female post liver transplant with low positive ANA test, a 78 y male following up clinically for slightly elevated AST and ALT, also found to have low positive ANA, anti-mitochondrial and anti-smooth muscle antibody tests, as well as a 36 y female post laparoscopic cholecystectomy, found to have low positive ANA result. Review of these 3 positive results (index of 1.4, 1.8, and 4.2) from cohort of 847 pre-pandemic samples indicated that they were random false positives which were not uncommon to any infectious disease serology tests.

The low specificity of the EI IgG assay would be an issue since serologic assays have been used for both population screening and epidemiologic studies [3]. The false positive results may be due, at least in part, to antibody cross reactivity to seasonal coronaviruses which share considerable homology with SARS-CoV-2 [6]. A serologic test with high specificity is essential to achieve a high positive predictive value (PPV) [7]. Assuming assay sensitivity of 100% and a disease prevalence of 5%, the PPV of the Abbott IgG assay was an estimated 92.6% and an estimated 5.6% for the EI IgG assay, highlighting the importance of a high specificity assay. The EI IgG assay specificity may be enhanced by raising the index (S/C) value at the expense of assay sensitivity. However the selection of optimal threshold index value requires a study using a collection of large number of well characterized clinical samples, which would have been performed by the assay manufacturer under normal circumstances before submission for regulatory (i.e. FDA) approval.

Based on the preliminary data from a total of 78 samples, both Abbott IgG and EI IgG assays cross reactivity results were negative. However, it is inconclusive as each of infectious diseases were represented by 2–11 positive samples. A more comprehensive evaluation may be necessary and should be performed by using a much larger sample collection. Additional collection of well characterized positive serology samples of non-SARS-CoV-2 coronavirus strains, such as coronavirus HKU1, NL63, OC43, or 229E would be desirable, but was not performed in the current study.

This clinical sensitivity study required a well characterized sample collection from COVID-19 infected person, with documentation of time of collection from post-onset of symptoms and/or from the first positive RT-PCR result. Our sample selection consisted of samples collected late in the disease course, mostly during follow up visits. Most of these samples already contained high levels of antibody levels as a response to the infection. Nonetheless, we found Abbott IgG assay to be less sensitive compared with EI IgG, however the difference is small based on the small size of samples included. At this time, it is unclear if antibodies against the SARS-CoV-2 nucleocapsid or the spike protein confer protection, however preliminary studies indicated similar agreement values of both SARS-CoV-2 nucleocapsid IgG assay and spike protein S1/S2 IgG assay with microneutraliztion assay [8].

In conclusion, the Abbott SARS-CoV-2 IgG assay demonstrated much higher specificity than EI SARS-CoV-2 IgG assay in this study. Due to the lack of clinical utilities, COVID-19 serology tests are currently confined to data collection for population health and epidemiology studies. Because the current disease prevalence is unknown and presumably very low in local populations, a highly specific test is critical in providing accurate data which would lead to accurate conclusions.

CRediT authorship contribution statement

Justin Manalac: Methodology, Validation, Supervision. Jennifer Yee: Methodology, Data curation, Validation, Writing - original draft. Kyle Calayag: Data curation, Methodology. Linda Nguyen: Methodology. Payal M. Patel: Methodology, Validation. Daniel Zhou: Methodology. Run-Zhang Shi: Conceptualization, Data curation, Writing, Supervision.