Evaluation of four laboratory-based SARS-CoV-2 IgG antibody immunoassays.

OBJECTIVES: To evaluate the diagnostic performances of four SARS-CoV-2 IgG antibody immunoassays.
METHODS: Following immunoassays were studied: Abbott's SARS-CoV-2 IgG assay, Diasorin's Liaison SARS-CoV-2 S2/S2 IgG assay, Euroimmun's Anti-SARS-CoV-2 IgG ELISA, and Roche's Elecsys Anti-SARS-CoV-2 assay. Specificity was retrospectively evaluated with 38 samples from 2019. Sensitivity samples (n = 147) were taken from SARS-CoV-2 real-time PCR-positive patients who developed COVID-19 symptoms ten days earlier.
RESULTS: Mean specificity was 96.6%. Mean sensitivity was 62.7% from ten days after onset of symptoms, 84.4% from 15 days after onset of symptoms, and 87.5% from 20 days after onset of symptoms.
CONCLUSIONS: Specificity was high, while Abbott and Roche were 100% specific. Sensitivity increased over time, with Abbott and Roche having the highest sensitivity at all time points with ≥90% from 20 days after symptoms' onset. These findings may assist in selecting SARS-CoV-2 IgG antibody immunoassays for additional diagnostics, epidemiological research, and vaccine development.

Introduction

Late 2019, a novel strain of coronavirus affecting primarily the respiratory system emerged. Being closely related to severe acute respiratory syndrome coronavirus (SARS-CoV), it was named severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). From early 2020, the outbreak evolved into a pandemic of coronavirus disease 2019 (COVID-19), the disease caused by SARS-CoV-2 (European Centre for Disease Prevention and Control, 2020). COVID-19 symptoms may vary in severity from having no symptoms to anosmia, rhinitis, sore throat, cough, fever, dyspnea, pneumonia, acute respiratory distress syndrome, sepsis, and septic shock, potentially fatal (World Health Organization, 2020a).

Currently, gold standard tests to diagnose COVID-19 are nucleic acid amplification tests (NAAT), such as polymerase chain reaction (PCR), on respiratory tract samples (World Health Organization, 2020b). However, false negative results occur due to sampling errors or insufficient amounts of viral genome at the site of sample collection (e.g., in mild cases and from eight days after onset of symptoms) (Yang et al., 2020). Therefore, additional diagnostic methods are needed. Detection of antibodies in serum may be suitable after acute illness. Mild or even asymptomatic infections might also be more easily detected, which may be of epidemiological importance (Guo et al., 2020).

Most commonly used methods for detecting antibodies are immunoassays and lateral flow assays (LFA). Immunoassay is a laboratory-based method mostly performed with an appropriate automated analysis device, thereby achieving a high-throughput detection. LFA is suitable for point-of-care testing but has a lower throughput detection and is more labor intensive.

Recently, several serological SARS-CoV-2 antibody assays have been developed and made available. Due to urgency, usually with limited validation (commissioned) by the developer. Therefore, we evaluated and compared the diagnostic performance of four immunoassays for the detection of SARS-CoV-2 IgG antibodies.

Materials and methods

Study design

This retrospective study was approved by the standing Committee on Ethics of the University Hospital Brussels. The study was conducted in the microbiology and chemistry laboratories of the University Hospital Brussels between May 7 and 26, 2020. All sera were left-over samples from blood samples taken for previous clinical examination and stored at ≤−20 °C in the hospital's laboratory.

Serum samples

Specificity samples (n = 38) were collected between January 1 and December 26, 2019 from 20 patients who received a quadrivalent influenza vaccine at least two weeks earlier, 11 patients clinically diagnosed with viral respiratory infection, and seven patients with primary Epstein-Barr virus infection. This patient group consisted of 25 females and 13 males with a median age of 22.5 years (M = 21.8 years; range 1–75 years).

Sensitivity samples (n = 147) were collected between March 7 and May 12, 2020 from patients hospitalized for severe COVID-19 who tested SARS-CoV-2 positive by real-time PCR on nasopharyngeal samples at least ten days after initiation of COVID-19 symptoms. This patient group consisted of 54 females and 93 males with a median age of 60.0 years (M = 60.7; range 24–93 years). Serum samples were collected on average 17 days after onset of symptoms (range 10–48 days).

Immunoassays

The SARS-CoV-2 IgG assay (Abbott Laboratories; #06R8620; 100 tests/kit; CE marked; for use on Architect i System) is a chemiluminescent microparticles immunoassay (CMIA) for qualitative detection of IgG antibodies to the nucleocapsid protein of SARS-CoV-2 in human serum or plasma using purified SARS-CoV-2 recombinant antigen coated microparticles. Following the manufacturer's recommendations, the cut-off index (COI) was <1.4 for a negative result and ≥1.4 for a positive result.

The Liaison SARS-CoV-2 S2/S2 IgG assay (Diasorin S.p.A; #311450; 110 tests/kit; CE marked; for use on Liaison XL Analyzer) is a chemiluminescence immunoassay (CLIA) for quantitative detection of IgG antibodies to the S1/S2 domains from the SARS-CoV-2 spike proteins in human serum or plasma using magnetic beads coated with S1 and S2 antigens. Following the manufacturer's recommendations, the cut-off value was <12.0 AU/mL for a negative result, ≥12.0 to <15.0 AU/mL for an equivocal result, and ≥15.0 AU/mL for a positive result.

The Anti-SARS-CoV-2 ELISA IgG (Euroimmun AG; #El 2606-9601 G; 96 tests/kit; CE marked; for use on Eurolab Workstation ELISA Analyzer I-2P) is an enzyme-linked immunosorbent assay (ELISA) for semi-quantitative detection of IgG antibodies to the S1 domain from the SARS-CoV-2 spike protein including the immunologically relevant receptor-binding domain in human serum or plasma using microplate wells coated with recombinant structural protein (S1 domain) of SARS-CoV-2. Following the manufacturer's recommendations, the COI was <0.8 for a negative result, ≥0.8 to <1.1 for an equivocal result, and ≥1.1 for a positive result.

The Elecsys Anti-SARS-CoV-2 assay (Roche Diagnostics GmbH; #09203079190; 300 tests/kit; CE marked; for use on Cobas e 801 Analyzer) is an electro-chemiluminescence immunoassay (ECLIA) based on a double-antigen sandwich principle for qualitative detection of antibodies (including IgG) to the nucleocapsid protein of SARS-CoV-2 in human serum and plasma using a recombinant protein representing the SARS-CoV-2 nucleocapsid antigen. Following the manufacturer's recommendations, the COI was <1.0 for a negative result and ≥1.0 for a positive result.

The four studied immunoassays were performed according to the manufacturer's instructions.

Data collection and data analysis

For all patients, clinical classification (e.g., COVID-19, Epstein-Barr virus infection, and viral respiratory infection), date of symptoms onset, demographic information (age, sex), laboratory order, and test results (e.g., Epstein-Barr nuclear antigen IgG CLIA, Epstein-Barr viral capsid antigen IgG and IgM CLIA, Mycoplasma pneumoniae antibodies, real-time PCR SARS-CoV-2) were extracted from the hospital's information system. Descriptive statistics were performed with IBM SPSS Statistics 26 (IBM Corp, 2019). Specificity (defined as the proportion of true negative patients who were correctly identified as not having SARS-CoV-2 IgG antibodies), sensitivity (defined as the proportion of true positive patients who were correctly identified as having SARS-CoV-2 IgG antibodies), confidence interval (CI), positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (LR+), negative likelihood ratio (LR-), and odds ratio (OD) were calculated with VassarStats (Lowry, 2020).

Results

Specificity

Specificity was determined from 38 samples. The total number of samples tested per assay differed slightly, because four samples had insufficient volume for testing all four immunoassays. Mean specificity was 96.6% (range 89.2%–100%). See Table 1 for specificity performance characteristics per immunoassay.

Table 1

Specificity performance characteristics of four laboratory-based SARS-CoV-2 IgG antibody immunoassays.

Immunoassay	TN	FP	EQa	Specificity % (95% CI)
Abbott	36b	-	-	100 (88.0–100)
Diasorin	34c	1d	-	97.1 (83.4–99.9)
Euroimmun	33e	3f	1g	89.2 (73.6–96.5)
Roche	36h	-	-	100 (88.0–100)

Abbreviations: CI = confidence interval, COI = cut-off index, EQ = equivocal, FP = false positive, TN = true negative.

Equivocal specificity data were considered false positive in the analyses.

Range TN values 0.01–0.60 (COI negative result <1.4).

Range TN values < 3.80–5.51 (cut-off value negative result <12.0 AU/mL).

Value FP sample 187 (cut-off value positive result ≥15.0 AU/mL); sample from patient with primary Epstein-Barr virus infection.

Range TN values 0.12–0.67 (COI negative result <0.8).

Range FP values 1.21–4.65 (COI positive result ≥1.1); two samples from patients diagnosed with viral respiratory infection and one sample from a patient who received a quadrivalent influenza vaccine.

Value EQ sample 1.00 (COI equivocal result ≥0.8 to <1.1); sample from patient diagnosed with viral respiratory infection.

Range TN values 0.06–0.20 (COI negative result <1.0).

Sensitivity

Sensitivity was determined from 147 samples. Total number of samples tested per assay differed slightly, because 17 samples had insufficient volume for testing all immunoassays. Mean sensitivity was 62.7% (range 52.7%–68.1%) from ten days after onset of symptoms, 84.4% (range 80.8%–86.5%) from 15 days after onset of symptoms, and 87.5% (range 82.8%–90.3%) from 20 days after onset of symptoms. See Table 2 for sensitivity performance characteristics per immunoassay.

Table 2

Sensitivity performance characteristics of four SARS-CoV-2 IgG antibody immunoassays.

Days after onset of symptoms	∑	TP	FN	EQa	Sensitivity % (95% CI)	PPV % (95% CI)	NPV % (95% CI)	LR+	LR-	OD
Immunoassay
≥ 10 Days
Abbottb	144	98	46	-	68.1 (59.7–75.4)	100 (95.3–100)	43.9 (33.1–55.3)	-	0.31	-
Diasorinc	146	77	62	7	52.7 (44.3–61.0)	98.7 (92.1–99.9)	33.0 (23.2–43.1)	18.5	0.49	37.9
Euroimmund	133	85	46	2	63.9 (55.1–71.9)	95.5 (88.3–98.6)	40.7 (30.1–52.2)	5.91	0.40	14.6
Rochee	145	96	49	-	66.2 (57.8–73.7)	100 (95.2–100)	42.4 (31.9–53.5)	-	0.34	-
≥ 15 Days
Abbott	52	45	7	-	86.5 (73.6–94.0)	100 (90.2–100)	83.7 (68.7–92.7)	-	0.13	-
Diasorin	52	42	8	2	80.8 (67.0–89.9)	97.7 (89.2–99.9)	77.3 (61.8–88.0)	28.3	0.20	147
Euroimmun	49	41	7	1	83.7 (69.8–92.2)	91.1 (77.9–97.1)	80.5 (64.6–90.6)	7.74	0.18	42.3
Roche	52	45	7	-	86.5 (73.6–94.0)	100 (90.2–100)	83.7 (68.7–92.7)	-	0.13	-
≥ 20 Days
Abbott	30	27	3	-	90.0 (72.3–97.4)	100 (84.5–100)	92.3 (78.0–98.0)	-	0.10	-
Diasorin	30	26	3	1	86.7 (68.4–95.6)	96.3 (79.1–99.8)	89.5 (74.3–96.6)	30.3	0.14	221
Euroimmun	29	24	4	1	82.8 (63.5–93.5)	85.7 (66.4–95.3)	86.8 (71.1–95.1)	7.66	0.19	39.6
Roche	31	28	3	-	90.3 (73.1–97.5)	100 (85.0–100)	92.3 (78.0–98.0)	-	0.10	-

Abbreviations: CI = confidence interval, COI = cut-off index, EQ = equivocal, FN = false negative, LR+ = positive likelihood ratio, LR- = negative likelihood ratio, OD = odds ratio, NPV = negative predictive value, PPV = positive predictive value, ∑ = total number of samples, TP = true positive.

Equivocal sensitivity data were considered false negative in the analyses.

Range FN values 0.01–1.38 and TP values 1.41–9.12 (COI negative result <1.4 and positive result ≥1.4).

Range FN values < 3.80–11.6, EQ values 12.1–14.6, and TP values 15.3–90.7 (cut-off value negative result <12.0 AU/mL, equivocal result ≥12.0 to <15.0 AU/mL, and positive result ≥15.0 AU/mL).

Range FN values 0.14–0.70, EQ values 1.0–1.04, and TP values 1.16– > 13 (COI negative result <0.8, equivocal result ≥0.8 to <1.1, and positive result ≥1.1).

Range FN values 0.06–0.98 and TP values 1.12–88.7 (COI negative result <1.0 and positive result ≥1.0).

Discussion

Following the emergence of SARS-CoV-2, several serological tests were quickly developed and made available, mostly with limited validation. To expand their validation, we evaluated and compared the diagnostic performance of four commercially available laboratory-based immunoassays for the detection of SARS-CoV-2 IgG antibodies. Mean specificity was 96.6% (range 89.2%–100%) with Abbott's and Roche's immunoassays being 100% specific. These high degrees of specificity are in line with a Cochrane review on SARS-CoV-2 antibody tests that found pooled specificity exceeding 98% (95% CI 96.0–100; Deeks et al., 2020).

Mean sensitivity increased from 62.7% from ten days after onset of symptoms, to 87.5% from 20 days after onset of symptoms, with Abbott's and Roche's immunoassays showing the highest sensitivity at all time points. Sensitivity increasing over time is in line with the time dependent appearance of antibodies. A Cochrane review showed SARS-CoV-2 antibody tests have a sensitivity of 30.1% (95% CI 21.4–40.7) for one to seven days, 72.2% (95% CI 63.5–79.5) for eight to 14 days, 91.4% (95% CI 87.0–94.4) for 15 to 21 days, and 96.0% (95% CI 90.6–98.3) for 22 to 35 days after onset of symptoms (Deeks et al., 2020).

The variation in sensitivity of the tested immunoassays may be partly due to the SARS-CoV-2 antigen targeted. Both immunoassays with the overall lowest sensitivity detect IgG antibodies to the S1 domain from the SARS-CoV-2 spike protein, while both immunoassays with the overall highest sensitivity detect IgG antibodies to the SARS-CoV-2 nucleocapsid protein. The viral spike protein is considered to be the preferred antigen, because it is very specific and because is the main antigen eliciting neutralizing antibodies (Petherick, 2020). However, the nucleocapsid protein being the most abundant viral protein is easier to detect and, although less specific than the viral spike protein, does not pose specificity problems based on our results.

Even from 20 days after onset of symptoms, the best performing immunoassays´ sensitivity seems to be relatively low (90%). In part, this may be because not all COVID-19 patients produce antibodies (Infantino et al., 2020). This notion is supported by our results showing that negative tested samples frequently test negative across all four immunoassays. The samples were from hospitalized and critically ill patients, all of whom are expected to normally develop antibodies. However, of the 147 samples collected at least ten days after the onset of symptoms, 37 samples tested negative on all assays. Of the 53 samples collected at least 15 days after the onset of symptoms, five samples tested negative on all assays. Of the 31 samples collected at least 20 days after the onset of symptoms, three samples test negative on all assays. This is consistent with the findings of a recent study (Quan-Xin et al., 2020) which reported SARS-CoV-2 IgG rates of merely 81.1% in asymptomatic patients and 83.8% symptomatic patients three to four weeks after exposure. Interestingly, already during the early convalescent phase (eight weeks after discharge from hospital), IgG levels declined in 93.3% of the asymptomatic patients and in 96.8% of the symptomatic patients. The median decrease was 71.1% (range 32.8%–88.8%) in the asymptomatic patients and 76.2% (range 10.9%–96.2%) in the symptomatic patients, with 40% and 10% respectively of them becoming seronegative again. Furthermore, during the early convalescent phase, a decrease in neutralizing serum antibodies levels was observed in 81.1% of the asymptomatic patients (median decrease of 8.3% [range 0.5%–22.8%]) and in 62.2% of the symptomatic patients (median decrease of 11.7% [range 2.3%–41.1%]). Therefore, clinicians, future studies, and policy makers should take into account that not only patients with asymptomatic or mild COVID-19 may not develop SARS-CoV-2 IgG antibodies or show a rapid decay of these antibodies (Ibarrondo et al., 2020), but also patients with severe COVID-19.

In addition, our findings may assist in selecting appropriate SARS-CoV-2 IgG antibody immunoassays for vaccine development. However, more research is needed to determine to what extent such assays may be useful therein. After many infections, the development and persistent presence of IgG antibodies can be used as an immunity marker. However, because the kinetics of SARS-CoV-2 antibodies appear to be consistent with the kinetics of other coronaviruses and may therefore decline relatively quickly, SARS-CoV-2 antibodies may not be the most suitable outcome measure in the development of COVID-19 vaccines. Maybe it is better to use primarily T cell-mediated immunity markers. SARS-CoV-2 antibody assays, however, may still fulfill a secondary role in vaccine research, for example to gain insight into the (early) immune response more easily and quickly (Jeyanathan et al., 2020).

Strengths and limitations

To our knowledge, the present study is the first to evaluate and compare the four widely commercially available immunoassays for the detection of SARS-CoV-2 IgG antibodies of Abbott, Diasorin, Euroimmun, and Roche. Another strength is the authors' absence of conflicting interests; the independently obtained results from this study may assist others select the most appropriate immunoassay.

A limitation of this study is the relatively small sample sizes. Especially, the results with regard to specificity are prone to errors due to chance findings because of the use of only 38 samples. In addition, cross-reactivity between SARS-CoV-2 and the four relatively harmless coronaviruses causing common cold could have been tested more closely by selecting samples with proven antibodies to such a coronavirus. Unfortunately, however, these samples are not available by default since diagnosis of infections with these viruses is not standard practice. Furthermore, the specificity samples were collected between January 1 and December 26, 2019, relatively short before the spread of SARS-CoV-2 in Europe. In future research, it would be better to use seronegative samples collected at least several months before the first report of SARS-CoV-2 infections from China. Finally, the specificity samples did not contain any sera from patients with SARS-CoV. Although SARS-CoV was never widespread in Europe and has not circulated in the population since 2003, it would have been scientifically interesting to test cross-reactivity with SARS-CoV samples (Petherick, 2020), because of the high degree of similarity between both the spike proteins and several nucleocapsid proteins of SARS-CoV and SARS-CoV-2 (Lisboa Bastos et al., 2020).

Conclusion

The present study shows that the four tested immunoassays for the detection of SARS-CoV-2 IgG antibodies all have a high degree of specificity. Their performances regarding sensitivity show more variance and even from 20 days after onset of symptoms, the best performing immunoassays´ sensitivity is only 90%. However, this relatively low degree of sensitivity may be partly due to a lack of production of antibodies in some COVID-19 patients.

Author Contributions

Jorg Tanis: Conceptualization, Methodology, Formal analysis, Investigation, Data curation, Writing – Original draft, Writing - Review & Editing, Visualization. Ellen Vancutsem: Conceptualization, Investigation, Resources, Writing – Review & Editing, Project administration. Denis Piérard: Conceptualization, Resources, Writing – Review & Editing, Supervision. Ilse Weets: Resources, Writing – Review & Editing. Maria Bjerke: Investigation, Resources. Johan Schiettecatte: Investigation, Resources. Deborah De Geyter: Conceptualization, Investigation, Writing – Review & Editing, Supervision.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.