Brief clinical evaluation of six high-throughput SARS-CoV-2 IgG antibody assays.

Serological SARS-CoV-2 assays are urgently needed for diagnosis, contact tracing and for epidemiological studies. So far, there is limited data on how recently commercially available, high-throughput immunoassays, using different recombinant SARS-CoV-2 antigens, perform with clinical samples. Focusing on IgG and total antibodies, we demonstrate the performance of four automated immunoassays (Abbott Architect™ i2000 (N protein-based)), Roche cobas™ e 411 analyzer (N protein-based, not differentiating between IgA, IgM or IgG antibodies), LIAISON®XL platform (S1 and S2 protein-based), VIRCLIA® automation system (S1 and N protein-based) in comparison to two ELISA assays (Euroimmun SARS-CoV-2 IgG (S1 protein-based) and Virotech SARS-CoV-2 IgG ELISA (N protein-based)) and an in-house developed plaque reduction neutralization test (PRNT). We tested follow up serum/plasma samples of individuals PCR-diagnosed with COVID-19. When calculating the overall sensitivity, in a time frame of 49 days after first PCR-positivity, the PRNT as gold standard, showed the highest sensitivity with 93.3% followed by the dual-target assay for the VIRCLIA® automation system with 89%. The overall sensitivity in the group of N protein-based assays ranged from 66.7 to 77.8% and in the S protein-based-assays from 71.1 to 75.6%. Five follow-up samples of three individuals were only detected in either an S and/or N protein-based assay, indicating an individual different immune response to SARS-CoV-2 and the influence of the used assay in the detection of IgG antibodies. This should be further analysed. The specificity of the examined assays was ≥ 97%. However, because of the low or unknown prevalence of SARS-CoV-2, the examined assays in this study are currently primarily eligible for epidemiological investigations, as they have limited information in individual testing.

Background

SARS-CoV-2 is a new Coronavirus, belonging to the group of betacoronaviruses, which emerged in December 2019 in Wuhan, China. It is the causative agent of an acute respiratory disease known as coronavirus disease 2019 (COVID-19). The spectrum of clinical signs can be very broad and asymptomatic infections are reported. The virus has rapidly spread globally. On 11 March 2020 the World Health Organization (WHO) declared COVID-19 as a pandemic. Nucleic acid amplification testing (NAT) is the method of choice in the early phase of infection [1]. However, for epidemiological studies, in determining the seroprevalence of SARS-CoV-2 in the general population or in special collectives there is an increasing demand in the detection of antibodies – especially of IgG antibodies [2]. In addition, the SARS-CoV-2 serostatus of asymptomatic individuals or patients with mild clinical course, who present late (a couple of weeks) after infection, is of interest. Ideally, a positive IgG status will offer a potential immunity, but if so, questions on how long it will last, still remain. Furthermore for therapeutic or prophylactic approaches, convalescent plasma may be used as vaccines and other drugs are under development [3]. For these purposes, sensitive and especially highly specific antibody assays are needed.

The spike (S) protein of SARS-CoV-2 has shown to be highly immunogenic and is the main target for neutralizing antibodies [4]. Currently there are different spike (S) and/or nucleocapsid (N) protein-based commercially or in-house developed assays available, but there is limited data on how these tests perform with clinical samples.

This study aims to provide a quick overview on some of these assays (two commercially available ELISA assays, four automated immunoassays and a plaque reduction neutralization test (PRNT)) focusing on the detection and neutralization capacity of IgG antibodies in follow up serum or plasma samples of individuals with PCR-diagnosed infections with SARS-CoV-2. When calculating the overall sensitivity we used the total time frame of 49 days after first PCR-positivity and focussed on the different antigens (S- or N-antigen) used as binding antigen(s) in the assays. Typically, the majority of antibodies are produced against the N-protein, which is the most abundant protein. Therefore it is to be expected that N-protein based assays are most sensitive. On the other hand, the receptor-binding domain of the S-protein is the host-attachment protein and so it is expected to be more specific and potentially neutralizing. To assess potential cross-reactivity, we examined defined follow-up samples of individuals infected with endemic coronaviruses and other viral diseases.

Materials and methods

Serum and plasma samples

We collected follow up serum or plasma samples (in the following simply stated as samples) from individuals with PCR-diagnosed infections with SARS-CoV-2 (n = 45) (TABLE S1). Most of these individuals had a moderate to severe clinical course and required an in-patient hospital stay at the intensive care unit. Additionally, follow up samples of recent PCR-diagnosed infections with SARS-CoV (2 patients from the 2003 outbreak), HCoV-OC43 (n = 2), HCoV-HKU1 (n = 1), HCoV-NL63 (n = 1), HCoV-229E (n = 2), recent serological/PCR-diagnosed infections with acute EBV (n = 5, all serologically EBV-VCA-IgM positive and four additionally weakly EBV-VCA-IgG positive), acute CMV (n = 5, all serologically IgM and weakly IgG and one additionally PCR-positive) and 19 samples of individuals from the era before the appearance of SARS-CoV-2 or were tested SARS-CoV-2 negative via PCR-testing (as negative control cohort) were collected (TABLE S2). The non-SARS-CoV-2 samples were used to assess potential cross reactivity and the risk of potential false positive results.

Immunoassay platforms

Samples were tested within one day, in batches, on multiple commercially available (mostly automated) immunoassay platforms (Table 1 ) according to the manufacturers’ protocol.

Table 1

Examined commercially available SARS-CoV-2 antibody assays.

Assay	SARS-CoV-2 antigen (recombinant)	Company	Interpretation of results	Platform	Sensitivity (according to manufacturer's specifications)	Specificity (according to manufacturer's specifications)
SARS-CoV-2-IgG	N protein	Abbott GmbH, Wiesbaden, Germany	Index (S/C)	Abbott Architect™ i2000 (CMIA)	0−100% (day 0 to ≥ 14 days after disease onset)	99.63 %
			<1.4 = neg.
			≥1.4 = pos.
Elecsys Anti-SARS-CoV-2*	N protein	Roche Diagnostics International AG, Rotkreuz, Switzerland	signal sample/cutoff (COI)	Roche cobas™ e 411 analyzer (ECLIA)	65.5−100% (day 0 to ≥ 14 days post confirmed PCR)	99.81 %
			<1.0 = neg.
			≥1.0 = pos.
Liaison® SARS-CoV-2 S1/S2 IgG	S1 and S2 protein	DiaSorin Deutschland GmbH, Dietzenbach, Germany	arbitrary units (AU/mL)	DiaSorin LIAISON®XL (CLIA)	25−97.4% (day 0 to >15 days post confirmed PCR)	98.9 %
			<12 = neg.
			12 < 15 = equivocal
			≥15 = pos.
COVID-19 VIRCLIA® IgG MONOTEST	S1 and N protein	Vircell Spain S.L.U., Granada, Spain	Antibody index (AI)	Vircell VIRCLIA® automation system (CLIA)	12−70% (day 0 to >7 days post confirmed PCR)	98 %
			<1.4 = neg.
			1.4 < 1.6 = equivocal
			≥1.6 = pos.
Anti-SARS-CoV-2-ELISA (IgG)	S1 protein	Euroimmun, Lübeck, Germany	Ratio	manual ELISA	75−93.8% (>10−20 days to ≥21 days after disease onset)	99.6 %
			<0.8 = neg.
			0.8-<1.1 = equivocal
			≥ 1.1 = pos.
Virotech SARS-CoV-2 ELISA IgG	N protein	Virotech Diagnostics GmbH, Rüsselsheim, Germany	Index	manual ELISA	7.7−100% (day 0 to ≥ 12 days after disease onset)	100 %
			<0.9 = neg.
			0.9−1.1 = equivocal
			>1.1 = pos.

Total antibody test, not differentiating between IgA, IgM or IgG antibodies; CMIA, chemiluminescent microparticle immunoassay; ECLIA, electrochemiluminescence immunoassay; CLIA, chemiluminescence immunoassay; neg, negative; pos, positive.

ELISA

The Euroimmun SARS-CoV-2 IgG ELISA (Euroimmun, Lübeck, Germany) and Virotech SARS-CoV-2 IgG ELISA (Virotech Diagnostics GmbH, Rüsselsheim, Germany; Table 1) were used, in an identical manner, according to the manufacturer’s recommendation. Samples were diluted 1:101 in sample buffer and incubated at 37° for 60 or 30 min., respectively, in a 96-well microtiter plate followed by each protocols’ washing and incubation cycles, including controls and required reagents. Optical density (OD) was measured for both assays at 450 nm using the microplate reader of a VIRCLIA® automation system (Vircell Spain S.L.U., Granada, Spain). Titers were calculated and results interpreted according to each manufacturer’s protocol.

Plaque reduction neutralization test (PRNT)

To test for neutralizing capacity of SARS-CoV-2 specific antibodies, Caco-2 cells (human colon carcinoma cells, ATCC DSMZ ACC-169 (American Type Culture Collection, Manassas, Virginia, USA)) were seeded on a 96-well plate 3–5 days prior infection. 2-fold dilutions of the test sera beginning with a 1:10 dilution (1:10; 1:20; 1:40; 1:80; 1:160; 1:320; 1:640 and 1:1280) were made in culture medium (Minimum essential medium, MEM; Sigma-Aldrich, St. Louis, USA) before mixed 1:1 with 100 TCID50 (Tissue culture infectious dosis 50) of reference virus (SARS-CoV-2 FFM1 isolate). FFM1 was isolated from a patient at University Hospital Frankfurt who was tested positive for SARS-CoV-2 by PCR. Virus-serum mixture was incubated for one hour at 37 °C and transferred onto the cell monolayer. Virus related cytopathic effects (CPE) were determined microscopically 48–72 hours post infection. To determine a potential neutralizing ability of patient serum, CPE at a sample dilution of 1:10 is defined as non-protective while a CPE at a dilution of ≥ 1:20, is defined as positive and protective, respectively.

Results

When calculating the overall sensitivity we used the total time frame of 49 days after first PCR-positivity and focussed on the different antigens (S- or N-antigen) used as binding antigen(s) in the assays. For the examined follow-up SARS-CoV-2 samples (TABLE S1), within the group of N protein-based assays, the Virotech SARS-CoV-2 ELISA IgG demonstrated a sensitivity of 66.7% (30/45), the Elecsys Anti-SARS-CoV-2 of 75.6% (34/45) and the SARS-CoV-2-IgG (Abbott) of 77.8% (35/45). Within the group of S protein-based assays, the Anti-SARS-CoV-2-ELISA (IgG) (Euroimmun) showed a sensitivity of 71.1% (32/45) and the Liaison® SARS-CoV-2 S1/S2 IgG of 75.6 (34/45). The S and N protein-based COVID-19 VIRCLIA® IgG MONOTEST demonstrated a sensitivity of 91.1% (41/45). The in-house developed PRNT showed the highest sensitivity with 93.3%. Regarding the specificity of the examined assays for the non-SARS-CoV-2 samples (TABLE S2), the Elecsys Anti-SARS-CoV-2 generated one positive result within the negative control cohort. The Anti-SARS-CoV-2-ELISA (IgG) (Euroimmun) generated two equivocal results, one for one HCoV-OC43 sample and one within the negative control cohort. The PRNT generated one positive result for one HCoV-OC43, and equivocal results for one HCoV-229E and two samples from the negative control cohort. With exception of the Anti-SARS-CoV-2-ELISA (IgG) (Euroimmun), all examined assays generated positive results (the PRNT only one) for the two SARS-CoV samples from the 2003 outbreak. None of the other non-SARS-CoV-2 samples cross-reacted in any other of the examined assays (Table 2 ).

Table 2

Sensitivity and specificity of the examined SARS-CoV-2 assays.

Assay	SARS-CoV-2 antigen (recombinant)	Sensitivity (%)	Specificity (%)	Specificity (%) incl. SARS-CoV (2003)**
SARS-CoV-2-IgG (Abbott)	N protein	77.8 (35/45)	100 (35/35)	94.6 (35/37)
Elecsys Anti-SARS-CoV-2*	N protein	75.6 (34/45)	97 (33/34)	91.7 (33/36)
Liaison® SARS-CoV-2 S1/S2 IgG	S1 and S2 protein	75.6 (34/45)	100 (35/35)	94.6 (35/37)
COVID-19 VIRCLIA® IgG MONOTEST	S1 and N protein	89 (40/45)	100 (31/31)	93.9 (31/33)
Anti-SARS-CoV-2-ELISA (IgG) (Euroimmun)	S1 protein	71.1 (32/45)	100 (20*/20)	100 (22/22)
Virotech SARS-CoV-2 ELISA IgG	N protein	66.7 (30/45)	100 (35/35)	94.6 (35/37)
PRNT (in-house developed)	whole virus	93.3 (42/45)	97.1 (34/35)***	94.6 (35/37)****

two equivocal results (HCoV-OC43, negative control cohort) were considered as negative.

including follow up samples of SARS-CoV (2003 outbreak), which is closely related to SARS-CoV-2.

three equivocal results (one HCoV-229E sample, two in the negative control cohort were considered negative).

one equivocal result for one SARS-CoV sample was considered negative.

The signal-to-cut-off (S/CO) ratios of examined assays and the corresponding PRNT titers for the tested SARS-CoV-2 follow-up samples are shown in Fig. 1 . With some exceptions, the assays generated consistent results. For sample 2 only the COVID-19 VIRCLIA® IgG MONOTEST and for sample 42 only the COVID-19 VIRCLIA® IgG MONOTEST and the PRNT generated a positive result. For samples 37, 38 and 39 only the S protein-based assays Liaison® SARS-CoV-2 S1/S2 IgG, the COVID-19 VIRCLIA® IgG MONOTEST and PRNT generated a positive result. In contrast, for the follow-up samples 40 and 41, both taken 42 days after PCR-diagnosed with SARS-CoV-2, only the N protein-based assays SARS-CoV-2-IgG (Abbott), the Elecsys Anti-SARS-CoV-2, COVID-19 VIRCLIA® IgG MONOTEST and PRNT detected antibodies, none of the S protein-based assays were able to detect antibodies. Irrespective of the used antigen, both ELISA assays did not generate a positive result for samples 37–41 and some other samples, indicating a lower sensitivity than the automated assays and PRNT. In samples 6 and 35, none of the examined assays detected antibodies. Overall, the generated titers by the examined assays have been shown to positively correlate with neutralizing antibodies in the PRNT.

Fig. 1

Titers for the examined assays: (a) SARS-CoV-2-IgG (Abbott); (b) Elecsys Anti-SARS-CoV-2; (c) Virotech SARS-CoV-2 ELISA IgG; (d) Anti-SARS-CoV-2-ELISA (IgG) (Euroimmun); (e) Liaison® SARS-CoV-2 S1/S2 IgG; (f) COVID-19 VIRCLIA® IgG MONOTEST; (g) PRNT Titer for tested samples; 1 = samples from one individual, 2 + 3 = samples from two different individuals; bold horizontal lines show assay specific cut-off.

Discussion

NAT is the method of choice in the acute or early phase of infection with SARS-CoV-2, while the detection of antibodies, e.g. IgA or IgM is not. In this study we focused on the detection of IgG antibodies, as they are of major interest in determining the SARS-CoV-2 seroprevalence in the general population (herd immunity) or in special cohorts. Furthermore, there is a demand in testing individuals with asymptomatic or mild clinical course who present late (weeks) after disease onset. For these purposes, there is a demand for (cost-effective) high-throughput assays, which can be automated and used for large sample sizes. The sensitivity of these assays depends on the used assay and moment of testing in the infection phase (low sensitivity a couple of days after infection vs. higher sensitivity a couple of weeks after infection [5,6]. The commercially available assays examined in our study, generated consistent results regarding the detection of SARS-CoV-2-IgG antibodies. The sensitivity (without differentiating the timepoint of sampling) varied within the group of assays using the same antigen as target for the antibodies. While the majority of antibodies are typically produced against the N-protein (which therefore might be the most sensitive target protein), antibodies produced against the S-protein are expected to be more specific and potentially neutralizing.

In the group of N protein-based assays the sensitivity varied from 66.7 to 77.8% and in the S protein-based assays from 71.1 to 75.6%. This might be due to differences within the used recombinant antigen and/or is a system-inherent feature. The dual target (S1 and N protein-based) assay for the Vircell VIRCLIA® automation system and the PRNT demonstrated the highest sensitivity with 89% and 93.3%, respectively. There is a large discrepancy in the determined sensitivities for the assays examined in our study to the sensitivities according to the manufacturers’ specifications and the data described in literature. This is not because of the small examined sample size, but because overall sensitivities (not differentiating between the time-points after positive PCR-testing) were given in this study. This was done for a better comparability of the examined assays in terms of demonstrating the differences in the used antigens of the assays on its ability to detect antibodies, independent from the time point of sampling. As gold standard, the PRNT is hands on- and time-intensive and can only be performed for smaller sample sizes in a BSL-3 laboratory. However, it is capable to detect neutralizing antibodies. In our study, the antibody titers generated with the commercially available assays correlated well with the PRNT titers. The mechanism of immunity, especially of protective immunity (if applicable) and how long it will last, need to be further investigated. A titer needed for potential protective immunity is not yet (officially) defined. Besides humoral mediated immunity, there is evidence that T-cell mediated immunity plays a role [7]. Interestingly, in samples of three individuals with mild clinical course of COVID-19, examined in our study (1, 2, 3 in FIG1), together with the COVID-19 VIRCLIA® IgG MONOTEST and PRNT, only the Liaison® SARS-CoV-2 S1/S2 IgG, using recombinant S protein(s) as antigens, detected antibodies in samples number 37, 38 and 39. In contrast, only the SARS-CoV-2-IgG (Abbott) and Elecsys Anti-SARS-CoV-2 (Roche), both using the recombinant N protein as antigens, detected antibodies in samples number 40 and 41. Both tested ELISA assays, with the Anti-SARS-CoV-2-ELISA (IgG) (Euroimmun) using the S protein and the Virotech SARS-CoV-2 ELISA IgG the N protein, failed to detect antibodies in each of these samples. It is important to notice that the Elecsys Anti-SARS-CoV-2 (Roche) does not differentiate between IgA, IgM or IgG antibodies, the positive result might be generated due to reactive, non-IgG antibodies. Despite the small sample size of the analysed cohort, there seem to be individual differences in the time frame and the target (S or N antigen) of the immune response against SARS-CoV-2. This might be due to the fact that the majority of produced antibodies are against the most abundant protein of a virus, in this case the N protein [8]. This need to be further analysed, as the scale of this phenomenon is currently not well known.

Regarding specificity, cross-reactivity of antibodies of endemic coronavirus infected individuals or of individuals with other active infectious diseases (e.g. EBV or CMV) are known phenomenons [9]. The assays examined in our study demonstrated a high specificity with only one false positive generated result within the negative control cohort in the Elecsys Anti-SARS-CoV-2 (Roche) and one positive result in the PRNT with one HCoV-OC43 sample. The cross-reactivity of the SARS-CoV samples from the outbreak of 2003 in the examined assays are of less importance as the virus is known to be eradicated. Nonetheless, as a false positive result might give a false sense of security, efforts should be made to further improve the specificity of the available assays. Talking about the specificity of these assays, it is important to keep in mind, that even an excellent specificity (e.g. > 99%) the positive predictive value (fraction of truly positive individuals) of an assay depends on the prevalence of the tested disease. Screening a population for a disease with a very low or unknown prevalence, a disproportionate number of false positive results are generated. To sum up, all in this study examined SARS-CoV-2 antibody assays are eligible in the detection of SARS-CoV-2 specific antibodies. The automated immunoassays demonstrated a higher overall sensitivity than the ELISA based assays. Especially the assay using the S and N protein as antigens showed the highest sensitivity within the group of commercially available assays examined in this study (including samples with individual characteristics). The titers generated with these assays correlated well with the PRNT, demonstrating the neutralizing capacity of detected antibodies. Because of the low prevalence of SARS-CoV-2 at the moment, these assays are currently primarily eligible for epidemiological investigations, as they are only of limited informative value in individual testing.

Authors’ contributions

NK and HR designed the study. NK, CR and SW performed experiments. NK, HR and SC analyzed data. NK and HR wrote the manuscript.

Declaration of Competing Interest

Sandra Ciesek received a speaker’s fee from Euroimmun. The other authors declare no conflicts of interest.