Performance of the rapid high-throughput automated electrochemiluminescence immunoassay targeting total antibodies to the SARS-CoV-2 spike protein receptor binding domain in comparison to the neutralization assay.

BACKGROUND: Neutralization tests (NT) are the gold standard for detecting and quantifying anti-SARS-CoV-2 neutralizing antibodies (NAb), but their complexity restricts them to research settings or reference laboratories. Antibodies against S protein receptor binding domain (RBD) have been shown to confer a neutralizing activity against SARS-CoV-2. Assays quantitatively measuring anti-S1-RBD-SARS-CoV-2 antibodies could be of great value for NAb screening of potential donors for convalescent-phase plasma therapy, assessing natural or vaccine-induced immunity, stratifying individuals for vaccine receipt, and documenting vaccine response.
METHODS: Elecsys Anti-SARS-CoV-2 S (Elecsys-S), a high-throughput automated electrochemiluminescence double-antigen sandwich immunoassay for quantitative measurement of pan-anti-S1-RBD-SARS-CoV-2 antibodies, was evaluated against NT on 357 patients with PCR-confirmed SARS-CoV-2 infection. NT was performed in a BSL-3 laboratory using a Slovenian SARS-CoV-2 isolate; the NT titer ≥1:20 was considered positive.
RESULTS: Elecsys-S detected pan-anti-S1-RBD-SARS-CoV-2 antibodies in 352/357 (98.6 %) samples. NAb were identified by NT in 257/357 (72 %) samples. The Elecsys-S/NT agreement was moderate (Cohen's kappa 0.56). High NT titer antibodies (≥1:160) were detected in 106/357 (30 %) samples. Elecsys-S's pan-anti-S1-RBD-SARS-CoV-2 antibody concentrations correlated with individual NT titer categories (the lowest concentrations were identified in NT-negative samples and the highest in samples with NT titer 1:1,280), and the Elecsys-S cutoff value for reasonable prediction of NAb generated after natural infection was established (133 BAU/mL).
CONCLUSION: Although NT should remain the gold standard for assessing candidates for convalescent-phase plasma donors, selected commercial anti-SARS-CoV-2 assays with optimized cutoff, like Elecsys-S, could be used for rapid, automated, and large-scale screening of individuals with clinically relevant NAb levels as suitable donors.

Introduction

Accurate assays for detecting antibodies against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) are needed to inform diagnostic, therapeutic, and public health decisions [1,2]. Many commercial and laboratory-developed anti-SARS-CoV-2 assays are available to measure neutralizing and non-neutralizing anti-SARS-CoV-2 antibodies against a range of viral proteins/isotopes with various analytical techniques [3,4].

Neutralization assays are the gold standard for detecting and quantifying anti-SARS-CoV-2 neutralizing antibodies (NAb), mainly used in research settings or offered as laboratory-developed tests by reference laboratories [3]. Antibodies against the receptor binding domain (RBD) of the S1 subunit of the spike (S) protein have shown neutralizing activity against SARS-CoV-2 [5]. Assays quantitatively measuring anti-S1-RBD-SARS-CoV-2 antibodies are very promising for screening potential donors for convalescent-phase plasma therapy, assessing natural or vaccine-induced immunity, stratifying individuals for vaccine receipt, and documenting vaccine response, which could inform return-to-work and travel decisions and other public health measures [1,6].

Elecsys Anti-SARS-CoV-2 S (Elecsys-S) (Roche Diagnostics, Mannheim, Germany) is a recently launched rapid (18-minute) high-throughput automated electrochemiluminescence double-antigen sandwich immunoassay for quantitative measurement of pan-anti-S1-RBD-SARS-CoV-2 antibodies. Elecsys-S was launched in Europe in September 2020 and received FDA EUA on November 25, 2020. The assay has been extensively evaluated by the manufacturer, showing 100.00 % (95 % confidence interval (CI), 99.7–100 %) analytical specificity on 1,100 samples, 99.98 % (95 %CI, 99.91–100 %) clinical specificity on 5991 samples, and 98.8 % (95 %CI, 98.1–99.3 %) clinical sensitivity on 1423 samples obtained 14 days or later after SARS-CoV-2 PCR-confirmation. The only published Elecsys-S evaluation shows clinical specificity of 99.8 % (95 %CI, 99.4–99.9 %) on 1159 pre−COVID-19 samples and 97.6 % (95 %CI, 93.2–99.1 %) clinical sensitivity on 125 sera taken 43–144 days after symptom onset [7].

Here we present the first manufacturer-independent performance evaluation of Elecsys-S against a neutralization test assessed on 357 patients with PCR-confirmed SARS-CoV-2 infection.

Material and methods

The study population comprised 357 anonymized plasma samples collected during July-November 2020 from 357 Slovenian patients with PCR-confirmed SARS-CoV-2 infection. In all patients, SARS-CoV-2 infection was confirmed by cobas 6800 SARS-CoV-2 (Roche Diagnostics) [8]. Blood samples were collected from 35 days up to 6 months after symptom onset or first PCR positivity.

To determine the concentration of pan-anti-S1-RBD-SARS-CoV-2 antibodies, Elecsys-S was performed on a cobas e411 analyzer, using the manufacturer’s cut-off value for positive result of ≥0.8 U/mL. The manufacturer reports a linear measurement range from 0.4 to 250 U/mL, which can be expanded by dilution. Recently, World Health Organization (WHO) introduced International Standard for anti-SARS-CoV-2 immunoglobulin [9]. The neat sample was assigned to contain 1000 binding antibody units (BAU)/mL. A mathematical transposition of Elecsys-S specific U to BAU follow the equation: Elecsys-S U = 0.972 x BAU. Since transposition results in minimal numeric changes only that are well within the range of typical assay variance, according to the manufacturer’s recommendations Elecsys-S U/mL were considered to be equivalent to the BAU/mL of the WHO standard, and all Elecsys-S results were expressed in BAU/mL.

A neutralization test (NT) was performed in a BSL-3 laboratory using SARS-CoV-2 virus isolated from a Slovenian patient in April 2020 (strain Slovenia/SI-4265/20, D614 G; EVA-GLOBAL-Ref-SKU: 005V-03961; B.1.1 Pango lineage). Pango lineage B.1.1 was the most common SARS-CoV-2 genetic variant present in Slovenia from March to October 2020 thus we consider strain Slovenia/SI-4265/20 as reference strain for our study population. NT was based on the standardized protocol [10]. Briefly, Vero E6 cells were seeded in a concentration of 105/well (96-well plate) 1 day before NT was performed. Serial dilutions of heat-inactivated plasma samples (56 °C, 30 min) were incubated with 100 TCID50 of SARS-CoV-2 for 1 h at 37 °C. Each plasma dilution-virus mixture was added to the cells in triplicate and incubated at 37 °C with 5 % CO2 for 4 days. The neutralization endpoint titer was determined as the endpoint plasma dilution that inhibited the SARS-CoV-2–induced cytopathic effect in at least 2 out of 3 parallels. The NT titer ≥1:20 was considered positive.

Correlation between Elecsys-S and NT was assessed using an ordinal one-way ANOVA followed by post-hoc Bonferroni’s multiple comparison test [11].

Further, we tested Elecsys-S’s ability to differentiate between plasma samples with high NT antibody titer (≥1:160) and low titer (<1:160). The 1:160 cutoff was based on the NAb titer for COVID-19 convalescent-phase plasma therapy recommended by the FDA [12]. The 357 samples were randomized into two groups: 184 were used to develop a univariate logistic regression model with logarithmic Elecsys-S values as a predictive variable and high/low NT titer as dependent variable, and 173 for model validation. A chi-squared test was performed to identify any significant difference between the groups. To establish the optimal cutoff value, a receiver-operating-characteristic (ROC) curve was plotted in the validation group. The Elecsys-S cutoff value was determined by identifying an inflection point on the validation ROC curve, corresponding to a combination of high specificity (>84 %) and sensitivity (>74 %). We chose higher specificity compared to sensitivity to develop an algorithm for a sufficiently high probability of high NT titer with a lower number of samples chosen.

A contingency table used validation data to assess overall agreement with 95 % CIs. Agreement between both tests was assessed using Cohen’s kappa statistics. All statistical analyses used Excel (Microsoft, Redmond, WA) and GraphPad Prism version 8 (GraphPad Software, San Diego, CA).

Results

Out of 357 samples, 352 (98.6 %) tested positive for pan-anti-S1-RBD-SARS-CoV-2 antibodies; all five Elecsys-S-negative samples were NT-negative. NAb were detected by NT in 257/357 (72 %) samples (Table 1 ). The agreement between Elecsys-S and NT was moderate (Cohen’s kappa 0.56; 95 %CI, 0.42–0.69). All but one NT-negative sample (99/100) tested positive using Elecsys Anti-SARS-CoV-2 (Elecsys-N), a pan-anti-SARS-CoV-2 assay targeting the SARS-CoV-2 nucleoprotein (N) [13]; the mean Elecsys-N value was 36 (range 0.9–166).

Table 1

Mean, minimum, and maximum concentrations of pan-anti-S1-RBD-SARS-CoV-2 antibodies (BAU/mL) measured by the Elecsys Anti-SARS-CoV-2 S assay in plasma samples of 357 patients with PCR-confirmed SARS-CoV-2 infection across different neutralization antibody titer (NT titer) categories.

	Anti-SARS-CoV-2 S (BAU/mL)
NT (titer)	N	Mean	Minimum	Maximum
<1:10	100	29.5	0.40	248.3
1:20	49	66.5	4.77	468.8
1:40	50	126.5	2.55	821.5
1:80	52	243.3	3.18	1936
1:160	44	470.0	1.45	2223
1:320	27	584.2	8.61	2445
1:640	23	527.8	12.84	1906
1:1280	12	858.9	139.40	3362

TOTAL: 357

High NT titer antibodies (≥1:160) were detected in 106/357 (30 %) samples (Table 1). Pan-anti-S1-RBD-SARS-CoV-2 concentrations significantly differed regarding corresponding NT titer categories (ordinary one-way ANOVA: F (7, 349) = 34.6; p < 0.0001; Fig. 1 ). The lowest pan-anti-S1-RBD-SARS-CoV-2 concentrations were identified in NT-negative samples (post-hoc Bonferroni’s test, p < 0.001), and the highest in samples with NT titer 1:1,280 (Fig. 1, Table 1).

Fig. 1

Correlation between neutralization antibody titers (NT titers) measured by neutralization test and logarithmic values (BAU/mL) of pan-anti-S1-RBD-SARS-CoV-2 antibodies measured by Elecsys Anti-SARS-CoV-2 S assay (Elecsys-S) in plasma samples of 357 patients with PCR-confirmed SARS-CoV-2 infection. Whiskers include data between the 2.5th and 97.5th percentiles. Differences in logarithmic Elecsys-S values were statistically significant among NT titers (ordinary one-way ANOVA: F (7, 349) = 34.6; p < 0.0001). *** = p < 0.001, ns = p > 0.05.

To establish the optimal Elecsys-S cutoff value for prediction of NAb presence, a univariate logistic regression model was developed and validated. As shown in Table 2 , no statistically significant differences in the distribution of low vs. high NT titer samples were found between the model development and model validation groups (p = 0.311). As shown in Fig. 2 , the selected Elecsys-S cutoff of 2.12 log10 BAU/mL or 133 BAU/mL resulted in a specificity of 84 % and sensitivity of 74.5 % for prediction of NAb generated after natural infection. Out of 357 samples tested, 118 (33 %) contained pan-anti-S1-RBD-SARS-CoV-2 antibodies in concentrations above selected Elecsys-S cutoff (Fig. 3 ).

Table 2

Distribution of plasma samples with high neutralization (NT) antibody titer (≥1:160) and low NT antibody titer (<1:160) in model development (N = 184) and model validation (N = 173) groups of samples used for development and validation of the univariate logistic regression model, respectively.

	Sample group
NT titer	Model development	Model validation	TOTAL
≥1:160	59 (32 %)	47 (27.2 %)	106 (29.7 %)
<1:160	125 (68 %)	126 (72.8 %)	251 (70.3 %)
TOTAL	184 (100 %)	173 (100 %)	357 (100 %)

Fig. 2

Receiver-operating-characteristic (ROC) curve representing the discriminating ability of the logistic model on validation data with area under the curve (AUC) 0.82 (95 % CI 0.75–0.90). The red dot represents an inflection point on the ROC curve to gain optimal discriminative measures (i.e., specificity >84 % and sensitivity >74.5 %). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 3

Distribution of logarithmic values (U/mL) of pan-anti-S1-RBD-SARS-CoV-2 antibodies measured by the Elecsys Anti-SARS-CoV-2 S assay (Elecsys-S) in plasma samples of 357 patients with PCR-confirmed SARS-CoV-2 infection. Black dots represent Elecsys-S positive results (N = 352) and red dots negative results (N = 5) using the Elecsys-S manufacturer’s cutoff (≥0.8 BAU/mL). The red dotted line represents the selected Elecsys-S cutoff of 2.12 log10 BAU/mL or 133 BAU/mL for prediction of the presence of neutralizing antibodies generated after natural infection. Out of 357 samples tested, 118 (33 %) were above the selected Elecsys-S cutoff. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Conclusions

As far as we know, apart from limited manufacturer’s data, for which Elecsys-S showed positive agreement of 92.3 % (95 %CI, 63.97–99.81 %) for 15 samples compared to a vesicular stomatitis virus-based pseudo-neutralization assay [14], there are no data in peer-reviewed literature concerning Elecsys-S performance against neutralization assay(s).

This study showed that, among Elecsys-S positive convalescent-phase individuals, two-thirds contained measurable NAb levels (≥1:20) and one-third in high titers (≥1:160). Pan-anti-S1-RBD-SARS-CoV-2 antibody concentrations correlated with individual NT titer categories, and an Elecsys-S cutoff value for reasonable prediction of NAb presence after natural infection was established (133 BAU/mL). If this Elecsys-S cutoff value for reasonable prediction of NAb presence is also applicable in post-vaccination sera needs further research.

Despite substantial interest, the results of randomized trials or matched treatment–control studies of convalescent-phase plasma therapy remain inconclusive [3,15]. However, in a recent study of 3082 hospitalized patients not requiring mechanical ventilation, a dose-dependent effect relative to donors’ NAb titers was observed, with higher NAb titers associated with a lower risk of death [15]. These again emphasize a need for exact quantification of NAb in donors’ plasma. Our results confirmed a recent suggestion that, although NT should remain the gold standard for assessing convalescent-phase plasma donors, selected commercial anti-SARS-CoV-2 assays with an optimized cutoff, like Elecsys-S, could be used to maximize the positive predictive value (PPV), making it possible to select individuals with clinically relevant NAb levels as suitable donors [3,16,17].

Previous studies showed that concomitant use of two different anti-SARS-CoV-2 assays improves the PPV compared with an individual assay alone for identifying potential convalescent-phase plasma donors while maintaining the negative predictive value [1,[17], [18], [19]]. This study showed that Elecsys-S with an optimized cutoff can provide such information in less than 20 min without the need to be backed with another anti-SARS-CoV-2 assay. In addition, the high throughput of cobas e analyzers allows large-scale screening of potential donors if desirable [20].

Funding

This study was funded by the Institute of Microbiology and Immunology, , , by the (grants P3-0083 and V3-2034), and by the European Virus Archive Global (EVAg) project, which has received funding from the European Union Horizon 2020 research and innovation program under grant agreement no. 653316.

CRediT authorship contribution statement

Katarina Resman Rus: Methodology, Validation, Data curation, Investigation, Writing - original draft, Writing - review & editing. Miša Korva: Methodology, Validation, Writing - review & editing. Nataša Knap: Methodology, Validation, Writing - review & editing. Tatjana Avšič Županc: Methodology, Validation, Writing - review & editing, Supervision. Mario Poljak: Conceptualization, Methodology, Writing - original draft, Writing - review & editing, Supervision.

Declaration of Competing Interest

The authors declare no conflicts of interest.