Comparison of the Anti-SARS-CoV-2 Surrogate Neutralization Assays by TECOmedical and DiaPROPH-Med with Samples from Vaccinated and Infected Individuals.

Anti-SARS-CoV-2-specific serological responses are a topic of ongoing evaluation studies. In the study presented here, the anti-SARS-CoV-2 surrogate neutralization assays by TECOmedical and DiaPROPH -Med were assessed in a head-to-head comparison with serum samples of individuals after vaccination as well as after previous infection with SARS-CoV-2. In case of discordant results, a cell culture-based neutralization assay was applied as a reference standard. The TECOmedical assay showed sensitivity and specificity of 100% and 61.3%, respectively, the DiaPROPH-Med assay 95.0% and 48.4%, respectively. As a side finding of the study, differences in the likelihood of expressing neutralizing antibodies could be shown for different exposition types. So, 60 of 81 (74.07%) of the samples with only one vaccination showed an expression of neutralizing antibodies in contrast to 85.71% (60 of 70 samples) of the samples with two vaccinations and 100% (40 of 40) of the samples from previously infected individuals. In conclusion, the both assays showed results similar to previous assessments. While the measured diagnostic accuracy of both assays requires further technical improvement of this diagnostic approach, as the calculated specificity values of 61.3% and 48.4%, respectively, appear acceptable for diagnostic use only in populations with a high percentage of positive subjects, but not at expectedly low positivity rates.

1. Introduction

Although protective immunity is considered a key element in order to overcome the COVID-19 pandemic, vaccine-induced immune-protection is not associated with sterile immunity and is prone to be undermined by virus variants [1]. Serological screening assays were rapidly developed along with the global spread of SARS-CoV-2 [2,3]. However, it soon became obvious that the reliability of such diagnostic strategies depends on various factors including uneven geographic distributions of cross-reacting antibodies [4].

Further, it was clear that the functionality of adaptive immunological components is more important than just sole quantity [5]. In line with this, it became evident that a diagnostic focus should be on the neutralizing the potency of detected anti-SARS-CoV-2-antibodies [6,7]. Correlations between the severity of clinical courses of COVID-19 disease and neutralizing antibody titers could be shown [8,9,10,11].

To circumvent the need for laborious and time-consuming tissue culture infecting dose-(TCID-) assays for the assessment of the neutralizing potency of anti-SARS-CoV-2-antibodies, alternative diagnostic strategies were implemented and tested. Those attempts comprised correlations of total and neutralizing anti-SARS-CoV-2-antibodies [12,13,14,15]. In addition, however, specific antigenic target structures were identified in order to develop surrogate neutralization assays for testing for antibodies specific to those critical epitopes [16,17].

In the meantime, numerous cellular and acellular surrogate neutralization assays have been introduced and evaluated [18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45]. Such evaluations of both neutralizing and overall anti-SARS-CoV-2-antibodies comprised both natural infections and post-vaccination states [46,47,48,49,50]. Next to neutralizing potency, antibody avidity was included in the long-term assessments as well [51].

In the study presented here, two surrogate neutralization assays, i.e., the TECO SARS-CoV-2 Neutralization Antibody Assay (TECOmedical AG, Sissach, Switzerland), for which previous assessments had suggested imperfect sensitivity and specificity in the 90% range [24,25], and the DIA-SARS-CoV-2-nAb (DiaPROPH Med, Kiev, Ukraine) surrogate neutralization assay, were compared in a head-to-head assessment with serum samples from vaccinated and infected individuals. The choice of the assays was based on the fact that first evaluation results for the TECOmedical assay were already available [24,25] allowing at least a rough estimation of the credibility of the results of the present study, while no peer-reviewed published data on the DiaPROPH assay were available so far. By applying this approach, this communication intends to contribute to the puzzle of diagnostic accuracy assessment of commercially available serological SARS-CoV-2 assays. Quantitative measurement of overall-anti-SARS-CoV-2 antibodies and avidity testing were comparatively added and contradicting results in the surrogate neutralization assays were finally decided by a cell culture-based neutralization assay.

2. Materials and Methods

2.1. Samples Included in the Assessment

For the assessment, 81 serum samples were obtained after the first vaccination against SARS-CoV-2, 70 serum samples after the second vaccination against SARS-CoV-2 and 40 samples from individuals with a history of natural infections with the SARS-CoV-2 virus. For the vaccinees, who were participants in the COV-ADAPT study [52], anti-SARS-CoV-2-NCP-ELISA (IgG) (Euroimmun, Lübeck, Germany) excluded anti-nucleocapsid-IgGs (NCP) to rule out infection with SARS-CoV-2 before vaccination. Negative results were a prerequisite for being included in the assessment. Vaccinations had been performed with various combinations of Vaxzevria (AstraZeneca, Cambridge, UK) or Comirnaty (Biontech, Mainz, Germany). Infections that occurred during a period when the Wuhan wild type variant was prevalent had been ensured by real-time PCR. Based on the various combinations of possible expositions, three different exposition types, i.e., natural infection, exposure to one vaccination and exposure to two vaccinations, were defined.

2.2. Performed Diagnostic Assessments

All sera were comparatively assessed for neutralizing anti-SARS-CoV-2 antibodies using the surrogate neutralization assays TECO SARS-CoV-2 Neutralization Antibody Assay (TECOmedical AG, Sissach, Switzerland) and DIA-SARS-CoV-2-nAb (DiaPROPH Med, Kiev, Ukraine; distributed by AlphaScience GmbH, Riedstadt, Germany) as described by the manufacturer using the DSX Automated ELISA System (Thermo Labsystems, Chantilly, VA, USA). Both assays are competitive ELISAs. In detail, during the TECO SARS-CoV-2 Neutralization Assay (later also referred to as the “TECOmedical surrogate neutralization assay”), anti-SARS-CoV-2-IgG present in tested sera are bound to a conjugate during preincubation (Figure 1). The conjugate consists of a HRP-(horseradish peroxidase-) labelled receptor binding domain of the viruses’ spike protein. Formed complexes block the conjugate’s interaction with ACE2-receptors on the solid phase. Added tetramethylbenzide (TMB) substrate then reacts with the remaining conjugate that successfully interacted with the mentioned receptors. When read at 450/620 nm, the sample’s OD is inversely correlated with the concentration of present neutralizing antibodies.

Figure 1

Principle of the TECOmedical SARS-CoV-2 Neutralization Antibody ELISA. (A) Serum samples and conjugate (HRP-labelled SARS-CoV-2-SP-RBD) are premixed. (B) During incubation, neutralizing antibodies (anti-SARS-CoV-2-RBD-IgG) and the conjugate form antibody–antigen complexes. (C) The mixture is added to the solid phase; preformed complexes are unable to interact with ACE2-receptors; unbound conjugate can interact freely. (D) Tetramethylbenzide substrate is added and reacts with HRP; the measured optical density is inversely correlated with the concentration of neutralizing antibodies. Yellow square: color change after enzymatic reaction.

In contrast to that, the DIA-SARS-CoV-2-nAb (later also referred to as “DiaPROPH-Med surrogate neutralization assay) makes use of an HRP-labelled recombinant human spike protein as a conjugate, which directly competes with neutralizing antibodies in the test wells (Figure 2). The latter prevents the conjugate’s interaction with recombinant human ACE2-receptors. TMB substrate is again used to detect HRP-labelled spike protein that successfully interacted with the solid phase. When read at 450/620 nm, the sample’s OD is inversely correlated with the concentration of the present neutralizing antibodies.

Figure 2

Principle of the DIA-SARS-CoV-2-nAb assay. (A) Serum samples and conjugate (HRP-labelled recombinant SARS-CoV-2-spike-protein) are simultaneously added to the solid phase. (B) Present neutralizing antibodies (anti-SARS-CoV-2-RBD-IgG) are bound to the conjugate and consequently block the interaction between the receptor-binding domain of the spike protein and recombinant human ACE-2-receptors on the solid phase. (C) Tetramethylbenzide substrate is added and reacts with HRP; the measured optical density is inversely correlated with the concentration of neutralizing antibodies in the sample. Yellow square: color change after enzymatic reaction.

In addition, all samples were subjected to an avidity assessment of anti-SARS-CoV-2 antibodies using the DIA-SARS-CoV-2-S-IgG-av avidity assay (DiaPROPH Med, Kiev, Ukraine; distributed by AlphaScience GmbH, Riedstadt, Germany; later also referred to as “DiaPROPH-Med antibody avidity assay”).

To determine a sample’s avidity, serum is placed in two wells simultaneously, hereafter referred to as the reference well and test well, respectively. In both wells, anti-SARS-CoV-2-IgG present in the sample are bound to immobilized recombinant SARS-CoV-2 spike protein on the solid phase. The test wells are then treated with a dissociation solution, which destroys low avidity antibody–antigen complexes formed in the first step. Remaining complexes in both wells are then detected by an HRP-labelled monoclonal anti-human IgG. TMB is added as the substrate for the enzymatic reaction. When read at 450/620 nm, the reference well’s OD is directly correlated with the concentration of present anti-SARS-CoV-2-IgG. The test well’s OD relative to the reference well’s OD is used to calculate the RAI (Relative Avidity Index). If the reference wells indicate quantitative IgG above a cut-off, results can be interpreted as low avidity (RAI < 40%) and high avidity (>40%). A schematic visualization is provided in Figure 3.

Figure 3

Principle of the DIA-SARS-CoV-2-S-IgG-av avidity assay. (A) Serum samples are added to the solid phase in duplicate (reference wells and test wells); present anti-SARS-CoV-2-RBD-IgG are bound to the antigen (recombinant SARS-CoV-2-spike protein) on the solid phase. (B) The test wells are treated with dissociation solution to destroy antibody-antigen complexes formed by low avidity antibodies. (C) Remaining antibody-antigen complexes are detected by the conjugate (HRP-labelled monoclonal anti-human-IgG). (D) Tetramethylbenzide substrate is added and reacts with HRP; the reference well indicates whether the sample contains a sufficient amount of IgG or not, if the first applies, the test well is used to calculate the RAI (Relative Avidity Index). Yellow square: color change after enzymatic reaction.

Measurement of the total IgG-antibodies directed against the receptor binding domain of the SARS-CoV-2 spike protein (anti-spike-RBD-IgG) were assessed applying the SARS-CoV-2-IgG-II-Quant assay on the Architect i2000SR analyzer (Abbott, North Chicago, IL, USA, later also referred to as “total S-antigen-specific anti-SARS-CoV-2-IgG assay”).

2.3. Tissue Culture Infecting Dose-(TCID-)Based Neutralization Assay

Serum samples that showed discrepant results in the surrogate neutralization assays were additionally tested with a tissue culture infecting dose (TCID)-based neutralization assay. Serum samples were serially diluted and incubated with 250 pfu/mL SARS-CoV-2 virus for 1h at room temperature. Antibody-virus mixtures were added to 96-well plates containing monolayers of Vero E6 cells (ATCC CRL-1586). Cells were cultured at 37 °C until cytopathic effect was observed. Neutralization titers were expressed as TCID. Titers ≤ 1:4 were considered negative.

2.4. Statistical Assessments

Both surrogate neutralization assays were tested against a composite reference standard consisting of their combined results. In detail, samples with concordantly positive results in both surrogate neutralization assays were considered as “truly” positive for neutralization antibodies, samples with concordantly negative results as “truly” negative. In case of discordant results of the composite reference standard, the TCID-(tissue culture infecting dose-)based neutralization assay was applied for confirmatory testing. Its results were considered as definite. Based on those assumptions, diagnostic accuracy values were calculated for both surrogate neutralization assays. To ensure hardest possible assessment conditions, “indetermined” results were counted as “negative” for the sensitivity assessment and as “positive” for the specificity assessment.

In addition, it was assessed whether different exposition types either to SARS-CoV-2 or to vaccination were associated with an increased or a decreased likelihood of expressing neutralizing antibodies. Further, the association of exposition types and antibody avidity values was assessed. Finally, correlation between titer levels of total S-antigen-specific anti-SARS-CoV-2 S-antibody titers on the one hand and antibody avidity values on the other hand with the levels of neutralizing antibodies as measured with both surrogate neutralization assays was assessed applying Spearman’s correlation coefficient.

Considering the low number of included samples, assessments were restricted to descriptive statistics and simple statistic approaches like Fisher’s exact test, Kruskal–Wallis rank sum testing and Wilcoxon rank sum testing.

3. Results

3.1. Diagnostic Accuracy for Both Surrogate Neutralization Assays

Based on the abovementioned assumptions on “true positivity” and “true negativity” for the surrogate neutralization assays, calculated sensitivity of the TECOmedical assay and the DiaPROPH-Med assay were 100% and 95.0%, respectively, while specificity was 61.3% and 48.4%, respectively. Details are provided in Table 1. In addition, the results of the Abbott total S-antigen-specific anti-SARS-CoV-2 IgG were given. For anti-spike IgG, a protective cut-off of 50 BAU/mL has been established by the World Health Organization (WHO). Cohen’s kappa (95% CI) between the gold standard used in this study, based on concordance of surrogate neutralization assays and TCID assay results against the protective BAU/mL is 0.440 (0.274–0.605). The sensitivity and specificity of the Abbott total S-antigen-specific anti-SARS-CoV-2 IgG protective BAU/mL with respect to the standard used here are 88.8 and 58.1, respectively.

Table 1

Diagnostic accuracy as calculated for the assessed surrogate neutralization assays.

Assay	n	Overall Positives n (%)	Correct Positives (Sensitivity n% (with 0.95 CI))	Overall Negatives n (%)	Correct Negatives (Specificity n% (with 0.95 CI))	Indetermined n (%)
TECOmedical	191	172 (90.05)	160 (100%, (n.e.))	19 (9.95)	19 (61.29%, (43.34, 76.62))	0
DiaPROPH Med	191	166 (86.91)	152 (95.00%, (90.28, 97.49))	22 (11.52)	15 (48.39%, (31.57, 65.57))	3 (1.57)
Total S-antigen-specific anti-SARS-CoV-2-IgG (protective BAU/mL)	191	155 (81.15)	142 (88.75, (82.81, 92.82))	36 (18.85)	18 (58.06, (40.31, 73.95))	0

95% CI = 95% confidence interval. n.e. = not estimable. n = total numbers.

3.2. Associations between Exposition Types and the Abundance of Neutralizing Antibodies Based on the Results of the Composite Reference Standard as well as Antibody Avidity and Correlation Assessments

Three types of exposition, i.e., one received vaccination, two received vaccinations, and natural SARS-CoV-2 infection were comparatively evaluated regarding an increased or a decreased likelihood of expressing neutralizing antibodies as well as with the levels of antibody avidity as measured with the DiaPROPH-Med avidity assay.

There was an association between exposition type and the overall likelihood of expressing neutralizing antibodies. While only 60 of 81 (74.07%) of the samples with only one vaccination showed an expression of neutralizing antibodies, this was the case for 85.71% (60 of 70 samples) of the samples from individuals with two vaccinations and 100% (40 of 40) of the samples from previously infected individuals. This association was statistically significant (Fisher’s exact test, p value < 0.001).

There was also a difference between the levels of antibody avidity as measured with the DiaPROPH-Med avidity assay for the samples from individuals with one and two vaccinations as well as from already infected individuals. Also, differences were seen for the association of exposition type and neutralizing antibodies as measured by the surrogate neutralization assays with best neutralizing effects after two vaccines. The details on the avidity levels and surrogate neutralization levels by exposition type are indicated in Table 2.

Table 2

Association of exposition types, antibody avidity as well as neutralizing effects of the measured antibodies.

Assay	Exposition Type	n	Mean (SD)	Median (Min, Max)	p Value
DiaPROPH-Med antibody avidity assay	One vaccination	81	33.58 (25.14)	30.84 (0, 96.83)	0.0001
	Two vaccinations	70	85.19 (11.14)	86.76 (59.24, 105.75)
	Previously SARS-CoV-2 infected	40	58.28 (27.08)	58.70 (0, 117.40)
TECOmedical surrogate neutralization assay	One vaccination	81	44.07 (27.63)	48.30 (0, 91.30)	0.0001
	Two vaccinations	70	98.27 (8.35)	99.70 (31.70, 99.90)
	Previously SARS-CoV-2 infected	40	80.35 (14.71)	81.05 (44.30, 99.30)
DiaPROPH-Med surrogate neutralization assay	One vaccination	81	77.60 (19.22)	81.67 (2.05, 96.26)	0.0001
	Two vaccinations	70	78.34 (31.33)	93.97 (0, 99.14)
	Previously SARS-CoV-2 infected	40	72.28 (17.06)	75.23 (0, 94.91)

SD = standard deviation. Min. = minimum. Max. = maximum. n = total numbers.

Because a normal distribution could not be assumed for the recorded avidity levels with Shapiro testing indicating p values of 0.0298, 0.00061, and <0.00001 for samples from individuals with one vaccination, 0.11161, <0.0001, and <0.00001 for samples from individuals with two vaccinations and 0.98165, 0.02723, and <0.00001 for samples from previously infected individuals, Kruskal–Wallis rank sum testing was conducted. Post-hoc conducted Wilcoxon rank sum testing indicated statistical significance for all two-group-comparisons (one vaccination vs. two vaccinations with p values < 0.0001, <0.0001, and 0.0002, one vaccination vs. COVID-19 infection with p-values of <0.0001, <0.0001, and 0.0055, two vaccination vs. COVID-19 infection with p-values of <0.0001 for all three tests).

Finally, correlation between anti-spike-RBD-IgG titers on the one hand and antibody avidity values on the other hand with the levels of neutralizing antibodies as measured with both surrogate neutralization assays was assessed using Spearman’s correlation coefficient. The measured coefficients ranged from 0.19 to 0.88. All pairwise correlations were positive, indicating that higher titers in one assay were associated with higher titers in the other assay. The p-values of all calculated correlation coefficients were less than 0.05. Details are provided in Table 3.

Table 3

Spearman’s correlation coefficient indicating correlation between levels of anti-spike-RBD-IgG titers on the one hand and antibody avidity values on the other hand with the levels of neutralizing antibodies as measured with both surrogate neutralization assays.

Spearman’s Rho N p Value	TECOmedical Surrogate Neutralization Assay	DiaPROPH-Med Surrogate Neutralization Assay	DiaPROPH-Med Antibody Avidity Assay	Total S-Antigen- Specific anti-SARS-CoV-2-IgG
TECOmedical surrogate neutralization assay	1
DiaPROPH-Med surrogate neutralization assay	0.3101 191 <0.0001	1
DiaPROPH-Med antibody avidity assay	0.6980	0.1859	1
	191	191
	<0.0001	0.0100
Total S-antigen-specific anti-SARS-CoV-2-IgG	0.8844	0.2652	0.5790	1
	191	191	191
	0.0001	0.0002	<0.0001

4. Discussion

The evaluation was conducted to comparably assess the diagnostic accuracy of two surrogate neutralization assays for the estimation of the neutralizing potential of anti-SARS-CoV-2 antibodies. The assessment confirmed good sensitivity of the TECOmedical assay, while its specificity scored worse than in previous assessments [24,25]. The DiaPROPH-Med surrogate neutralization test performed slightly worse in direct comparison in terms of its sensitivity and specificity, but has a more balanced price-performance ratio. Accordingly, respective diagnostic results should be interpreted with care. Of note, indetermined results as indicated by the DiaPROPH-Med surrogate neutralization assay were interpreted in the strictest possible way to ensure a thorough assessment. So, “indetermined” was interpreted like “negative” for the sensitivity assessment and like “positive” for the specificity assessment. However, the sensitivity of both surrogate neutralization assays is markedly higher than considering protective BAU/mL of quantitative total S-antigen-specific anti-SARS-CoV-2 IgG testing.

As a side finding, it could be shown with both assays that the abundance of neutralizing antibodies was increased in individuals after two vaccines as well as in individuals with natural infection compared to individuals with just one vaccine. This finding is not surprising, as vaccination trials have shown increased neutralizing potential after repeated vaccination [53]. For maturation-related avidity testing, the findings pointed in similar direction. Finally, correlation analysis of anti-spike-RBD-IgG-antibodies, neutralizing antibodies, and antibody avidity did not lead to conclusive associations allowing reliable predictions, although positive correlation was shown for the most parameters.

The evaluation has a number of limitations. First, only a moderate number of samples could be included in the assessment. Second, cell culture-based neutralization assays could only be run in case of contradicting results due to organizational reasons and funding restrictions.

5. Conclusions

The assessment confirmed acceptable diagnostic sensitivity for both surrogate neutralization assays. In contrast, further technical improvement of both assays seems advisable to increase their specificity and so, their results should be interpreted with care. Nevertheless, surrogate neutralization assays appear superior compared with the assessment of protective BAU/mL based on quantitative total S-antigen-specific anti-SARS-CoV-2 IgG detection.