Diagnostic accuracy of three SARS-CoV2 antibody detection assays, neutralizing effect and longevity of serum antibodies.

Evidence is currently insufficient to know whether SARS-CoV-2 antibodies (Abs) protect from future infection and how long immunity will last. The kinetics of the immune response to SARS-CoV-2 infection and role of serology in estimating individual protective immunity is yet to be established. We evaluated diagnostic performances of three serological assays - Abbott Architect CMIA IgG, bioMerieux VIDAS ELFA IgG/IgM, and Diesse Chorus ELISA IgG/IgM, and analyzed longevity and potential neutralizing effect of SARS-CoV-2 Abs in COVID-19 patients. Clinical sensitivities of assessed IgG tests two to three weeks post symptom onset (PSO) were very high: 96.77 % for Architect, 96.77 % for Chorus, and 100.00 % for VIDAS. Sensitivities of two assessed IgM assays were moderate: 74.07 % for Chorus, and 76.92 % for VIDAS. Specificities were excellent for all assessed IgG assays: 99.01 % for Architect and 100 % for Chorus and VIDAS. Chorus and VIDAS IgM assays also achieved excellent specificity of 99.01 % and 100 %, respectively. In most cases IgG Abs were still present eight months PSO. Neutralizing antibodies were detected in majority of serum samples from convalescent patients. Serum samples from severe COVID-19 patients had higher antibody titers and higher neutralizing activity. We observed a strong positive correlation among SARS-CoV-2 IgG antibody titer and neutralizing activity. The strongest positive correlation to neutralizing activity was found for VIDAS IgG assay.

Introduction

In response to the coronavirus disease 2019 (COVID-19) pandemic diagnostics rapidly expands and improves in quality. For judicious application of serology tests, it is essential to understand their performance characteristics and limitations. During the last few months several commercial severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) serological assays with CE-IVD mark have been proposed in the EU market. However, validation studies have not been published for many of them (Wolff et al., 2020; Chew et al., 2020; Manalac et al., 2020). Therefore, a careful validation of diagnostic assay is a priority.

Growing evidence is emerging about the protective effect of SARS-CoV-2 immunoglobulins (Chen et al., 2020; Bloch et al., 2020). It is worth pointing out that not all antibodies that bind to the virus particles are neutralizing. Neutralizing antibodies are a part of humoral response of the adaptive immune system against pathogens and play critical roles in blocking virus particle from interacting with its host cells, thus contributing to viral clearance, or controlling disease progression. Antibodies to SARS-CoV-2 spike protein receptor binding domain neutralize virus infection (Ni et al., 2020; Bošnjak et al., 2020; Hussain et al., 2020). Whether SARS-CoV-2 antibodies convey a level of immunity that would prevent reinfection has not been elucidated. Protective antibodies that prevent infection in cell cultures have been discovered in sera of recovered COVID-19 patients (Rogers et al., 2020). How long does COVID-19 immunity lasts is still unknown. In this study, we aimed to evaluate clinical performances of three commercial automated serologic assays, which are based on various antigens and therefore measure different aspects of the immune response. We also aimed to investigate neutralizing effect and dynamic variance of SARS-CoV-2 Abs.

Materials and methods

Samples

Positive serum samples were obtained from 60 COVID-19 patients diagnosed by RT–PCR from nasopharyngeal swab specimens (Corman et al., 2020). Patients were hospitalized in Clinic for Infectious Diseases, Clinical Hospital Center Rijeka, Written informed consent form was obtained in which patients agreed to donate additional serum samples within 6 months after hospital discharge. Half of the COVID-19 patients (n = 30) were diagnosed with mild clinical course while 30 patients had moderate/severe COVID-19, according to NIH COVID-19 treatment guidelines (Anon, 2020a). For evaluation of seroconversion dynamics and persistence of SARS-CoV-2-specific Abs, sera of COVID-19 patients were collected with documented time point of collection post symptom onset (PSO) and/or time point from the first positive RT-PCR result. Sera were grouped in following categories: 2−4 weeks PSO, 8−10 weeks PSO, 15−20 weeks PSO and samples from convalescent individuals who were infected 26−35 weeks ago. Sera samples in each time group were obtained by serially testing individual patients across each time point. Negative serum samples (n = 100) were retrospectively selected from stored residual samples collected and frozen prior December 2019 from healthy asymptomatic adults and from patients with different respiratory infections.

SARS-CoV-2 commercial assays

Serum samples were evaluated for the presence of SARS-CoV-2 Abs with three commercial assays on automated analytical platforms. Abbott Architect SARS-CoV-2 IgG Chemiluminescent microparticle immunoassay (CMIA), (Abbott Diagnostics, Chicago, USA) was performed on the Abbott Architect i1000SR platform, bioMerieux VIDAS SARS-CoV-2 IgG/IgM Enzyme Linked Fluorescent Assay (ELFA) on VIDAS (BioMérieux, Marcy-l’Etoile, France) instrument and Chorus SARS-CoV-2 IgG/IgM Enzyme Linked Immuno Assay (ELISA) on Chorus TRIO instrument (DIESSE Diagnostica Senese S.p.A. Siena, Italy). All assays were performed according the manufacturer’s instructions. The intended use of each assay is qualitative detection of Abs; however, each assay provides semi-quantitative result expressed as ratio/index value as relative strength of signal.

Architect IgG assay measures Abs specific for nucleocapsid (N) antigen. Signal/cut-off (S/CO) ratio of ≥1.4 is interpreted as positive and <1.4 as negative. Recently, the manufacturer proposes an optional grayzone between 0.49−1.4. The manufacturer reported Architect SARS-CoV-2 IgG assay sensitivity of 100 % when tested ≥14 day PSO and specificity of 99.63 %.

VIDAS SARS-CoV-2 IgG/IgM assays measure Abs specific for receptor binding domain of the S1 subunit of the Spike glycoprotein (RBD/S1). Test value (index) ≥1.00 is considered to be positive and <1.00 as negative. The manufacturer proclaims sensitivity for IgG assay of 96.6 % ≥16 days after PCR positive results and specificity of 99.9 %. For IgM assay overall specificity of 99.4 % and sensitivity of 100 % (≥16 days after PCR positive results) is declared.

In Chorus SARS-CoV-2 IgG/IgM assays whole virion is used and Abs directed against all viral proteins are detected. Positive test is defined as the result - index (OD sample/OD cut-off) >1.1 and negative as result <0.9. The manufacturer reports diagnostic sensitivity for IgG test of 100 %; reached at day 10 PSO, and diagnostic specificity of 99.6 %. For IgM assay sensitivity of 87.5 % and specificity of 98.1 % is reported >12 days PSO.

SARS-CoV-2 surrogate neutralisation test

We analyzed dynamic changes of neutralizing antibodies (nAbs) in serum samples using cPass™ SARS-CoV-2 Surrogate Virus Neutralization Test (sVNT) (GenScript Biotech, Leiden, Netherlands), a blocking ELISA detection assay (Anon, 2020b). The sVNT detects nAbs that block interaction between RBD of viral Spike glycoprotein with angiotensin-converting enzyme 2 (ACE2) cell surface receptor. Optical density (OD) of the sample is inversely dependent on the titer of anti-SARS-CoV-2 nAbs. Inhibition of RBD-ACE2 binding is calculated according to the formula below:

According to the Manufacturer’s instructions (Version 3.3) the cut-off value <20 % is considered negative and ≥20 % is considered positive. For neutralization tests all sera samples were analyzed in duplicates.

Statistics and ethics

Continuous variables are expressed as the mean or median and were compared with the Mann–Whitney U test Correlation between data was done using Pearson r test. A P value of <0.05 was considered statistically significant. Results were analyzed using Microsoft Excel program 2018 and the Statistical Package for Social Sciences (SPSS) 20.0. Research was approved by the Research Ethics Committee of Clinical Hospital Center Rijeka. All participants signed Informed consent form prior serum sample obtaining.

Results

Assays characteristics

Architect IgG, VIDAS IgG and IgM, Chorus IgG and IgM kits were validated with 100 negative samples collected prior to COVID-19 outbreak and with 60 positive samples from symptomatic patients with confirmed COVID-19 disease (14–21 days PSO). Thirty patients had mild clinical course of the disease, while 30 patients had moderate/severe COVID-19.

Detailed data of SARS-CoV-2 IgG assays performance was shown in Table 1 . Specificity of the Architect IgG test was 99.01 % and overall accuracy rate reached 99 %. VIDAS and Chorus IgG assays reached 100 % specificity. Accuracy of Chorus IgG assay was 99.99 %, while VIDAS IgG assay has shown 100 % accuracy. Of the automated assays included in this study, using the cut-off values set by the manufacturers, the best sensitivity values of 100 % were obtained for VIDAS IgG assay. Chorus and VIDAS IgM assays achieved specificity of 99.01 % and 100 %, respectively. VIDAS IgM assay exhibited higher positive diagnostic rate as 76.92 %, while Chorus IgM assay showed detection rates of 74.07 %.

Table 1

Performance of three automated SARS-CoV-2 IgG and two SARS-CoV-2 IgM assays in control subjects and symptomatic COVID-19 patients 14-21 days PSO.

Assay	Performance measure	Estimate of performance (95 %) CI
		IgG assay	IgM assay
Architect	Sensitivity (%)	96.77 (88.83−99.61)
	Specificity (%)	99.01 (94.61–99.97)
	Accuracy (%)	99 (96−99.92)
VIDAS	Sensitivity (%)	100 (94.04−100)	76.92 (66−85.71)
	Specificity (%)	100 (96.38−100)	100 (96.38−100)
	Accuracy (%)	100 (97.72−100)	99.93 (97.81−100)
Chorus	Sensitivity (%)	96.77a (88.83−99.61)	74.07b (63.14−83.18)
	Specificity (%)	100 (96.38−100)	99.01 (94.61−99.97)
	Accuracy (%)	99.99 (99.96−100)	98.94 (96.14−99.88)

Abbreviations: CI = confidence interval.

One equivocal sample is counted as positive.

Ten equivocal results are counted as positive.

Positive sera (14–21 days PSO) tested with Architect IgG assay had a median S/CO of 6.98, while the same samples tested with VIDAS IgG assay had a more than twice median index (I) of 15.95 (results are not shown). Positive sera tested with Chorus IgG assay had a median I of 5.4. Positive sera tested with VIDAS IgM assay had a median of I of 2.11, while the same sera tested with Chorus IgM assay had a median I of 1.02.

Comparison of different assays in monitoring the dynamics of serological response

In order to evaluate dynamics of Abs response against SARS-CoV-2 and their neutralizing effect, 30 COVID-19 patients with severe and 30 with mild disease were enrolled with serial/sequential acute and convalescent specimens analysis. Sera were collected 2−4 weeks PSO, 8−10 weeks PSO, 15−20 weeks PSO and 26−35 weeks (6–8 months) PSO. Summary of the reports analysing temporal dynamic changes of the antibody response using different serological assays is shown in Table 2, Table 3 .

Table 2

Dynamics of SARS-CoV-2 specific antibody response using different serological assays – a percent of positive sera.

Weeks PSO	Architect SARS-CoV-2 IgG (%)	VIDAS SARS-CoV-2		Chorusa SARS-CoV-2
		IgG (%)	IgM (%)	IgG (%)	IgM (%)
2−4	97.5	100	70	97.5	67.5
8−10	97.5	100	80	97.5	70
15−20	87.5	90	25	90	26.3
26−35	25 (50)b	85	10	50	12.5

Abbreviations: PSO = post symptom onset.

Chorus IgG/IgM equivocal results are counted as positive.

Result in parenthesis - considering an optional grayzone and counting ten equivocal results as positive.

Table 3

Dynamics of SARS-CoV-2 antibody response using different serological assays – mean values.

Weeks PSO	Architect mean S/Co (CV) IgG	VIDAS mean index (CV)		Chorus mean index (CV)
		IgG	IgM	IgG	IgM
2−4	7.06 (0.07)	20.26 (0.90)	4.54 (1.32)	5.15 (0.30)	1.03 (0.60)
8−10	5.10 (0.01)	13.53 (0.69)	6.87 (0.59)	3.70 (0.69)	1.53 (0.30)
15−20	3.29 (0.59)	11.51 (0.89)	1.03 (1.49)	3.44 (0.62)	1.04 (0.76)
26−35	1.04 (1.48)	7.90 (1.06)	0.64 (1.32)	2.30 (0.61)	0.90 (1.07)

Abbreviations: CV = coefficient of variation; S/CO = signal/cut-off; PSO = post symptom onset.

In sera collected 2−4 weeks PSO SARS-CoV-2 IgG response was detected in 97.5 % (Architect), 100 % (VIDAS), and 97.5 % (Chorus) of cases, while IgM response was detected in 70 % (VIDAS) and 67.5 % (Chorus) of cases (Table 2). IgG antibodies persist in serum for ten weeks, beginning to decline afterwards. More than six months after infection IgG Abs could be detected in 25 % of cases using Architect, in 50 % cases using Chorus IgG assay, or in 85 % cases using VIDAS IgG assay. Considering an optional cut-off of 0.49 for Architect IgG assay, and counting equivocal results between 0.49–1.4 as positive, the percent of positive results 26−35 weeks PSO increased from 25 % to 50 %. The highest percent of IgM positive sera was recorded 8−10 weeks PSO (VIDAS IgM assay) starting to decline rapidly soon after. More than six months after infection SARS-CoV-2 IgM Abs could be detected in 10 % and 12.5 % cases (VIDAS and Chorus), respectively. Comparing results obtained with three SARS-CoV-2 IgG tests it is obvious that IgG Abs reach the highest values in the four week PSO (Table 3). Values obtained using the VIDAS IgG assay are 3–4 times higher than those obtained with the other two IgG assays. IgM Abs peaked a little later and 8−10 weeks after infection reached mean index of 6.87 (VIDAS) or 1.53 (Chorus) and then began to decline.

Levels of SARS‐CoV‐2 IgG in mild and moderate/severe COVID-19 patients using VIDAS IgG test

To explore the correlation of antibody titres and disease severity, patients were divided into two groups - mildly affected convalescents (n = 30) and moderate/severe group of COVID-19 patients (n = 30). Considering superior clinical performance in comparison to other two assessed SARS-CoV-2 IgG tests, we selected VIDAS IgG assay for analysis of the difference between IgG Abs levels in two patient groups at three time points: 8−10, 15−20 and 26−35 weeks PSO (Table 4 ). Results showed that the concentrations of IgG Abs in the group of moderate/severe COVID-19 patients were significantly higher (p < 0.05) than that of mild group at all-time points.

Table 4

Comparison of the mean values of IgG Abs between non-severe and severe groups.

Patient group	Weeks PSO
	8−10	15−20	26−35
Mild	9.73	6.87	3.74
Moderate/severe	16.25	18.29	11.13
	p = 0.031	p = 0.00056	p = 0.00362

SARS-CoV-2 Surrogate Virus Neutralization Test (sVNT)

We investigated neutralizing activities of IgG Abs against SARS-COV-2 in sera samples of 20 mild cases and 20 severely affected COVID-19 patients using GenScript cPass sVNT. Ten sera collected in pre-COVID-19 era were used as negative controls. Neutralizing effect of SARS-COV-2 IgG Abs is shown in Table 5 . According to the manufacturer’s instructions (version 3.3) percent of inhibition of RBD-ACE2 binding (titer) <20 % is proposed to be considered negative, while titer ≥20 % positive (Anon, 2020b). At the time of re-submitting this manuscript, the GenScript cPass test received the FDA EUA authorization. In the latest updated 4.0 version of manufacturer user manual cut-off criteria for in vitro diagnostics is 30 % inhibition. Considering that all tested negative pre-COVID-19 era control sera have shown mean inhibition titer of 32.3 % with very low dispersion of data (CV = 0.04), we decided to optimize manufacture’s recommended cut-off. We performed statistical analysis of the of negative sera samples data and placed the cut-off at the mean value plus five standard deviations. Hence, an adjusted sVNT assay threshold of 40 % was further used to interpret the results obtained in this study.

Table 5

Neutralization effect of SARS-CoV-2 IgG antibodies.

Patient group	Pre-COVID-era	Two months PSO		Four months PSO
	Mean % inhibition (CV)	% sera with titer ≥40	Mean % inhibition (CV)	% sera with titer ≥40	Mean % inhibition (CV)
Mild		100	72.94 (0.24)	82	64.76 (0.31)
Moderate/Severe		100	93.08 (0.07)	100	84.8 (0.22)
			p<0.5		p<0.5
Control sera	32.3 (0.04)

Abbreviations: CV = coefficient of variation, PSO = post symptoms onset.

All serum samples from both mild and moderate/severe group of COVID-19 patients collected two months PSO showed neutralizing activity (mean inhibition of 72.94 % and 93.08 %, respectively). Further, all sera from moderate/severe cases collected four months PSO also showed neutralizing activity with mean inhibition of 84.8 %. However, 82 % serum samples from mild cases collected four months PSO showed neutralizing activity (mean inhibition of 64.76 %). Mild cases sera without neutralizing effect were also serological negative at indicated time point (IgG/IgM Abs were not detected). One discordant serum sample from mild COVID-19 patient that tested positive by Architect IgG assay and negative by VIDAS and CHORUS IgG assays did not show neutralizing activity. Our results indicate that patients recovering from severe disease develop higher neutralizing anti-SARS-CoV-2 IgG Abs titers.

Correlation of SARS-CoV-2 IgG serology assays with neutralizing activity

Correlation coefficient is used to assess the strength and direction of the linear association between three SARS-CoV-2 IgG assays and neutralizing activity. Although all three assays showed positive correlation to neutralizing activity, Abbott Architect IgG assay (Fig. 1 A) showed moderate positive correlation (r = 0.756). The strongest positive correlation (r = 0.824) was found for VIDAS SARS-CoV-2 IgG assay (Fig. 1B). Chorus SARS-CoV-2 IgG assay also showed strong positive correlation (r = 0.813) to neutralizing activity (Fig. 1C).

Fig. 1

Correlation of different SARS-CoV-2 IgG serology assays with neutralizing activity. Architect IgG (A), VIDAS IgG (B) and CHORUS IgG (C) assay (n = 40). Pearson correlation coefficient (r) and p value are depicted in plot for each assay.

Discussion

Comparative performance data are crucial to guide the use of serology in COVID-19 diagnostics. The ability of one assay to detect SARS-CoV-2 Abs must be validated in real-world clinical laboratory practice taking into consideration the time-frames for an individual to develop detectable levels of antibodies. Here we report the clinical performance characteristics of three automatized SARS-CoV-2 serological assays based on different antigens in comparison to current “gold-standard” RT-PCR. Architect N protein-based SARS-CoV-2 IgG assay has shown specificity of 99.01 %, close to manufacturer’s package insert data of 99.63 %. Studies performed in USA between March and April 2020 revealed Architect IgG assay specificity of 99.6 % and 99.9 % (Manalac et al., 2020; Bryan et al., 2020), while report from Singapore stated specificity as high as 100 % (Chew et al., 2020). These results suggest that different background immunity status across different regions of the world may affect the sensitivity and specificity of assays. Assay performance is highly time-sensitive, reflecting dynamics of seroconversion. Based on currently available data, median seroconversion time for both SARS-CoV-2 IgG and IgM Abs is two to three weeks after COVID-19 onset (Ou et al., 2020; Zhao et al., 2020; Okba et al., 2020). In that order we used sera collected two-three weeks PSO for sensitivity determination. Using manufacturer’s recommended S/CO ratio ≥1.4 to be interpreted as reactive we report Architect IgG assay clinical sensitivity of 96.77 %, which does not fully match the manufacturer’s stated performance criteria of 100 %. Chew et al. reported Architect SARS-CoV-2 IgG assay sensitivity of 84.4 % at ≥21 days PSO and Patel at al. of 92.5 % (Chew et al., 2020; Patel et al., 2020). However, at the time of writing this manuscript Abbott has updated the Architect SARS-CoV-2 IgG assay to include an optional grayzone between 0.49 and 1.40. With mentioned implementation, previously false negative results (S/CO 0.9) from our study would be assigned as equivocal, thus improving assay sensitivity. We believe that inclusion of grayzone in seropositivity threshold determination would improve the sensitivity of the Architect IgG assay. Our observation is in accordance with Chew et al. who also propose lowering of the cut-off of Architect IgG assay to 1.0 or 0.8 (Chew et al., 2020).

Excellent clinical sensitivity and specificity of the RBD/S1 protein-based VIDAS SARS-CoV-2 IgG assay assessed here (both reached 100 %) is in accordance with the manufacturer’s data sheet as well as with recent study of Renard et al. Renard et al. (2021). Although the same study declared for VIDAS SARS-CoV-2 IgM test 100 % positive percent agreement (PPA) with PCR (24–31 days PSO), and manufacturer proclaims 90.6 % PPA with PCR (8–15 days PSO), we report sensitivity of 76.92 % (14–21 days PSO). Observed specificity of VIDAS IgM assay of 100 % was in line with the manufacturer’s data.

Data on clinical performance of the whole virion-based Chorus ELISA assay to detect SARS-CoV-2 IgG/IgM antibodies are limited. Our validation of Chorus SARS-CoV-2 IgG assay showed clinical sensitivity of 96.77 %, specificity of 100 %, and overall accuracy of 99.99 %. While high specificity was in line with manufacturer’s declared performance, clinical sensitivity does not fully match the stated performance criteria of 100 %. With regard to Chorus SARS-CoV IgM test, sensitivity of 74.07 % (lower than manufacturer's sensitivity claims of 87.5 %) and specificity of 99.01 % was observed in our study. It should be noted that we considered all equivocal results as positive, which increased sensitivity of Chorus IgG/IgM assays.

In general, our data support the use of all three validated assays for SARS-CoV-2 IgG/IgM detection. Although we found Architect IgG assay to be less sensitive compared to the other two assessed SARS-CoV-2 IgG assays, Architect assay could potentially meet the sensitivity targets through threshold adjustment. Similar to Chew et al., we found that Architect IgG assay had a slightly lower specificity (99.01 %) than its claim in manufacturer’s data sheet (99.63 %) (Chew et al., 2020). It could be important if we know that specificity of at least 99.5 % is required to achieve a high positive predictive value in low-prevalence populations. Architect IgG assay is N protein-based, and therefore more often associated with cross-reactivity than S protein-based assays. However, advantage of Architect SARS-CoV-2 IgG test, in regard to other two validated assays, is that it is suitable for screening of large numbers of patients because of high throughput.

In rapidly evolving field of SARS-CoV-2 immunity research, presence and duration of Abs against SARS-CoV-2 are still unknown (Shah et al., 2020; Vabret et al., 2020). Data from literature show production of specific IgG and IgM Abs one week after COVID-19 onset with IgG titers increasing during first 3 weeks and beginning to decline by 8 weeks after onset (Seow et al., 2020). We observed the highest IgG response 2−4 weeks PSO, lasting 10 weeks, followed by gradual decline. However, majority convalescents maintained high IgG titers more than eight months PSO. According to VIDAS SARS-CoV-2 IgG results, as most sensitive of evaluated SARS-CoV-2 assays, 90 % sera were positive 5 months, while 85 % sera were still positive 6–8 months PSO. It should be pointed out that majority of sera with positive Abs levels at late time point using VIDAS IgG assay, had equivocal values with Architect IgG assay, which again emphasizes the need for Architect IgG test threshold adjustment. Similarly, Chorus IgG assay showed 50 % positivity at same time point. Regarding the IgG antibody median titer levels, twofold reduction in titer was observed 5 months PSO, regardless of the test/antigen used. This difference in longevity of Abs measured with VIDAS IgG and other two SARS-CoV-2 IgG assays can be partly explained by the difference in cut-off value of three assays but also by the fact that automated assays presented here are all based on different antigen components (N, S1, all viral proteins respectively). As expected, antibody responses against each of these antigens may develop with varying kinetics. We observed lower sensitivity of N-based Abbott IgG assay in comparison to RDB/S1 protein-based VIDAS assay. Results of our study are in line with other studies that showed that antibodies directed against the N protein seems to decrease earlier than antibodies against the S protein (Shah et al., 2020). Caruana et al. also found more pronounced serum levels decrease of Abs specific for N protein (Architect) in comparison to Abs specific for S protein (VIDAS) (Caruana et al., 2020). Therefore, the sensitivity of assays targeting only N protein may be impaired according to the timing of infection vs. PSO.

In contrast to the antibody response against acute viral infections, in the present study we found that SARS-CoV-2 IgM Abs reached their highest positivity rate 8−10 weeks PSO, a little later than IgG Abs. IgM Abs were not detectable after 3 months in most cases. However, in few cases low levels of IgM Abs were still present 8 months PSO.

Some studies reported significantly higher antibody titer values in patients with severe form of COVID-19 as compared to patients with mild form of the disease (Zhao et al., 2020; Poland et al., 2020), while others reported that memory B cells are capable of producing nAbs after mild COVID-19 infection (Rodda et al., 2020). Here we demonstrate that IgG Abs concentration correlates to disease severity. Comparing IgG Abs levels in mild and moderate/severe COVID-19 patients at different time-points we found that, no matter at what stage PSO, levels of IgG Abs in sera of patients with moderate/severe form of the disease were significantly higher. If estimated according to antibody longevity, immunity after SARS-CoV-2 infection is thought to be temporary, lasting only several months. In this study, some individuals had the waning IgG response after five months, while some maintained a high titer more than eight months after infection, regardless of the severity of the disease. However, a limitation in our sample set is that sera from asymptomatic patients and critical ill patients were unavailable.

Since no correlate of protection for SARS-CoV-2 has been defined yet, it remains to determine the titer of neutralizing antibodies providing patients protection from reinfection. Neutralizing antibodies inhibit viral replication in vitro and their presence correlate with immunity to future infection, at least temporarily. Experimental study with SARS CoV-2 re-infected rhesus macaques showed immunologic control and sterilizing immunity (Chandrashekar et al., 2020). Therefore, we selected sera from mild and moderate/severe COVID-19 patients, and negative control sera to assess their neutralizing capacity against SARS-CoV-2 using ELISA-based GenScript cPas™ sVNT. Surprisingly, the application of the manufacturer’s recommended cut-off of 20 % (which was valid at the time the experiments were performed) classified all presumably negative sera samples as positive. In the meantime, the manufacturer updated the results interpretation criteria adjusting the cut-off to 30 % (sVNT version 4.0). However even with the application of adjusted threshold we observed weak neutralizing activity in 100 % of pre-CoVID-19 negative control sera samples (inhibition of 30 %–34 %). As an assay specificity may be enhanced by raising the cut-off level, after statistical analysis of the of negative sera samples data we placed sVNT inhibition threshold at the 40 %. It is worth pointing out that before the manufacturers cut-off adjustment, Tan CW et al. in their validation study proposed an alternative cut-off of 30 % for GenScript sVNT (Tan et al., 2020). We show here that raising the test threshold to 30 % would not be enough, and setting the threshold to 40 % would make the GenScript sVNT results more reliable. One limitation of our study is the lack of testing pre-COVID-19 control sera for seasonal human coronaviruses (hCoV) presence, since the majority of the population has Abs against the hCoV. Hence, the false positive results observed in sVNT may be, at least in part, due to cross-reactivity of pre-COVID-19 sera to NL63 hCoV, which use the same ACE2 receptor as SARS-COV-2. However, it is unlikely that all pre-COVID-19 negative sera examined here contain nAbs to non-SARS-COV-2 coronavirus strains in approximately equal titer. Reported absence of cross-neutralization between endemic hCoV and SARS-COV-2 is in contrast to the premise that positive sVNT results may be due to past infection with other hCoV strains (Legros et al., 2021). In fact, cross-reactivity to SARS-CoV-2 in the neutralization assays was previously detected only with SARS-CoV-1 convalescent human plasma (Perera et al., 2021; Lv et al., 2020). Given the very low prevalence of SARS-CoV-1 infection, and the fact that SARS-CoV-1 does not circulate in the population from 2004, it is unlikely that SARS-CoV-1 nAbs were detected in pre-CoVID-19 sera samples. However, selection of the most appropriate sVNT cut-off value for clinical use requires more additional studies using a collection of large number of well-defined reference sera. The cut-off value of commercially available serological test is not universal, and could be adjusted before use as a routine diagnostic assay as appropriate for each region and for each disease condition.

Using the sVNT we observed high neutralizing activity of all COVID-19 patients sera collected two months PSO, with mean inhibition percent significantly higher in moderate/severe vs mild cases. Similarly, Meyer et al. reported an increase in assay sensitivity reflecting higher disease severity in sera samples taken ≥14 days PSO (Meyer et al., 2014). In our study, the percent of sera with significant neutralization effect collected from mild COVID-19 patients decreased from 100 % to 82 % four months PSO, showing mean inhibition of 64.76 % at this time point. However, at the same time point 100 % sera from moderate/severe affected patients showed neutralizing effect, with significantly higher mean inhibition percent of 84.8 %. Therefore, although all COVID-19 patients developed robust neutralization titers during the course of infection, severe COVID-19 was associated with higher antibody production and neutralization titers.

In a natural infection COVID-19 patients develop a broad-base Abs against multiple SARS-CoV-2 proteins, and neutralizing Abs against S protein represent a small subset (Jiang et al., 2020). As shown before, neutralizing Abs target both RBD and non-RBD (S1-NTD, S2) epitopes of S protein (Barnes et al., 2020) therefore, the surrogate virus neutralisation test cannot completely replace classical virus culture-based neutralization assays, because sVNT does not measure all neutralising Abs but only the ones directed against the RBD. The GenSys sVNT assay measures only one aspect of neutralisation, which represents only a part, albeit a major part of the neutralisation of the virus by antibody. It should be emphasized that the other arms of the immune system, such as cellular immunity, may also play a significant role in COVID-19 immunity. We point out that GenScript sVNT could be used as an additional assay to assess the immune status of COVID-19 patients, however optimisation of assay threshold set by manufacturer is needed to correctly classify the results.

We explored association between the SARS-CoV-2 serology results and neutralizing activity. While all three commercial assays showed positive correlation to neutralizing activity, the strongest correlation was found for VIDAS SARS-CoV-2 IgG assay. This is not surprising because VIDAS assay measure Abs specific for RBD/S1 subunit of the S protein, and only nAbs to RBD of S1 protein would be detected in the GenScript sVNT.

Further, in present study we found strong positive correlation with Chorus IgG assay, which allows detection of Abs directed against all viral proteins, and neutralizing activity of analyzed sera. Although whole virion-based assays showed a higher cross-reactivity compared to whole spike or S1 domain-based assays (30) we demonstrated here high diagnostic accuracy of Chorus SARS-CoV-2 assay. In this context, this whole-virion based assay could be used with equal reliability as others S or N-protein based commercially available serological assays. Additionally, it can be used in parallel to S or/and N-protein based assays to confirm/exclude discordant results. Monotest format makes Chorus assay more suitable for urgent testing than for testing a large number of samples. Although several SARS-CoV-2 variants emerge, there is no reported effect of the new virus variant on serological diagnostic assays (Ascoli, 2021). However, a whole-virion assay design that includes detection of different Abs may have some advantages over assays that rely on the detection of Abs against one epitope as the virus continues to evolve.

Although N-protein based Architect IgG assay evaluated here showed moderate positive correlation with neutralizing activity, this result is not reliable for predicting the presence of nAbs. We note that one serum sample that tested positive by Architect IgG assay, was negative by sVNT and VIDAS IgG assay. Anti-N Abs detected by Architect SARS-CoV-2 IgG assay lack neutralizing activity, and therefore any correlation with nAbs is co-incidental.

In order to use these tests properly, it is important to understand their performance characteristics and limitations. N protein is a main differentiator of immune responses to natural COVID-19 versus those to spike protein-based vaccines. Therefore, test targeting the N protein, such is Architect IgG assay, could be used to distinguish natural infection from vaccination in an RT-PCR negative patient who has received a spike protein-based COVID-19 vaccine. As N-antibodies are useful indicator of infections, N protein-based assays should be well suited for epidemiological surveillance. In clinical practice, IgG and IgM anti-SARS-CoV-2 detection and measurement are of significant importance for COVID-19 diagnosis in cases of ambiguous results of RT-PCR analysis. Thus, highly specific and sensitive S-protein based serological assays, used in conjunction with a molecular method, may have supporting role in the clinical diagnosis of COVID-19 in unvaccinated individuals. They also may be useful in seroprevalence studies of unvaccinated populations providing a more accurate estimate of the true number of infections. By evaluating anti-RBD IgG levels, the VIDAS SARS-CoV-2 IgG assay may be used for detection of antibodies resulting from most anti-SARS-CoV-2 vaccination, that correlate with neutralization.

In summary, all three clinically validated IgG tests are high-quality assays with clinical sensitivity greater than 95 % and specificity superior or equal to 99 %. We observed high neutralizing activity of mild to severe COVID-19 patients sera. Higher antibodies titers and neutralizing activity was observed in sera of severe patients. The strongest positive correlation to neutralizing activity was found for RBD/S1-based VIDAS IgG assay. Although the presence of antibodies cannot be equated with an individual’s immunity to SARS-CoV-2, their longevity of at least eight months in most cases, and neutralizing effect shown by this study, indicates at least some degree of protection against future infection.

Author statement

Marina Bubonja-Šonje: conceptualization, methodology, writing original draft.

Lara Batičić: investigation.

Maja Abram: reviewing and editing, funding acquisition.

Đurđica Cekinović Grbeša: formal analysis, reviewing and editing, validation.

All authors approved this revised manuscript version to be submitted.

Funding

This work was supported by University of Rijeka (MBŠ, grant number uniri-biomed-18-215 and MA, grant number uniri-biomed-18-277).

Declaration of Competing Interest

The authors report no declarations of interest.