Comparative study of six SARS-CoV-2 serology assays: Diagnostic performance and antibody dynamics in a cohort of hospitalized patients for moderate to critical COVID-19.

BACKGROUND: To overcome the COVID-19 pandemic, serology assays are needed to identify past and ongoing infections. In this context, we evaluated the diagnostic performance of 6 immunoassays on samples from hospitalized patients for moderate to critical COVID-19.
METHODS: 701 serum samples obtained from 443 COVID-19 patients (G1: 356 positive RT-PCR patients and G2: 87 negative RT-PCR cases) and 108 pre-pandemic sera from blood donors were tested with 6 commercial immunoassays: (1) Elecsys Anti-SARS-CoV-2, Roche (Nucleocapsid, N), (2) Elecsys Anti-SARS-CoV-2 S, Roche (Spike, S), (3) Vidas SARS-COV-2 IgM/IgG, BioMérieux (S), (4) SARS-CoV-2 IgG, Abbott (N), (5) Access SARS-CoV-2 IgG, Beckman Coulter (Receptor Binding Domain), and (6) Standard F COVID-19 IgM/IgG Combo FIA, SD Biosensor (N).
RESULTS: Global sensitivities of the evaluated assays were as follows: (1) Roche anti-N = 74.5% [69.6-79.3], (2) Roche anti-S = 92.7% [84.7-100], (3) Vidas IgM = 74.9% [68.6-81.2], (4) Vidas IgG = 73.9% [67.6-80.1], (5) Abbott = 78.6% [63.4-93.8], (6) Beckman Coulter = 74.5% [62-86.9], (7) SD Biosensor IgM = 73.1% [61-85.1], and (8) SD Biosensor IgG = 76.9% [65.4-88.4]. Sensitivities increased gradually from week 1 to week 3 as follow: (1) Roche anti-N: 63.3%, 81% and 82.1%; (2) Vidas IgM: 68.2%, 83.2% and 85.9%; and (3) Vidas IgG: 66.7%, 79.1% and 86.6%. All immunoassays showed a specificity of 100%. Seropositivity was significantly associated with a higher frequency of critical COVID-19 (50.8% vs. 38.2%), p = 0.018, OR [95% CI] = 1.668 [1.09-2.553]. Inversely, death occurred more frequently in seronegative patients (28.7% vs. 13.6%), p=3.02 E-4, OR [95% CI] = 0.392 [0.233-0.658].
CONCLUSION: Evaluated serology assays exhibited good sensitivities and excellent specificities. Sensitivities increased gradually after symptoms onset. Even if seropositivity is more frequent in patients with critical COVID-19, it may predict a recovery outcome.

Introduction

Coronavirus disease 2019 (COVID-19) is the new pandemic due to infection with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). SARS-CoV-2 is a single-stranded RNA virus of positive polarity belonging to the coronaviridae family. The SARS-COV-2 genome encodes 4 structural proteins: S protein (spike), M protein (membrane), E protein (envelope), and N protein (nucleocapsid). SARS-CoV-2, like SARS-CoV, binds the S protein to a cellular receptor in order to infect human cells. Computer modeling together with the replication of SARS-CoV-2 infection in HeLa cells identified angiotensin-converting enzyme 2 (ACE2) as the receptor for the SARS-CoV-2. The S protein consists of 2 subunits: the S1 subunit binds to ACE2 via its receptor binding domain (RBD) and the S2 subunit mediates fusion between the virus and the cell membrane.

Humoral response against SARS-CoV-2 targets mainly the S and N proteins. As a subset of antibodies (Ab), called neutralizing Ab (Nab), targeting the RBD appear to block viral binding and neutralize viral infectivity in vitro; this domain is an important vaccine and therapeutic target. In patients with COVID-19, Ab appear around 2–3 weeks after symptom onset. Nevertheless, anti-SARS-COV-2 Ab may remain undetectable in mild COVID-19 and in asymptomatic individuals.[7,8] Furthermore, the intensity of the IgG response is subject to inter-individual variability depending on the immunocompetence of the host and the duration of the virus presence in the body.

Currently available commercial immunoassays detect one or more Ab isotypes (IgG, IgA, and IgM), separately or as total Ab. The majority of serological tests use S and/or N proteins as targets for antibodies. These serology tests include lateral flow immunoassays (LFIA), enzyme-linked immunosorbent assays (ELISA), and chemiluminescence immunoassays (CLIA). Assays performance relies on their clinical specificity and sensitivity. The method used, the Ab isotype and the targeted antigen, can influence the test performance. Some authors recommended a sensitivity of 95% or more and a specificity of 99.5% or more based on samples obtained 14 days or more after the symptoms onset or a positive result for an RNA test.

Reverse transcription-polymerase chain reaction (RT-PCR) detecting SARS-CoV-2 RNA in nasopharyngeal swab or bronchoalveolar lavage is the primary tool for confirming clinical suspicion of COVID-19. However, some studies showed a lack of sensitivity of the RT-PCR test. In fact, some false-negative results are due to poor swab sampling or to the absence of the virus in nose-pharynx for some individuals. A meta-analysis of 10 published studies showed an 87% pooled sensitivity for RT-PCR. Moreover, the RT-PCR is unable to detect past infection in recovered patients. Hence, serology testing has an added value in detection of both active and past infections.

Some comparative studies showed a high concordance between qualitative results but weak to moderate correlations between quantitative results. In this context, we aimed to assess and compare sensitivities and specificities of six SARS-CoV-2 serology assays together with the study of kinetics of Ab production in a cohort of hospitalized patients for moderate to critical COVID-19.

Material and methods

Subjects

To determine the sample size, we used an online calculator: http://www.raosoft.com/samplesize.html. The Tunisian population being composed of 11,154,400 people, for a type I error (alpha) of 5% and 80% power, the necessary sample size was 385 people. This prospective study included 443 COVID-19 patients and 108 healthy voluntary blood donors from the same ethnic origin (Tunisian). Seven hundred and one serum samples were collected between March 2020 and April 2021 from 443 patients hospitalized for moderate to critical COVID-19 in the pulmonology, COVID-19 (temporary) and intensive care unit (ICU) departments of Charles Nicolle Hospital in Tunis (Table 1). Nasopharyngeal sampling for RT-PCR testing was performed for all patients. In patients with a negative RT-PCR result, chest computerized tomography (CT) scan was carried out. Thus, COVID-19 patients were classified as follows: (1) G1: 356 patients with a positive RT-PCR and (2) G2: 87 patients for which COVID-19 diagnosis was based on chest CT findings.

Table 1.

COVID-19 patients features.

COVID-19 patients	n = 443
Sex ratio (male/female)	1.3 (250/193)
Age ± SD (years)	60.59 ± 16.29
Samples	701
Patients with 1 sample	237
Patients with 2 samples	154
Patients with 3 samples	52
Time to the 1st sampling after symptoms onset (days)	8 [6–12.75]
Time to sampling after symptoms onset (days)	12 [7–17]
Samples obtained during the 1st week after symptoms onset	177
Samples obtained during the 2nd week after symptoms onset	271
Samples obtained during the 3rd week after symptoms onset	165
Samples obtained more than 21 days after symptoms onset	88
RT-PCR positive	356 (80.4%)
Clinical presentation
Moderate CoViD-19	246 (55.5%)
Critical CoViD-19	197 (44.5%)
Clinical outcome
Recovery	365 (82.4%)
Death	78 (17.6%)

Inclusion criteria:

• Age ≥18 years

• Patients with any of the following respiratory symptoms: cough, polypnea, dyspnea, and hypoxia.

• Positive result for RT-PCR and/or abnormal lung CT scan findings

Exclusion criteria:

• Negative RT-PCR and chest CT without COVID-19 suspicious findings COVID-19 was considered critical if any of the following findings occurred:

• Oxygen saturation ≤92% or acute respiratory distress needing mechanical ventilation

• Shock

• Multiorgan failure requiring hospitalization in ICU

Pre-pandemic serum samples were obtained in 2018 from 108 healthy subjects. Controls were adult blood donors matched in age, gender, and ethnicity with the COVID-19 patients. Ethnicity (Tunisian) of both patients and controls was determined by an oral survey.

All patients and controls gave written informed consent to participate in the study, and the local Ethics’ committee of Charles Nicolle Hospital approved this study. No benefits in any form have been received or will be received from a commercial party related directly or indirectly to the subject of this manuscript.

Methods

Blood sampling

Blood samples from all subjects were collected at the Immunology laboratory of the Charles Nicolle Hospital on dry tubes. Upon receipt, each tube was centrifuged at 3000 rpm for 15 min. The serum, aliquoted in 1 mL tubes and frozen at −80°C, was used for the measurement of anti-SARS-CoV-2 Ab.

SARS-CoV-2 antibodies measurement

Five platforms were used to compare 6 commercial SARS-CoV-2 serology immunoassays following the respective manufacturers recommendations (Table 2):

Table 2.

Detailed information on SARS-CoV-2 serology immunoassays.

Assay	Provider	Platform	Method	Ab isotypes	Antigen	Unit	Cut-off	Sensitivity (according to manufacturer)	Specificity (according to manufacturer)
Elecsys Anti-SARS-CoV-2	Roche	Cobas e 411	ECLIA	Total Ab	N	COI	>1	100% (≥14 days since symptom onset)	99.81
Elecsys Anti-SARS-CoV-2 S	Roche	Cobas e 411	ECLIA	Total Ab	RBD	U/ml	≥0.8 U/ml	100% (≥28 days since symptom onset)	99.98
Vidas SARS-COV-2 IgM	bioMérieux	miniVidas	ELFA	IgM	S	i	≥1	100% (≥16 days since symptom onset)	99.6
Vidas SARS-COV-2 IgG	bioMérieux	miniVidas	ELFA	IgG	S	i	≥1	96.6% (≥16 days since symptom onset)	100
SARS-CoV-2 IgG	Abbott	Architect C4000	CMIA	IgG	N	Index (S/C)	≥1.4	100% (≥14 days since symptom onset)	100
Access SARS-CoV-2 IgG	Beckman Coulter	Access 2	CLIA	IgG	RBD	S/CO	≥1	100% (≥18 days since symptom onset)	99.8
Standard F COVID-19 IgM/IgG Combo FIA	SD biosensor	SD biosensor F2400	FIA	IgM and IgG (separately)	N	COI	≥1	98.85% (≥14 days since symptom onset)	90.62

ECLIA: electrochemiluminescence immunoassay, ELFA: enzyme-linked fluorescent assay, CMIA: chemiluminescent microparticle immunoassay, CLIA: chemiluminescence immunoassay, FIA: fluorescent immunoassay, Ab: antibody, N: nucleocapsid, S: Spike, RBD: Receptor Binding Domain, COI: cut-off index, i: index, S/CO: signal to cut-off.

• The Roche Cobas e411 was used for evaluating both Elecsys Anti-SARS-CoV-2 (N protein) and Elecsys Anti-SARS-CoV-2 S (S protein) tests by electrochemiluminescence immunoassay (ECLIA).

• The miniVidas was used for assessing vidas SARS-CoV-2 IgM and IgG (S protein) assays by enzyme-linked fluorescent assay (ELFA).

• The Abbott Architect C4000 was used to test the SARS-CoV-2 IgG (N protein) kit by chemiluminescent microparticle immunoassay (CMIA).

• The Beckman Coulter Access 2 was used for testing the Access SARS-CoV-2 IgG (RBD) assay by CLIA.

• The SD Biosensor F2400 was used to evaluate the Standard F COVID-19 IgM/IgG Combo FIA assays by fluorescent immunoassay (FIA).

Due to limited availability, the number of performed tests for each immunoassay varied (Table S1). Therefore, the humoral response kinetics were investigated only with the Elecsys Anti-SARS-CoV-2 (n = 614), the Vidas SARS-COV-2 IgM (n = 401), and the Vidas SARS-COV-2 IgG (n = 414) assays.

Statistical analysis

Statistical evaluation was carried out using the Statistical Package for the Social Sciences (SPSS) version 11 (IBM®, Armonk, USA). p-values <0.05 were considered significant. The used tests for the statistical analysis are summarized in Table S2.

Chi-square or Fisher exact tests were used to test the association between categorical variables. Odds ratio (OR) together with 95% confidence intervals [95% CI] were calculated to estimate the strength of the association.

ANOVA, Mann–Whitney U, and Kruskal–Wallis tests were used to analyze quantitative variables as appropriate.

Specificities and sensitivities were calculated in G1 (positive RT-PCR patients) according to the following formulas:

• Specificity = True Negative / (True Negative + False Positive)

• Sensitivity = True Positive / (True Positive + False Negative)

Confidence intervals (CI) for specificities and sensitivities were calculated using the following formula: p ± 1.96 √p (1–p)/N.

Receiver-operating characteristic (ROC) curves were used to assess performances (specificity and sensitivity) and identify optimal cut-off for the evaluated SARS-CoV-2 serology assays.

The Cohen’s kappa coefficient (κ) was calculated to assess the agreement (matching) between qualitative results of the evaluated serology assays.

The Spearman’s rank correlation coefficient was determined to test the correlation between quantitative serology results.

Cumulative seroconversion rates were estimated using the Kaplan–Meier analysis.

Results

As summarized in Table 1, mean age of the 443 COVID-19 patients was at 60.59 ± 16.29 years with a sex ratio (Men/Women) of 1.3 (250/193). RT-PCR was positive in 356 (80.4%) patients (G1). Clinically, 197 (44.5%) patients suffered from a critical COVID-19 and death occurred in 78 (17.6%) patients. Critical COVID-19 frequency was higher in male patients (125, 50%) comparatively to females (72, 37.3%); p = 0.008, OR [95% CI] = 1.68 [1.14–2.46]. However, no significant association was observed between gender and death outcome (16.4% vs. 19.2%), p = 0.448.

Median time to the first serology sampling post-symptoms onset was 8 [6–12.75] days. Median time to serum sampling after symptoms onset was 12 [7-17] days. 177 samples were collected during the first week after symptoms onset, 271 samples during the 2nd week, 165 samples during the 3rd week, and 88 samples more than 21 days after symptoms onset (Table 1).

Serology results in reverse transcription-polymerase chain reaction-confirmed COVID-19 patients (G1) and controls

Overall, sensitivities of the evaluated assays were (Table 3): (1) Roche anti-N = 74.5% [69.6–79.3], (2) Roche anti-S = 92.7% [84.7–100], (3) Vidas IgM = 74.9% [68.6–81.2], (4) Vidas IgG = 73.9% [67.6–80.1], (5) Abbott = 78.6% [63.4–93.8], (6) Beckman Coulter = 74.5% [62–86.9], (7) SD Biosensor IgM = 73.1% [61–85.1], and (8) SD Biosensor IgG = 76.9% [65.4–88.4]. Sensitivities increased gradually from week 1 to week 3 (Table 3): (1) Roche anti-N: 63.3%, 81%, and 82.1%; (2) Vidas IgM: 68.2%, 83.2%, and 85.9%; and (3) Vidas IgG: 66.7%, 79.1%, and 86.6%. All assays showed negative results for 100% of the pre-pandemic samples revealing a 100% [97.2–100] specificity.

Table 3.

Specificities and sensitivities of the investigated serology assays.

Test	Specificity [95% CI]	Overall sensitivity [95% CI]	Sensitivity 1st week [95% CI]	Sensitivity 2nd week [95% CI]	Sensitivity 3rd week [95% CI]	Sensitivity >21 days [95% CI]
Roche anti-N	100% [97.2–100]	74.26% [70.8–77.7]	50% [41.9–58]	78.75% [73.5–83.9]	87.58% [82.2–92.9]	82.27% [73.8–90.7]
Roche anti-S	100% [97.2–100]	93.24% [87.5–98.9]	—	—	—	—
Vidas IgM	100% [97.2–100]	80.79% [76.9–84.6]	63.95% [53.8–74.1]	84.21% [78.4–90]	90.82% [85.4–96.2]	87.5% [78.1–96.8]
Vidas IgG	100% [97.2–100]	80.43% [76.6–84.2]	58.62% [48.2–68.9]	80.72% [74.7–86.7]	90.17% [84.6–95.6]	95.91% [90.3–100]
Abbott	100% [97.2–100]	81.48% [66.8–96.1]	—	—	—	—
Beckman Coulter	100% [97.2–100]	74.35% [64.6–84]	—	—	—	—
Biosensor IgM	100% [97.2–100]	74.46% [65.6–83.2]	—	—	—	—
Biosensor IgG	100% [97.2–100]	78.72% [70.4–86.9]	—	—	—	—

ROC curves were built to determine the optimal cut-off which offer the best sensitivity/specificity combination for Roche anti-N and Vidas IgM/IgG assays as follow (Figure 1):

Figure 1.

SARS-CoV-2 serology results comparison in patients and controls. (a, c, e, g, I, k, m, and o) Box plots underlining significant higher anti-SARS-CoV Ab levels in COVID-19 patients comparatively to controls for each assay. (b, d, f, h, j, l, n, and p) ROC curves determining optimal cut-offs for evaluated assays.

• Roche anti-N: area under curve (AUC) = 92.8% [0.904–0.952], p < 0.001; Cut-off = 0.27: Sensitivity = 84%, specificity = 100%

• Roche anti-S: AUC = 95.4% [0.9–1], p < 0.001; Cut-off = 0.82 UI/ml: Sensitivity = 92.7%, specificity = 100%

• Vidas IgM: AUC = 93.3% [0.914–0.964], p < 0.001; Cut-off = 0.74: Sensitivity = 79.2%, specificity = 100%

• Vidas IgG: AUC = 85.1% [0.798–0.896], p < 0.001; Cut-off = 0.79: Sensitivity = 76.1%, specificity = 100%

• Abbott: AUC = 84% [0.674–1], p < 0.001; Cut-off = 1.44: Sensitivity = 78.6%, specificity = 100%

• Beckman Coulter: AUC = 87.5% [0.798–0.952], p < 0.001; Cut-off = 0.78: Sensitivity = 76.6%, specificity = 100%

• Biosensor IgM: AUC = 91.3% [0.855–0.972], p < 0.001; Cut-off = 0.83: Sensitivity = 76.9%, specificity = 100%

• Biosensor IgG: AUC = 83% [0.736–0.923], p < 0.001; Cut-off = 0.92: Sensitivity = 76.9%, specificity = 100%

Using the calculated cut-off for Roche anti-N, 0.27 COI, the sensitivity more than 14 days post-symptoms onset was 93.02% [0.868–0.991].

Serology results in reverse transcription-polymerase chain reaction-negative group (G2)

The G2 group consisted of 87 patients with a negative RT-PTR test. Serology results in G2 patients are summarized in Table 4. The pooled analysis showed that 32 (36.8%) patients had at least one positive serological test (Table 4).

Table 4.

Serology results in negative RT-PCR patients (G2).

Assay	Number of tested patients	Results
		Positive	Negative
Roche anti-N	80	22	58
Roche anti-S	5	3	2
Vidas IgM	33	15	18
Vidas IgG	34	11	23
Abbott	3	2	1
Beckman Coulter	13	5	8
Biosensor IgM	6	1	5
Biosensor IgG	6	2	4
Pooled serology	87	32	55

Agreements between reverse transcription-polymerase chain reaction and serology results

Agreements of investigated assays with RT-PCR were (Table 5): (1) Roche anti-N: 76.4%, κ = 0.321, p < 0.001; (2) Roche anti-S: 86.5%, κ = 0.218, p = 0.049; (3) Vidas IgM: 76.1%, κ = 0.146, p = 0.003; (4) Vidas IgG: 78.1%, κ = 0.22, p < 0.001; (5) Abbott: 74.1%, κ = 0.069, p = 0.718; (6) Beckman Coulter: 78.2%, κ = 0.354, p = 0.001; (7) Biosensor IgM: 71.3%, κ = 0.165, p = 0.102; and (8) Biosensor IgG: 73.4%, κ = 0.16, p = 0.119. As shown in Table 5, agreement of the Roche anti-N and the Vidas IgM/IgG assays with RT-PCR increased gradually from week 1 to week 3 post-symptoms onset.

Table 5.

Agreement of the evaluated serology assays with RT-PCR.

		RT-PCR		Agreement	κ	p
		Positive	Negative
Roche anti-N	Positive	404	50	76.4%	0.321	<0.001
	Negative	94	63
1st week	Positive	66	9	67.8%	0.354	<0.001
	Negative	39	35
2nd week	Positive	170	18	77.8%	0.248	<0.001
	Negative	35	16
3rd week	Positive	111	15	81.9%	0.246	0.003
	Negative	11	7
Roche anti-S	Positive	62	7	86.5%	0.218	0.049
	Negative	3	2
Vidas IgM	Positive	284	38	76.1%	0.146	0.003
	Negative	57	19
1st week	Positive	49	6	57.6%	0.215	0.027
	Negative	21	9
2nd week	Positive	114	13	80.8%	0.243	0.003
	Negative	16	8
3rd week	Positive	84	14	78.7%	−0.035	0.709
	Negative	9	1
Vidas IgG	Positive	297	34	78.1%	0.22	<0.001
	Negative	56	24
1st week	Positive	49	2	72.1%	0.365	<0.001
	Negative	22	13
2nd week	Positive	119	14	77.6%	0.197	0.01
	Negative	23	9
3rd week	Positive	87	13	80.2%	0.045	0.633
	Negative	9	2
Abbott	Positive	19	3	74.1%	0.069	0.718
	Negative	4	1
Beckman Coulter	Positive	53	5	78.2%	0.354	0.001
	Negative	12	8
Biosensor IgM	Positive	60	10	71.3%	0.165	0.102
	Negative	17	7
Biosensor IgG	Positive	63	11	73.4%	0.16	0.119
	Negative	14	6

Inter-assay agreement and correlation between quantitative results

An inter-test agreement analysis was performed for the anti-nucleocapsid tests on the one hand and for the anti-spike tests on the other hand. Reference assays were Roche anti-N for the anti-N assays and Vidas IgM/IgG for the anti-S tests. Agreements with the Roche anti-N assay were as follows: (1) Abbott: 98.4%, κ = 0.957, p < 0.001 and (2) Biosensor IgM and Biosensor IgG: 92.5%, κ = 0.841, p < 0.001. Agreements with the Vidas IgM/IgG test were as follows: (1) Roche anti-S: 99.4%, κ = 0.86, p < 0.001 and (2) Beckman Coulter: 95.65%, κ = 0.911, p < 0.001.

For correlations between quantitative results, a side-by-side comparison was performed (Figure 2). The strongest correlation was found between 2 IgG anti-S assays, the Vidas IgG, and the Beckman Coulter, Spearman Rho = 0.816, p < 0.001 (Figure 2(a)). Inversely, the weakest correlation between quantitative results was observed between the Abbott and the Beckman Coulter, Spearman Rho = 0.301, p = 0.036 (Figure 2(a)). The other inter-assays correlation Spearman Rho varied from 0.543 (Roche anti-N vs. Abbott) to 0.789 (Vidas IgM vs. Vidas IgG).

Figure 2.

Correlations between quantitative serology results. (a) Regression curves highlighting correlations between evaluated assays. (b) Spearman Rho correlation coefficients. Values are color-coded according to the 4 following intervals: (1) 0.8 to 1, (2) 0.6 to 0.8, (3) 0.4 to 0.6, and (4) 0.2 to 0.4.

Clinical evaluation of the serology assays

The Roche anti-N and the Vidas IgM/IgG assays were used for the clinical evaluation of the serology results. Total anti-SARS-CoV-2 Ab (Roche anti-N) frequency was significantly higher in patients with critical COVID-19 (70.9% vs. 59.4%), p = 0.018. Nevertheless, total Ab level was not associated with COVID-19 severity, p = 0.208 (Table 6). No significant association was found between disease severity and seropositivity or Ab titers with the Vidas IgM/IgG assay (Table 6).

Table 6.

Association of serology positive results with COVID-19 severity and outcome.

Assay	CoViD-19		p	Outcome		p
	Moderate	Critical		Recovery	Death
Roche anti-N positive	123 (59.4%)	127 (70.9%)	0.018	216 (69%)	34 (46.6%)	3.02 E-4
Roche anti-N COI	11.78	11.89	0.208	14.08	2.18	1.36 E-7
Vidas IgM positive	77 (66.4%)	75 (75%)	0.167	133 (73.1%)	19 (55.9%)	0.044
Vidas IgM i	5.23	6.48	0.183	6.47	2.25	0.001
Vidas IgG positive	74 (63.2%)	76 (72.4%)	0.147	133 (72.7%)	17 (43.6%)	4.26 E-4
Vidas IgG i	10.78	10.84	0.970	11.97	5.33	0.001

Inversely, frequencies of positive total Ab (Roche anti-N), IgM, and IgG were significantly higher in patients with recovery outcome; p < 0.001, p = 0.044 and p < 0.001, respectively. In addition, Total Ab, IgM, and IgG levels were significantly higher in patients with recovery outcome; p < 0.001, p < 0.001, and p < 0.001, respectively (Table 6).

Antibody dynamics after symptoms onset

Using the Roche anti-N assay, median titers of total Ab anti-N increased gradually from the 1st week to the 4th week after symptoms onset; 1st week = 1.045, 2nd week = 7.48, 3rd week = 18.17, 4th week = 23.7, p < 0.001 (Figure 3(a)). Likewise, anti-S IgG (Vidas IgG) median titers augmented progressively; 1st week = 1.97, 2nd week = 16, 3rd week = 26.95, 4th week = 29.42, p < 0.001 (Figure 3(a)). IgM anti-S median titers (Vidas IgM) peaked at the 3rd week then plateaued during the 4th weak; 1st week = 1.53, 2nd week = 5.47, 3rd week = 6.61, 4th week = 6.11, p < 0.001 (Figure 3(a)).

Figure 3.

Dynamics of antibody response in COVID-19 patients. (a) Error bars highlighting the kinetics of mean Ab titers with the Roche anti-N and the Vidas IgM/IgG assays. (b) Cumulative seroconversion rate in male and female patients. (c) Cumulative seroconversion rate in patients positive and negative RT-PCR results. (d) Cumulative seroconversion rate in patients with moderate and severe COVID-19.

Cumulative seroconversion rate after symptoms onset did not differ between male and female patients, p = 0.087 (Figure 3(b)). Inversely, the median seroconversion time was significantly shorter in patients with a positive RT-PCR, p < 0.001 (Figure 3(c)). Besides, a critical COVID-19 status was significantly associated with a shorter median conversion time, p < 0.001 (Figure 3(d)).

Discussion

In COVID-19 patients with negative RT-PCR tests, serology assays are able to determine whether a patient has previously been infected with SARS-CoV-2 or not. Although serology does not evaluate infectiousness, it could overcome the shortcomings of molecular tests. Hence, we present in this study the diagnostic performance of 6 commercial SARS-CoV-2 serology assays together with the Ab dynamics after symptoms onset in 443 COVID-19 patients.

In the present study, sensitivities of assessed assays ranged from 74.26% for the Roche anti-N assay to 93.24% for the Roche anti-S test. In samples obtained during the 1st week after symptoms onset, sensitivities fluctuated between 50% for the Roche anti-N test and 63.95% for the Vidas IgM assay. For the sera obtained more than 14 days after symptoms onset, sensitivities increased to reach 87.58%, 90.82%, and 90.17% for the Roche anti-N, the Vidas IgM and the Vidas IgG assays respectively. Our results corroborate those of previous studies. In the study of Pflüger et al., in which 5 automated serology assays were evaluated in 75 COVID-19 patients and 320 pre-pandemic sera, sensitivities 10 days or less after symptoms onset varied from 27% (Euroimmun) to 73% (Wantai), while it increased after 10 days and ranged from 68.4% for the Diasorin assay to 81.6% for the Wantai test. Another comparative study of 7 serology assays in 698 patients with PCR-confirmed COVID-19 revealed that sensitivities varied from 81.5% for the Beckman Coulter assay to 89.4% for an in-house ELISA in samples obtained 14 days or more after first positive RT-PCR. In line with these data, a recent meta-analysis of 27 published studies revealed pooled sensitivities of 69% for IgM, 76% for IgG, and 78% for total Ab assays. In addition, pooled sensitivities of IgM, IgG, and Total Ab were 25%, 34%, and 36%, respectively during the 1st week, but increased to 65%, 62% and 80% at 8–14 days, and 85%, 90%, and 93%, respectively, 14 days post-symptoms onset. Therefore, these findings together with our results indicate that serology assays are sensitive enough 14 days after symptoms onset for COVID-19 diagnosis when RT-PCR result is negative or performed late. This meta-analysis also revealed that ELISA tests had the highest sensitivity while the LFIA assays had the lowest sensitivity. Moreover, detection of total Ab targeting N and S combined could provide a better sensitivity comparatively to assays based on S or N alone. Nevertheless, the majority of published studies have shown lower sensitivities than those claimed by the manufacturers. This could be explained by the limited number of investigated patients used for the assays development by the manufacturers and possibly to high cut-offs. Hence, we used ROC curves to determine optimal cut-off for each assay. Overall, only the calculated cut-off for Roche anti-N assay, 0.27, was far below the manufacturer’s cut-off (≥1). Using this lower cut-off would increase the sensitivity to 93.02% for samples obtained more than 14 days after symptoms onset. A similar finding was reported by Favresse et al. Using a ROC curve cut-off of 0.165, Favresse et al. increased sensitivity to 95.1% for samples obtained at ≥14 days from RT-PCR positivity or symptom onset.

In this study, none of the 6 evaluated immunoassays revealed positive results in pre-pandemic samples. Consequently, specificity for all tests was at 100% [97.2–100]. Specificity of SARS-CoV-2 serology assays was disparately estimated. For instance, Péré et al., noted a 100% specificity in 117 pre-pandemic sera with the Abbott assay. In a comparative study, specificities were 99.1%, 99.4%, 99.7%, and 100% for Euroimmun and Diasorin, Wantai, Roche anti-N, and Siemens assays, respectively. In another comparative study, specificities for the assessed commercial assays ranged from 98.7% (Diasorin) to 100% (Roche anti-N), while it was at only 97.7% for the in-house ELISA. Nevertheless, some other studies evaluating mostly rapid LFIA and/or in-house ELISA assays revealed poor specificity.[18-20] Using the Alltest rapid LFIA assay, Perez-Garcia et al. noted that specificity of IgM, IgG, and total Ab were 80%, 68%, and 67%, respectively. Guo et al. developed an in-house ELISA using a recombinant nucleocapsid as coating antigen which exhibited an 85% specificity for IgM against SARS-CoV-2. Comparing 7 rapid LFIA tests, Van Elslande et al. noted an 85.4% specificity for total Ab anti-SARS-CoV-2. Poor specificity due to false-positive results could be explained by cross-reactivity with other seasonal human coronaviruses. The specificity of serology assays is critical in low-prevalence settings. For example, if the prevalence of infection is 1%, a test with 98% specificity would identify 2 false-positives for each true positive, thus exhibiting a positive predictive value of 33.33%. Therefore, at least 99% specificity is required for population screening of COVID-19 infection.

Besides, inter-assay agreements were high as the κ coefficient varied between 0.841 and 0.957 for the anti-N assays, and 0.86 to 0.911 for the anti-S tests. Head-to-head correlation between quantitative results of the assessed tests was moderate to high with Spearman Rho ranging from 0.301 (Abbott vs. Beckman Coulter) to 0.817 (Vidas IgG vs. Beckman Coulter). Good qualitative agreement with a moderate quantitative correlation was also reported in previous studies. In fact, in the study of Pflüger et al., and despite the high inter-assay concordance for qualitative results, Pearson correlation coefficient (r2) varied from 0.153 (Diasorin vs. Wantai tests) to 0.755 for 2 IgG assays (Diasorin vs Euroimmun). Moreover, the study of Suhandynata et al. showed only 18 discrepant results out of 339 tests while r2 across the 3 investigated platforms (Diazyme, Roche, and Abbott) ranged from 0.11 to 0.31. Low correlation might be to the fact that the mainstream of compared assays are qualitative tests.

Our results showed that seropositivity was associated with critical COVID-19 while SARS-CoV-2 Ab titers did not differ according to disease severity. This could be explained by the fact that patients with severe covid-19 have higher viral loads which could be at the origin of an earlier seroconversion caused by higher amounts of antigens which may further stimulate the immune system. Pflüger et al. showed that critically ill patients had higher IgG antibody responses compared to those with mild/severe disease. Furthermore, Oved et al. noted that SARS-CoV-2 Ab titers were significantly correlated to disease severity with a significant trend from asymptomatic to mild to moderate/severe patients. Two other studies[23,24] reported significantly higher titers in patients with moderate/severe COVID-19 comparatively to those with mild disease. This increased humoral response could be due to higher viral loads in severe COVID-19 patients. Nevertheless, a study performed in 176 COVID-19 patients reported similar Ab levels between mild/moderate and severe disease groups. Interestingly, Sun et al. showed that anti-Spike IgG titers were significantly higher in non-ICU patients while anti-Nucleocapsid IgG levels were statistically higher in ICU patients. This peculiar finding suggests that non-ICU patients would produce higher levels of neutralizing Ab than ICU patients. Overall, discrepancies between the above studies could be due to disparities in definition of the disease severity, the use of different serology assays with diverse targeted antigens and ab isotypes and different time to sampling after symptoms onset.

We noted that death outcome was significantly associated with less frequent positive results of SARS-CoV-2 serology and lower titers of total Ab, IgM, and IgG. This result is no surprise as it corroborates those of previous reports.[26-28] In fact, a strong humoral response in patients with severe COVID-19 have been reported in survivors while a weak humoral response was predictive of a death outcome.[26-28] Moreover, a significant increase of IgM, IgG, and IgA anti-S over time was associated with a survival outcome. Inversely, in a cohort of 79 severe/critical COVID-19 patients, no difference in IgG titers was noted between survivors and non-survivors, while IgM titers significantly declined over time in survivors. However, anti-SARS-CoV-2 Ab measurements in this study were performed later (on day 25 and repeated on day 27) than the other studies and the time between the 2 sampling operations was too short (2 days) to study IgM dynamics.

Antibody dynamics study revealed that total Ab anti-N (Roche anti-N) and IgG anti-S (Vidas IgG) increased gradually from week 1 to week 4 post-symptoms onset while IgM anti-S (Vidas IgM) reached the plateau at week 3. Likewise, a gradual increase in IgG anti-SARS-CoV-2 was reported in 60 COVID-19 patients using the Diazyme IgG, the Roche anti-N and the Abbott assays whereas IgM (Diazyme IgM) peaked 8–14 days positive PCR and decreased during the 3rd week. Similarly, Ren et al. noted an increase of N-IgG and S-IgA levels between weeks 1–2 and week 3 after symptoms onset. In a cohort of 38 COVID-19 patients, N-IgM and S-IgM reached a peak in the 2nd week post-symptoms onset then declined in the 3rd week in some non-ICU patients while N-IgG and S-IgG continued to increase during the 3rd week. Nevertheless, if antibodies to N and S protein had a parallel dynamic in non-ICU patients, the dynamic pattern in ICU patients was more chaotic. In fact, in most ICU patients, S-IgG increased slowly compared to N-IgG. The authors suggested that an early class switch from IgM to IgG anti-S could protect against severe COVID-19. Overall, data from the above studies indicate that IgG anti-SARS-CoV-2 (N and/or S) levels continued to increase during the 3rd/4th weeks post-symptoms onset/positive RT-PCR while IgM titers waned in some patients.

In the present study, the median seroconversion time did not differ between male and female patients while it was significantly shorter in patients with positive RT-PCR result and critical COVID-19. The impact of disease severity on the median seroconversion time post-symptoms onset was disparately estimated. In a study on 80 hospitalized patients for COVID-19, the median seroconverting time post-exposure was not different between critical and non-critical cases. Inversely, Ren et al. noted a significant delayed response of IgM, IgG, and IgA against N and S in severe COVID-19 patients comparatively to those mild/moderate disease. Therefore, this reported delayed humoral response in severe patients highlights the role of Nab in COVID-19 outcome. However, the above discrepancies might be due to different study designs and COVID-19 severity definition and requires replication in larger independent cohorts.

There are some limitations in the present study. The numbers of performed tests were relatively low for the Abbott, the Beckman Coulter, and the Biosensor SD assays. The group of patients with 3 samples was small (n = 52) which impacted the Ab dynamics study. Of note, follow-up of patients and serial sampling was challenging midst the epidemic when there was an urgent need to free up hospital beds to admit new patients. Unfortunately, SARS-CoV-2 viral load was not available and correlation with Ab titers could not be assessed.

Conclusions

Based on these findings, serology assays are useful in the diagnosis of suspected COVID-19 infections during the week 2–3 after symptoms onset. Robust humoral response is predictive of recovery outcome.

Click here for additional data file.

Supplemental Material, sj-pdf-1-iji-10.1177_20587384211073232 for Comparative study of six SARS-CoV-2 serology assays: Diagnostic performance and antibody dynamics in a cohort of hospitalized patients for moderate to critical COVID-19 by Sameh Chamkhi, Tarak Dhaouadi, Imen Sfar, Salma Mokni, Alia Jebri, Dhouha Mansouri, Salma Ghedira, Emna Ben Jemia, Samia Ben Boujemaa, Mohamed Houissa, Hichem Aouina, Taïeb Ben Abdallah and Yousr Gorgi in International Journal of Immunopathology and Pharmacology