Evaluation of two automated and three rapid lateral flow immunoassays for the detection of anti-SARS-CoV-2 antibodies.

INTRODUCTION: Several SARS-CoV-2 immunoassays have been developed recently. The purpose of this study was to assess the performance of five immunoassays for the detection of SARS-CoV-2 antibodies.
METHODS: Two quantitative automated immunoassays (Maglumi™2019-n-Cov IgG and IgM and Euroimmun Anti-SARS-CoV-2 IgG and IgA assays) and three lateral flow rapid tests were performed. This retrospective study included 200 residual sera from patients and healthy volunteers. Case serum samples (n = 128) were obtained from COVID-19 patients confirmed by RT-qPCR and CT-scan. Days since onset of symptoms was collected from their medical records. Control non-SARS-CoV-2 samples (n = 72) were obtained from anonymous stored residual serum samples.
RESULTS: Maglumi™ IgG/IgM tests showed overall less sensitivity than Euroimmun IgG/IgA test (84.4 % versus 64.3 %). Both tests showed similar specificities of IgG at 99 % and 100 %, respectively. The results from the lateral flow assays were easily interpretable with unambiguous coloured reading bands. The overall sensitivity of the three tests was similar (around 70 %) without any significant differences. The sensitivity of the three lateral flow assays and also of the serological quantitative assays increased during the second week after symptom onset and all reached similar values (91 %-94 %) after 14 days.
CONCLUSION: This study shows accurate and equivalent performance of the five serological antibody assays (ELISA, CLIA and three lateral flow tests) in detecting SARS-CoV-2 antibodies 14 days after the onset of COVID-19 symptoms. This is compatible with their application in specific clinical contexts and in determining epidemiological strategies for the COVID-19 pandemic.

Introduction

Since the emergence of the novel respiratory virus SARS-CoV-2 during December 2019 in the region of Wuhan (China), the virus has spread rapidly all over the world causing a pandemic coronavirus disease (COVID-19) [1]. Adequate diagnosis of SARS-CoV-2 infection is essential for prompt therapeutic management of patients, control of the epidemic and the establishment of infection control measures. Even though RT-qPCR is considered the reference method for screening and diagnosis, the sensitivity of this method may vary depending on the quality and origin of the sample, the time of infection and the viral load [2,3].

On 30 January 2020, the World Health Organization (WHO) declared the COVID-19 outbreak to be a Public Health Emergency of International Concern, and shortly thereafter called for research on in-vitro diagnostics for use at the community level [4]. In response, several serological tests, including the enzyme-linked immunosorbent assay (ELISA), the chemiluminescence immunoassay (CLIA) and lateral flow rapid testing, are now under development or have already entered the market. The Foundation for Innovative New Diagnostics (FIND) (https://www.finddx.org/) lists over 150 rapid COVID-19 antibody tests that are “Communauté Européenne” (CE) marked [5].

The detection of IgG, IgM and IgA antibodies against the SARS-CoV-2 can play a complementary role to the RT-qPCR test in the diagnosis of COVID-19 and in assessing the immune status of individuals. Moreover, serological data will enable the gathering of important epidemiological information, providing more realistic data on the spread of the epidemic, and on morbidity and mortality. In addition, the detection of antibodies against SARS-CoV-2 will play a key role in determining appropriate lockdown exit strategies and in vaccine development [[2], [3], [4], [5], [6], [7]]. However data concerning the performance of these assays are scarce [8,9]. The purpose of this study was therefore to assess the performance of CE marked assays available in Belgium—three lateral flow rapid tests and two quantitative automated immunoassays—for the detection of SARS-CoV-2 IgG, IgM and IgA antibodies.

Material and methods

Patients and serum samples

This retrospective study included 200 residual sera from patients and healthy volunteers from Laboratoire Hospitalier Universitaire de Bruxelles - Universitair Laboratorium Brussel (LHUB-ULB) and the Microbiology Department of Cliniques Universitaires Saint Luc-UCLouvain (CUSL) in Brussels, Belgium. Case serum samples (n = 128) were obtained from COVID-19 patients confirmed by RT-qPCR and CT-scans. The RT-qPCR kits used were: RealStar® SARS-CoV-2 RT-PCR kit 1.0 (Altona Diagnostics, Hambourg, Germany) at LHUB-ULB and Genesig® Real-Time PCR Coronavirus (COVID-19) (Primerdesign Ltd, Chandlers Ford, United Kingdom) at CUSL. Information about days since onset of symptoms was collected from the medical records. Control non-SARS-CoV-2 samples (n = 72) utilised anonymous stored residual serum samples, selected as follows: 1) Sera selected from January 2018 to August 2019 (n = 62) included samples with a potential cross-reaction to the SARS-CoV-2 immunoassays, namely, EBV infection (n = 5), CMV infection (n = 11), M. pneumoniae infection (n = 8), Parvovirus infection (n = 1), HBV infection (n = 1), Bartonella henselae infection (n = 1), Brucella spp infection (n = 1), autoimmune pathologies (Anti-DNA, n = 1; Anti-PL12, n = 1; Anti Scl-70, n = 1). 2) Sera from healthy volunteers (n = 10) obtained during the epidemic period (April 2020).

The study was approved by the Ethical Committee (ref CUSL: 2020/06avr/203)

Serological assays

ELISA assay

The Euroimmun Anti-SARS-CoV-2 ELISA IgG and IgA assays (Euroimmun, Luebeck, Germany) were performed on serum samples according to the manufacturer’s instructions for ELISA automated systems: the ETI-MAX 3000 (DiaSorin, Saluggia, Italy) at LHUB-ULB, and the Analyzer 1™ (Euroimmun) at CUSL. These ELISA assays provide a semiquantitative in vitro determination of human antibodies of the immunoglobulin classes IgG and IgA against the SARS-CoV-2. The microplate wells are coated with recombinant S1 structural protein. The results are evaluated semi-quantitatively by calculation of a ratio of the extinction of samples over the extinction of the calibrator. The ratio interpretation was as follows: <0.8 = negative, ≥0.8 to <1.1 = borderline, ≥1.1 = positive. Borderline data were considered positive for the statistical analyses.

CLIA assay

The Maglumi™2019-n-Cov IgG and IgM are fully automated quantitative chemiluminescent immunoassays (CLIA) using magnetic microbeads coated with SARS-CoV-2 recombinant antigen labelled with ABEI, a non-enzyme small molecule with a special molecular formula that enhances stability in acid and alkaline solutions. The IgM and IgG assays were performed on serum samples, according to the manufacturer’s instructions, on the Maglumi™ 800 analyser (Snibe Diagnostic, Shenzhen, China). The thresholds of positivity for these automated immunoassays are 1.0 AU/mL for IgM and IgG.

Lateral flow tests

Three lateral flow tests were used according to the manufacturer’s instructions with 10 μL of serum. The results were read and interpreted 10 min after the test.

The 2019-n-CoV IgG/IgM rapid test cassette (LaboOn Time) (LabOn Time, Bio Marketing Diagnostics, or Akiva, Israel) is a lateral flow chromatographic immunoassay for the qualitative detection of IgG and IgM antibodies against SARS-CoV-2 in human whole blood, serum or plasma specimens. This test contains anti-human IgM and anti-human IgG as the capture reagent and SARS-CoV-2 antigen as the detection reagent. A goat anti-mouse IgG is employed in the control line system.

The Novel Coronavirus (2019-n-CoV) antibody IgG/IgM assay (colloidal gold) (Avioq) (Avioq, Bio-Tech, Shandong, China) is intended for the in vitro qualitative determination of IgG and IgM antibodies against SARS-CoV-2 in human whole blood, serum or plasma specimens and uses a colloidal gold-immunochromatographic system. This test contains recombinant SARS-CoV-2 antigen labelled by colloidal gold and colloidal gold-labelled rabbit antibody, fixed monoclonal IgG anti-SARS-CoV-2 antibody and fixed monoclonal IgM anti-SARS-CoV-2 antibody. A goat anti-rabbit IgG antibody is employed in the control line system.

QuickZen COVID-19 IgM/IgG Kit (QuickZen) (ZenTech, Angleur, Belgium) is an immune colloidal gold technique intended for the qualitative detection of IgG and IgM against SARS-CoV-2 in human whole blood, serum or plasma specimens. The reagent-binding pad is coated with colloidal gold-labelled recombination antigen and rabbit IgG antibodies serve as control.

Statistical analyses

Statistical analysis was performed with SPPS software. A receiver operator characteristic (ROC) curve was constructed and used for comparisons of the area under the curve (AUC) of the ROC curves. The Cohen Kappa index was calculated for agreement between all analysed assays. A p value <0.05 was considered statistically significant. Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were calculated for each serological test.

Results

Sensitivity and specificity obtained with quantitative (ELISA and CLIA) serological assays are summarized in Table 1 . Overall the ELISA assay showed higher sensitivity than the CLIA (84 % versus 64 %, respectively). In contrast, the specificity of CLIA IgM (100 %) was greater than that observed for ELISA IgA (86 %). Both tests showed similar specificities of IgG at 99 % for CLIA and 100 % for ELISA. As shown in Table 2 , during the first week after the onset of symptoms, the ELISA IgA analysis was significantly more sensitive than the CLIA IgM (p < 0.001). Although not-statistically significant, higher sensitivity was also observed for the ELISA IgG compared to the CLIA IgG (p = 0.45). The sensitivity of the CLIA IgG/IgM increased one week after symptom onset, reaching levels equivalent to those of the ELISA IgG/IgM assay from the third week.

Table 1

Analytical sensitivities, specificities, positive predictive values (PPV) and negative predictive values (NPV) for SARS-CoV-2 antibody detection.

	Number of serum samples
	SARS-CoV-2 positive sera	Control negative sera	Sensitivity(IC 95 %)	Specificity(IC 95 %)	PPV (IC 95 %)	NPV (IC 95 %)
ELISA
IgA	107/128	62/72	83.6(76.2–89,0)	86.1 %(76.3−92.3)	91.5 %(85.0−95.3)	74.7 %(64.4–82.8)
IgG	79/128	71/72	61.7(53.1−69.7)	98.6(92.5−99.8)	98.8(93.3−99.8)	59.2(50.2−67.5)
Combined (IgA or IgG)	108/128	63/72	84.4 %(77.1−89.7)	87.5 %(77.9 %93.3)	92.3 %(86.0−95.9)	75.9 %(65.7−83.8)
CLIA
IgM	74/126	72/72	58.7%(50−66.9)	100 %(94.9−100)	100 %(95.1−100)	58.1 %(49.3−66.4)
IgG	67/126	72/72	53.2%(44.5−61.7)	100 %(94.9−100.0)	100 %(94.6−100.0)	55 %(46.4−63.2)
Combined(IgM or IgG)	77/122	72/72	64.3 %(55.6−72.1)	100 %(94.9−100)	100 %(95.5−100)	61.5 %(52.5−69.9)
Lateral flow test
Avioq
IgM	88/128	69/72	68.8%(60.3−76.1)	95.8 %(88.5−98.6)	96.7 %(90.8−98.9)	63.3 %(53.9−71.8)
IgG	88/128	69/72	68.8%(60.3−76.1)	95.8 %(88.5−98.6)	96.7 %(90.8−98.9)	63.3 %(53.9−71.8)
Combined IgM or IgG	88/128	69/72	68.8%(60.3−76.1)	95.8 %(88.5−98.6)	96.7 %(90.8−98.9)	63.3 %(53.9−71.8)
QuikZen
IgM	88/128	72/72	68.8%(60.3−76.1)	100 %(94.9−100)	100 %(95.8−100)	64.3 %(55.1−72.6)
IgG	63/128	72/72	49.2%(40.7−57.8)	100 %(94.9−100)	100 %(94.3−100)	52.6 %(44.2−60.7)
Combined IgM or IgG	91/128	72/72	71.1%(62.7−78.2)	100 %(94.9−100)	1000%(95.9−100)	661%(56.8−74.3)
LabOn Time
IgM	62/128	72/72	48.4%(40−57)	100 %(94,9−100)	100 %(94,2−100)	52.2 %(43,9−60,3)
IgG	86/128	72/72	67.2%(58.7−74.7)	100 %(94.9−100)	100 %(95.7−100)	63.2 %(54−71.4)
Combined IgM or IgG	92/128	72/72	71.9%(63.5−78.9)	100 %(94.9−100)	100 %(96−100)	66.7 %(57.3−4.8)

Table 2

Analytical sensitivities for SARS-Cov-2 serological test depending on the onset of COVID-19 symptoms.

	Number of days
	0 to7 daysN (%)	8 to 14 daysN (%)	15 or more daysN (%)
ELISA
IgA or IgG	19/29 (65.5)	55/62 (88.7)	31/33 (93.93)
IgA	19/29 (65,5)	54/62 (87.09)	31/33 (93.93)
IgG	5/29 (17.2)	41/62 (66.12)	30/33 (90.9)
CLIA
IgM or IgG	5/28 (17.85)	43/62 (69.35)	30/32 (93.75)
IgM	4/28 (14.3)	40/62 (64.51)	28/32 (87.5)
IgG	2/28 (7.14)	34/62 (54.8)	28/32 (87.5)
Lateral flow test
Avioq	8/29 (27.58)	46/62 (74.19)	31/33 (93.93)
QuickZen	10/29 (34.48)	48/62 (77.41)	30/33 (90.9)
LabOn Time	11/29 (37.9)	47/62 (75.8)	31/33 (93.93)

Among the negative control samples, ten false positive results were observed with ELISA IgA (13.8 %). Six of these cross-reacted with serum containing antibodies against EBV (n = 2), M. pneumoniae (n = 3), Anti-PL12 (n = 1) and four were without any known confounding factor. No false positive results were obtained with the CLIA IgG/IgM test.

The distribution of the negative data points for both tests (Fig. 1, Fig. 2 ) showed a better separation of the ELISA IgG and CLIA IgG detected values than of the ELISA IgA and CLIA IgM values (ELISA IgA 0.612 ratio and CLIA IgM 0.531 UA/mL mean value, versus ELISA IgG 0.327 and CLIA IgG 0.132 mean values).

Fig. 1

ELISA IgA (A) and IgG (B) data distribution obtained for negative control and infected SARS-CoV-2 patients’ sera.

Fig. 2

CLIA IgM (A) and IgG (B) data (UA/mL) distribution obtained for negative control and infected SARS-CoV-2 patients’ sera.

Comparative analyses of the ROC curves (Fig. 3 ) from ELISA and CLIA showed significantly higher AUC for ELISA IgA (0.893; 95 % CI: 0.840−0.934) than for CLIA IgM (0.766; 95 % CI: 0.698−0.825) (p < 0001). In contrast, no significant AUC differences were observed between ELISA IgG (0.803; 95 % CI: 0.740−0.857) and CLIA IgG (0.826; 95 % CI: 0.766−0.877) (p = 0.485).

Fig. 3

Comparative ROC curves for ELISA IgG/IgA and CLIA IgG/IgM.

Results for the lateral flow assays were easily interpretable with unambiguous coloured reading IgG lines. The colour intensity in the line regions correlated with the concentration of SARS-Cov-2 antibodies. In some cases, IgM line presented slight difficulty for reading in all commercial lateral flow tests. As predicted by the manufacturer, the results were obtained after ten minutes or less in all cases. All the tests performed in this study provided valid results. Overall sensitivity was similar (around 70 %) without any significant differences between the three tests (Table 1). However, the sensitivity for IgM was significant lower (p < 0.001) with LabOn Time as compared to the QuickZen and Avioq assays. The Avioq lateral flow test showed three false positive results of concomitant IgM and IgG reactivity in three samples from a previous epidemic period without any known confounding factor. In contrast, no false positive results were observed among the pool of 31 sera containing antibodies with potential cross reactivity. Globally, no significant differences were observed between the specificity values of the three tests with a positive predictive value of 100 % for QuickZen and LabOn Time and 97.7 % for the Avioq assays. As shown in Table 2, similarly to the serological quantitative assays (ELISA and CLIA), the sensitivity of the three lateral flow assays increased during the second week after the onset of symptoms and achieved similar values (91 %–94 %) after 14 days.

Table 3 shows the percentage of agreement between the five immunoassays. Overall, the best agreement was observed between the CLIA and LabOn Time assays (91 %; Cohen Kappa index of 0.819) and between the two lateral flow assays LabOn Time and Avioq (91.5 %; Cohen Kappa index of 0.829). The absence of agreement between the five serological assays observed during the first week following symptom onset lessened 14 days after symptom onset, with all assays achieving 97%–100% agreement.

Table 3

Agreement between analyzed serological assays.

		Euroimmun% (Kappa)	Maglumi	Avioq	LabOn Time
Overall	Maglumi	82 (0.654)
	Avioq	81 (0.626)	89 (0.779)
	LabOn Time	86.5 (0.734)	91 (0.819)	91.5 (0,829)
	QuickZen	84 (0.685)	88 (0.759)	87 (0.738)	89.5 (0.788)
First week of symptoms	Maglumi	51.7 (0.213)
	Avioq	55 (0.213)	75,8 (0.362)
	LabOn Time	65.5 (0.358)	69 (0.298)	82.8 (0,627)
	QuickZen	69 (0.434)	72.4 (0.345)	72.4 (0,359)	82.8 (0.627)
After 14 days of symptoms	Maglumi	97 (0.787)
	Avioq	100 (1)	97 (0.787)
	LabOn Time	100 (1)	97 (0.787)	100 (1)
	QuickZen	96.7 (0.784)	97 (0.82)	97 (0.784)	97 (0.784)

Discussion

Serological testing is a complementary test in COVID-19 diagnosis and a strategic vehicle in the second phase of the pandemic, necessary for epidemiological study and lockdown exit programmes. Immunoassays could provide identification of non-contagious and potentially protected individuals to support progressive de-confinement strategies in the process of gradually restoring safe economic and social activity [10,11]. Different types of serological tests are available on the market and could be applied to the massive testing challenge the world is currently facing.

Fully automated CLIA and ELISA assays allow the quantitative determination of antibodies against SARS-CoV-2 by clinical laboratories with increased screening capacity. In this study, the ELISA IgG/IgA was tested on a fully automated microtiter plate analyser. The workflow for ELISA tests depends on the analyser used: in our laboratory, the capacity was 90 tests per 3−4 h. In contrast, the CLIA IgG/IgM assays are fully automated random-access tests. This technology allows up to 180 tests per hour, depending on the platform used, with the results delivered in about 40 min. The performance of Maglumi IgG/IgM tests has been successfully evaluated by Padoan et al., showing the reliability of these immunoassays for assessing the immunological response in the sera of COVID-19 patients. These tests show that it takes at least 12 days to reach 100 % sensitivity for IgG and a 88 % positive rate for IgM [12]. In another study that compared the Maglumi IgG/IgM and Euroimmun IgG/IgA, 100 % sensitivity was observed with Maglumi IgG 10 days following onset of symptoms. These authors reported a lower sensitivity for the Maglumi IgM (60 %) compared to the Euroimmun IgA (100 %) [13]. In the current study, the Maglumi IgG/IgM tests also showed overall lower sensitivity than the Euroimmun IgG/IgA test (64.3 % vs 84.4 %), but in contrast, their specificity increased to 100 %. The mean days since symptom onset in the population tested for this study was 11.46 days (median of 10 days). This can explain the lower sensitivity of the Maglumi IgG/IgM tests in our study as compared to those in the literature. Obka et al. [8] also described higher sensitivity observed in the Euroimmun IgA test than in the Euroimmun IgG test (83.6 % vs 61.7 %). This was particularly evident in the two first weeks after symptom onset, but with less specificity for the former test (86.1 % vs 98.6 %).

Recently, many commercial lateral flow assays have been developed and CE-labelled. In this study, three of these were also evaluated. The results showed that their global sensitivity, specificity, NPV and PPV were equivalent to the ELISA IgG/IgA or the CLIA IgG/IgM tests. In addition, excellent concordance between the five immunoassays tested in this study was observed 14 days after the onset of symptoms. Similar results were reported by Li et al., showing good sensitivity (88.6 %) and specificity (90.6 %) with a rapid test in a large cohort of patient samples in China. They observed similar performances with serum, plasma and fingerstick blood [14]. Rapid serological tests can be performed in the laboratory or used as point-of-care tests (POCT). The latter will provide accurate results within 10−15 min with equivalent sensitivity and specificity as the quantitative automated immunoassays, particularly two weeks after onset of symptoms. The opportunity to test outside of the clinical laboratory by lateral flow assay makes it possible to reach larger population groups without saturating the capacity of the laboratories. POCT may play an important role in large-scale testing in order to evaluate herd immunity against SARS-CoV-2. However, mistakes in the interpretation of results in situations that are not under the control of trained staff must be taken into consideration. For this reason, the development of automated reader devices could help to reduce errors and increase sensitivity. In addition, such a device could support the transmission of the results to a public health institution to provide real-time information about seroprevalence at the population level.

While still awaiting the results of large seroprevalence studies in the community, Wu et al. report a detection rate of around 10 % of SARS-CoV-2 IgG in asymptomatic subjects from a single-centre investigation [15]. The causal relationship between humoral response and illness severity is still unclear. Zhao et al. revealed a strong positive correlation between clinical severity and antibody titres two weeks after illness onset [16]. Yongchen et al. highlight the complementary role of immunoassays to RT-qPCR in the diagnosis of COVID-19, particularly in critical patients with a negative RT-qPCR. They observed seronegative asymptomatic patients four weeks after positive RT-qPCR results [17]. In the current study, two asymptomatic patients tested seronegative by all the immunoassays 20 days after a positive RT-qPCR result. Further studies in asymptomatic patients are necessary to understand better the humoral responses in this population and to understand the role of the real herd immunity in determining lockdown exit strategies.

This study has some limitations. First, this is a retrospective study performed with residual samples and the lack of fresh serum could impact the accuracy of the results. Second, no reliable gold standard for serological tests is currently available for comparative studies, and little literature exists concerning a comparison of immunoassay methods for SARS-CoV-2 detection. Furthermore, the criteria for assessing the time of illness onset were recovered from medical records and may contain imprecisions due to subjectivity in the perception of symptoms and timing.

In conclusion, this study shows the accurate and equivalent performance of five serological antibody assays (ELISA, CLIA and three lateral flow tests) in detecting SARS-CoV-2 antibodies 14 days after the onset of COVID19 symptoms, making them compatible for application in the clinical context and in developing epidemiological strategies for the COVID-19 pandemic.

Declaration of Competing Interest

The authors declare that they have no conflict of interest. 2019-nCoV IgG/IgM rapid test cassette (Lab on time, Bio Marketing Diagnostics, Or Akiva, Israel), Novel Coronavirus (2019-n-CoV) antibody IgG/IgM assay (colloidal gold) (Avioq, Bio-Tech, Shandong, China) and QuickZen COVID-19 IgM/IgG Kit (ZenTech, Angleur, Belgium) have offered the reagents for validation.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.