Evaluation of three immunochromatographic tests in COVID-19 serologic diagnosis and their clinical usefulness.

Results of three rapid immunochromatographic tests (ICTs) were compared with those obtained with two automated immunoassays for evaluation of their usefulness. One hundred fifty-nine patients and 67 healthy volunteers were included. Different assays demonstrate 41-45% of diagnostic sensitivities and 91-98% of specificities, with substantial agreement (89.3-91.2%), but a high percentage of weak positive results (13-22%) was observed with ICTs. ICTs performances were comparable to those of automated immunoassays. ICTs could have a role as screening approach due to their easy usability. Subjective interpretation, significant rate of uncertain results, uncertainty on viral antigens source are undoubtedly drawbacks.

As COVID-19 pandemic is still ongoing [1], specific antibodies detection is of outmost importance. Besides the availability of immunoenzymatic (ELISA) and chemiluminescence (CLIA) anti SARS-CoV-2 antibodies immunoassays, many other rapid antibody-detecting tests based on immunochromatographic techniques have been recently commercialized. Although the WHO does not clearly endorse their use for patient management, interest is raising around these devices, mainly supported by their easy usability [2, 3]. The aim of this study was to compare diagnostic performances of three rapid immunochromatographic tests with those of an automated ELISA and CLIA immunoassays, in order to evaluate their potential usefulness as diagnostic and/or epidemiological tools.

Study population consisted of 159 patients (78 males; 81 females; median age 58 ± 20 years; range 6–97 years) admitted to the emergency room, medical, and intensive care units (ICUs) of the Azienda Ospedaliera Universitaria Integrata of Verona with symptoms suggestive of SARS-CoV-2 infection, between the end of February and the beginning of April. Control group consisted of 67 healthy volunteer’s (10 males; 57 females; median age 49 ± 9 years; range 26–66 years). For each patient, upper respiratory specimen and blood sample were collected for COVID-19 molecular and serologic diagnosis, respectively. Negative and inconclusive RT-PCR test results were investigated by systematically repeating the molecular test during some consecutive days. RT-PCR results were then matched with clinical and epidemiological data in order to identify confirmed and suspected COVID-19 cases.

In COVID-19 confirmed cases (symptomatic patient with SARS-CoV-2 positive molecular detection), date of symptoms onset was used to timing infection at the moment of specimens’ collection. Three stages were identified: early (0–7 days from symptoms onset), intermediate (8–13 days), and late (≥ 14 days).

Blood serum samples were analyzed by different assays according to the manufacturer’s protocols. The SARS-CoV-2 IgG/IgA ELISA (Euroimmun AG, Luebeck, Germany) is an automated semi quantitative immunoassay for detection of IgG and IgA directed against S1 domain of spike (S) viral protein. The SARS-CoV-2 IgG/IgM CLIA immunoassay (MAGLUMI 2019-nCoV IgG/IgM, SNIBE–Shenzhen New Industries Biomedical Engineering Co., Ltd, Shenzhen, China) is an automated quantitative CLIA that detects antibodies directed against spike and nucleocapsid (N) viral proteins. VivaDiagTM (VivaChek Biotech-Co., Ltd, Hangzhou, China), COVID-19 IgG/IgM Rapid Test Cassette (Zhejiang Orient Gene Biotech Co., Ltd Huzhou, Zhejiang, China), and PRIMA Professional (PRIMA Lab SA, Balerbna, Switzerland) are lateral flow immunochromatographic assays. They differ in sample (serum, plasma, or whole blood) quantity need (5–20 μl) and time of incubation, but test principle is identical. Results are qualitative (i.e., positive/negative), interpreted by visual reading and are generated in 10–15 min.

SARS-CoV-2 molecular detection was performed with a commercial real-time PCR method, Seegene Allplex TM2019-nCoV2 (Seegene, Seoul, South Korea). [4].

For each assays, IgG, IgM, and IgA positive and negative rates were calculated. Sensitivities were assessed on confirmed COVID-19 cases, combining IgG and IgM/IgA positive results, while specificities were estimated on the group of healthy volunteer’s. Agreement with Cohen’s Kappa test was used to compare ICTs vs ELISA Euroimmun and CLIA MAGLUMI. The study has been cleared by the local Ethical Committee (University Hospital of Verona; SOPAV-2; protocol no. 35747) and was performed in agreement with the Declaration of Helsinki, under the terms of relevant local legislation.

SARS-CoV-2 molecular detection was initially positive in 75 (47%) patients, negative or inconclusive in 48 (30%), and 36 (23%) patients, respectively. Nineteen out of 36 patients with initial inconclusive RT-PCR, resulted positive when molecular assay was repeated in the following days. Molecular results, matched with clinical and epidemiological data, allowed the identification of 94 COVID-19 and 65 suspected cases (patients seeking medical care for respiratory symptoms with negative or inconclusive RT-PCR results, radiological findings not available). Date of symptoms onset were available for 82 out of 94 (87%) COVID-19 confirmed cases. In 40 cases, blood samples were collected < 7 days after symptoms onset (early stage), in 21 patients between 8 and 13 days (intermediate stage), and in 21 cases ≥ 14 days (late stage).

IgG and IgM positive rates varied between 20.7–28.9% and 11.3–27.6%, respectively (Table 1). With ICTs, weak positivity rates varied from 13–22% for IgG (22% VivaDiag; 13% COVID-19 IgG/IgM Rapid Test Cassette; 15% PRIMA Professional) to 20–89% for IgM (20% VIVADIAG; 66% COVID-19 IgG/IgM Rapid Test Cassette; 89% PRIMA Professional). With respect to the ELISA assay, IgA displayed an overall positive rate higher than that of IgG (29% vs 20%).

Table 1

Positive and negative rates and sensitivity of the different serological assays used in this study

	ELISA-Euroimmun		CLIA-MAGLUMI		ICT-VivaDiag		ICT-COVID-19 G/M		ICT-PRIMA Professional
	IgG no (%)	IgA no (%)	IgG no (%)	IgM no (%)	IgG no (%)	IgM no (%)	IgG no (%)	IgM no (%)	IgG no (%)	IgM no (%)
Positivity (%)	33 (20.7)	47 (29.5)	45 (28.3)	24 (15)	41 (25.7)	44 (27.6)	38 (23.8)	39 (24.5)	46 (28.9)	18 (11.3)
Negativity (%)	126 (78.7)	112 (70.6)	114 (73.1)	145 (88.7)	118 (73.7)	115 (71.8)	121 (75.6)	120 (75.6)	113 (70.6)	141 (88.1)
Sensitivity	41%		42%		42%		40%		42%

Overall assays sensitivities were 40–42% (Table 1). According to disease stage, sensitivities increased from 2.5–12.5% at early stage to 90–95% at late stage (Table 2). Specificities were as follows: 91% for ELISA-Euroimmun, 98% for CLIA-MAGLUMI, ICT-VivaDiag, and COVID-19 IgG/IgM Rapid Test Cassette, 97% for ICT-PRIMA Professional (Table 3). Combining IgG/IgM/IgA results, positive rates varied between 0 and 10% at early stage, increased to 66% at intermediate stage, and peaked at 95% ≥ 14 days after symptoms onset.

Table 2

Positivity rate of IgG, IgM, and IgA according to disease stage: early (1), intermediate (2), late (3), and relative sensitivities of IgG/IgM or IgA combined

			ELISA-Euroimmun		CLIA-MAGLUMI		ICT-VivaDiag		ICT-COVID-19 G/M	ICT-PRIMA Professional
Stage of disease	Days from symptoms onset	Samples (no)	IgG no (%)	IgA no (%)	IgG no (%)	IgM no (%)	IgG no (%)	IgM no (%)	IgG no (%)	IgM no (%)	IgG no (%)	IgM no (%)
1	0–7 days	40	0 (0)	1 (2.5)	4 (10)	2 (5)	2 (5)	4 (10)	1 (1*) (2.5)	4 (3*) (10)	2 (5)	0 (0)
		Sensitivity	2.5%		12.5%		10%		10%	5%
2	8–13 days	21	8 (38)	10 (47)	13 (62)	7 (33)	10 (2*) (47)	10 (2*) (47)	10 (1*) (47)	10 (5*) (47)	14 (4*) (66)	5 (5*) (24)
		Sensitivity	47.6%		57%		47.6%		47.6%		66.7%
3	≥ 14 days	21	18 (86)	19 (90)	20 (95)	13 (62)	20 (3*) (95)	20 (3*) (95)	19 (1*) (90)	17 (12*) (77)	20 (1*) (95)	5 (4*) (24)
		Sensitivity	95%		95%		95%		90%	95%

*number of weak positives

Table 3

Assays’ specificities on healthy volunteers

	ELISA-Euroimmun		CLIA-MAGLUMI		ICT-VivaDiag		ICT-COVID-19 G/M		ICT-PRIMAProfessional
	IgG	IgA	IgG	IgM	IgG	IgM	IgG	IgM	IgG	IgM
Positives no	1	6	1	0	0	1	1	0	2	2
Negatives no	66	61	66	67	67	66	66	67	65	65
Specificity (%)	98	91	98	100	100	98	98	100	97	97

(%)

(%)

(%)

(%)

(%)

(%)

(%)

(%)

(%)

(%)

Cohen’s Kappa values revealed substantial agreement between ELISA-Euroimmun assay and VivaDiag (89.3% agreement; Cohen’s K, 0.73), COVID-19 IgG/IgM Rapid Test Cassette (91.2% agreement; Cohen’s K, 0.78), and PRIMA Professional (89.4% agreement; Cohen’s K, 0.74). Substantial agreement was found also between CLIA-MAGLUMI assay and VivaDiag (89.9% agreement; Cohen’s K, 0.75), COVID-19 IgG/IgM Rapid Test Cassette (90.6% agreement; Cohen’s K, 0.76), and PRIMA Professional (86.2% agreement; Cohen’s K, 0.67).

Since December 2019, when SARS-CoV-2 pneumonia outbreak occurred in Wuhan, many efforts have been made worldwide to quickly develop new diagnostic assays. Molecular detection tests are considered the “gold standard” for diagnosing SARS-CoV-2 infection, though some reports have highlighted a high rate of false negative and/or inconsistent RT-PCR results [5]. Sensitive and specific serologic tests, able to detect neutralizing antibodies, are needed in order to establish seroprevalence and immunity.

Large variability among the positivity rates of anti-SARS-CoV-2 IgG and IgM has been observed: in the first 8/10 days after symptoms onset, assays sensitivities were lower than 50% even if IgG/IgM/IgA positive results were combined [6]. Sensitivities reached 90–95% after 14 days of disease, but patients number included in this group was too low to be statistically significant.

When assays performance has been analyzed according to disease stage, VivaDiag was the best ICT for IgM detection, with a positivity rate increasing from 10 to 95% from early to late stage. PRIMA Professional failed to detect any IgM positive sample in early stage and identified only 24% of positive samples in the late phase. COVID-19 IgG/IgM Rapid Test Cassette displayed a diagnostic performance comparable to that of VivaDiag, except for lower IgM sensitivity in late stage. Compared to CLIA-MAGLUMI, ICT tests displayed an overall better sensitivity, with the exception of PRIMA Professional whose performance was rather limited. With respect to IgG, substantial agreement between ELISA Euroimmun and CLIA-MAGLUMI has been already reported [7]. Among ICTs, PRIMA Professional showed the best performance with a positivity rate increasing from 5% (early phase) to 66% (intermediate) and 95% (late stage). Compared with the automated immunoassays, the ability of ICTs to detect anti-SARS-CoV-2 IgG was equivalent to that of CLIA-MAGLUMI and better than ELISA-Euroimmun, whose IgG positive rates ranged between 0 and 86% at 14 days after symptoms onset.

In the early stage of disease, the positivity rates of IgG/IgM/IgA combined ranged between 2.5 and 10%. These data confirm previous reports which showed the emergence of IgM together with IgG, mostly during the second week of illness [8-10]. IgM was still detected in almost half of blood samples collected in the fourth week after symptoms’ onset. This evidence reaffirms that the diagnostic value of serologic tests in the acute phase of SARS-CoV-2 infection is substantially limited and IgM testing could be questioned.

Some important perspectives need to be considered. First, ICT assays are plagued by subjective interpretation which may be difficult with weak positive bands: between 13 and 22% of IgG results were classified as “weak” in our study. Second, none of the ICT evaluated clearly declared the characteristics of the antigens used and/or the target viral protein(s), and this represents a foremost caveats in assessing the neutralizing activity of the specific anti-SARS-CoV-2 antibodies detected.

This study has some limitations. First, patients were retrospectively identified, and thereby further prospective studies would be needed to validate our preliminary findings. Then, about half of blood samples from COVID-19 confirmed cases had been collected in the early stage, when antibodies are usually absent, thus partially impacting the diagnostic sensitivity. Finally, assays’ specificities were deduced from a limited number of healthy volunteers who may not represent specificities obtained using sera from patients with other diseases.

In conclusion, despite heterogeneous diagnostic performances, some ICTs appear interesting as “first line” serologic tools since their good specificities would help ruling out the infection. The combined use of immunocromatographic and quantitative assays, able to detect neutralizing antibodies, within diagnostic algorithm could be seen as a valuable approach for the serologic diagnosis of COVID-19.