Rapid Serological Tests for Sars-Cov-2: Diagnostic Performance of Four Commercial Assays.

To assess the diagnostic performance of lateral flow immunochromatographic assays (LFA) of four different manufacturers to identify SARS-CoV-2 antibodies (IgM, IgG or total), comparing them with the nucleic acid amplification test (NAAT) or clinical defined (definite or probable SARS-CoV-2 infection respectively).
METHODS: 119 serum samples were randomly selected by convenience and distributed in the groups: (1) Group with SARS-CoV-2 infection [n=82; RT-qPCR positive (definite, n=70), and probable (n=12)]; (2) other diseases [n= 27; other viruses identified (n=8), SARS of other etiologies (n=19)]; (3) healthy control group (n=10). LFA essays of four manufacturers were compared: MedTest Coronavírus (COVID-19) IgG/IgM (MedLevensohn, Brazil); COVID-19 IgG/IgM ECO Test (Ecodiagnóstica, Brazil); Camtech COVID-19 IgM/IgG Rapid Test Kit (Camtech Diagnostics Pte Ltd, Singapore); and one Step COVID-19 Test for total antibodies (Guangzhou Wondfo Biotech Co, China).
RESULTS: The four tests studied showed high diagnostic performance characteristics for the diagnoses of definite or probable SARS-CoV-2 infection. The best measures were for the Wondfo test: sensitivity (86.59%; 95%CI, 77.26-93.11%); specificity (100%; 90.51-100%); DOR (257; 60-1008); LR+ (33.43; 4.82-231.85); LR- (0.13; 0.08 - 0.23); accuracy (90.76%; 84.06- 95.29%); Matthews Correlation coefficient (MCC) 0.82. Although considering only the probable SARS-CoV-2 infection (PCR-) cases, all the kits studied showed limited values.
CONCLUSION: Our data demonstrate the excellent performance of LFA for the diagnoses of definite or probable SARS-CoV-2 infection. There was substantial heterogeneity in sensitivities of IgM and IgG antibodies among the manufacturers. LFA tests cannot replace molecular diagnostics, but should be used as additional screening tool.

Serological tests to detect SARS-CoV-2 antibodies aim to identify previous infection and confirm the presence of current infection.

Lateral flow immunochromatographic assays showed excellent diagnostic performance.

There was substantial heterogeneity in sensitivities of IgM and IgG antibodies between the kits.

Serology tests cannot replace molecular diagnostics, which is considered the gold standard, but should be used only as an additional screening tool.

Introduction

The SARS-CoV-2 pandemic presents important diagnostic challenges [1]. Serology tests to detect the presence of antibodies to SARS-CoV-2 aim to identify previous infection and to confirm the presence of current infection [1, 2]. SARS-CoV-2 serology testing relies on targeted antibodies binding to SARS-CoV-2-specific antigens. This study aimed to assess the diagnostic performance of lateral flow immunochromatographic assays (LFAs) of 4 different manufactures and compare them with the nucleic acid amplification test (NAAT, definite) or the clinical defined test (probable) in individuals with SARS-CoV-2 infection. Tests were selected based on their availability and approval by the National Health Surveillance Agency (ANVISA) in Brazil at the time of the study.

Materials and Methods

The samples consisted of serum specimens sent to the Virology Laboratory, Hospital de Clínicas, Universidade Federal do Parana (HC-UFPR), Brazil. Immunological and RT-qPCR for SARS-CoV-2 assays were performed in the HC-UFPR Virology Laboratory, which is certified by the Health Secretary of Paraná, Brazil. This study was approved under a waiver of informed consent by the HC-UFPR Institutional Review Board, Brazil.

Patients admitted to the HC-UFPR between March 1 and August 7, 2020, were eligible if they had respiratory symptoms suspicious of COVID-19. The performances of the 4 different LFAs were evaluated in serum samples obtained on corresponding dates the respiratory samples were collected for the performance of NAAT. A total of 119 serum samples were randomly selected for convenience and distributed in the following groups (Fig. 1).

Fig. 1

Standards for Reporting of Diagnostic Accuracy Studies (STARD) diagram of participant flow through the validation of LFA for identification of SARS-CoV-2 antibodies in serum samples; reference standard RT-qPCR for SARS-CoV-2 on nasopharyngeal swabs and clinical, epidemiology, and radiologic criteria [3]. aParticipants who tested RT-qPCR negative for SARS-CoV-2 on nasopharyngeal samples. bParticipants were not tested for SARS-CoV-2 by RT-qPCR as the samples were taken before the emergence of the virus in China. LFA, lateral flow immunochromatographic assay.

Group with SARS-CoV-2 (COVID-19, n = 82)

(a) Definite SARS-CoV-2 infection (n = 70): patients who tested positive for SARS-CoV-2 on RT-qPCR from nasopharyngeal samples. Participants admitted to the hospital COVID-19 unit or intensive care unit, n = 60 (86%); outpatients (n = 10; 14%); male, 38 (54%); median (IQR) age, 50 (38, 58.5) years; time after symptoms onset, 17 (12, 23) days. Two participants (2.9%) were asymptomatic.

(b) Probable SARS-CoV-2 infection (n = 12): patients who tested RT-qPCR negative for SARS-CoV-2 on nasopharyngeal samples but fulfilled the World Health Organization (WHO) clinical diagnostic case definitions for SARS-CoV-2 [3] (see online suppl. Table 1; see www.karger.com/doi/10.1159/000516776 for all online suppl. material). All participants were admitted in the hospital COVID-19 unit or intensive care unit; male, 6 (50%); median (IQR) age, 61.5 (47.5, 74.5) years; time after symptoms onset, 11 (7.5, 19) days (definite vs. probable, p = 0.046 and 0.118, respectively).

Group with Other Diseases (n = 27)

RT-qPCR for SARS-CoV-2 on Nasopharyngeal Samples Negative

(a) Other viruses were identified (n = 8) by RT-qPCR on a nasopharyngeal swab tested against a respiratory virus panel: rhinovirus (n = 6) and coronavirus 229e/NL63 (1 case). One case was positive for Epstein-Barr virus and cytomegalovirus identified in blood.

(b) Severe acute respiratory syndromes: RT-qPCR for SARS-CoV-2 on respiratory samples was either negative and did not fulfill the WHO case definitions for SARS-CoV-2 [3] or other etiologies were identified (n = 19). Median (IQR) age was 54 (41, 76) years; male, 13 (48%) (definite vs. probable vs. other diseases, p = 0.078 and 0.851, respectively); time after symptoms onset, 4 (3, 8.5) days.

Group without Disease (Healthy Control Group, n = 10)

Ten serum samples from blood donors were collected in 2015 (HIV, HCV, HBV, HTLV I/II, syphilis, and Chagas disease negative). This group was not tested for SARS-CoV-2 by RT-qPCR as the samples were taken before the emergence of the virus in China [3].

The research was conducted ethically in accordance with the World Medical Association Declaration of Helsinki. This study was approved under a waiver of informed consent by the HC-UFPR Institutional Review Board, Brazil.

Lateral Flow Immunochromatographic Assays

The kits use capture reaction to detect SARS-CoV-2 IgM/IgG or total antibody in the samples, a 1-step method with results obtained within 15 min, using serum or plasma samples.

IgG, IgM, and Total Antibodies for SARS-CoV-2

The MedTest Coronavirus (COVID-19) IgG/IgM (MedLevensohn, Brazil), the COVID-19 IgG/IgM ECO Test (Ecodiagnóstica, Brazil), and the Camtech COVID-19 IgM/IgG Rapid Test Kit (Camtech Diagnostics Pte Ltd, Singapore) detect IgM and IgG. Samples were tested in parallel in the 4 assays. The tests were performed at room temperature according to the manufacturer's instructions. For all tests, the recommended sample volume of 10 μL serum was added to the specimen well on the individual test cassettes followed by the supplied buffer. The result was visually read after 10 min, by 2 researchers; in case of doubt, a third researcher checked it. For combination of the IgM and the IgG kit, the test card or cassette had 2 test lines (M and G lines) and a quality control line (C line). The M line was fixed with a monoclonal anti-human IgM antibody for detecting SARS-CoV-2 antibody, the G line was fixed with a reagent for detecting SARS-CoV-2 antibody, and the C line was fixed with a quality control antibody. Any visible band for IgG, IgM, or unspecified immunoglobulin was indicative for a positive result. The test card of the Wondfo kit has only 1 test line (T line) and a quality control line (C line). This kit does not differentiate IgM or IgG, and thus results were interpreted as positive or negative for SARS-CoV-2 antibody.

RT-qPCR for SARS-CoV-2

The RT-qPCR for SARS-CoV-2 was composed of dual positive results from a single NAAT targeting 2 different SARS-CoV-2 genes. Samples were collected with a rayon swab and transported immediately to the virology laboratory in a viral transport medium. Samples were taken from the oral cavity and subsequently from the nasal cavity using a nasopharyngeal rayon swab. We performed RT-qPCR using the XGEN-Master COVID-19 (XGEN) for qualitative detection of nucleic acid in RT-qPCR format-reverse transcription, followed by amplification of a conserved region of the ORF1ab and N genes for SARS-CoV-2 [4, 5], using specific primers and a fluorescence-labeled probe in respiratory samples. Specificity: 100% for SARS-CoV-2 (ORF1ab gene), 10 copies/reaction, with probability ≥95%. Sensitivity − SARS-CoV-2 (ORF1ab gene): 10 copies/reaction, with probability ≥95%. Sensitivity − SARS-CoV-2 (N gene): 50 copies/reaction, with probability ≥95%.

Statistical Analyses

The results were presented as the median (IQR) and number (n) and percentage, as appropriate. Categorical variables were compared between groups using the Fisher's exact test, and continuous variables were compared using the Mann-Whitney or Kruskal-Wallis test for nonparametric data, as appropriate. We performed the comparison of concordance and discordance proportions of the different kits with the McNemar test for paired nominal data. Results were considered significant at the 5% alpha level.

Clinical Performance Characteristics of the Tests

We evaluated, for each kit, the clinical performance of the LFA for SARS-CoV-2 IgM/IgG or total antibody (index test) in predicting the SARS-CoV-2 infection. The RT-qPCR for SARS-CoV-2 was the reference method. We analyzed separately the diagnostic performance for the detection of IgM and IgG antibodies in each test. For the calculation of SARS-CoV-2 IgG clinical performance, we included samples from patients who presented symptoms onset ≥8 days. If an IgG result was positive before the 8th day, we considered it true positive (n = 75). For IgM, we included all the samples collected.

The following clinical performance measures were calculated: sensitivity, specificity, accuracy (efficiency), positive and negative predictive values (PPV and NPV), Youden index [6], and positive and negative clinical utility index (CUI+ and CUI−). The CUI values were classified as follows: excellent, ≥ 0.81; good, ≥ 0.64; fair, ≥ 0.49; poor, ≤ 0.49; and very poor, ≤ 0.36 [7, 8]. We calculated the positive and negative likelihood ratio (LR+ and LR−) and diagnostic odds ratio (DOR), in which an LR+ value ≥10.0 indicates that a positive test almost confirmed the disease, a value of ∼6.0 indicates that the disease was present, and a value of ∼1.0 indicates that the test was not able to show whether the disease was present or not. An LR+ value ≤0.1 indicates that the disease was practically absent [9, 10]. The Matthews correlation coefficient (MCC) is a value between −1 and +1.

As the Wondfo test detects total antibodies, in order to compare its results with other tests, we evaluated the clinical performance results of all kits considering any positive results (IgM or IgG), and in a subsequent analysis, we evaluated them separately (IgM and IgG). Positive rates and levels of agreement between the kits were assessed using Cohen's kappa coefficients of agreement, which may be interpreted as follows: values ≤0 as indicating no agreement (i.e., purely random), 0.01–0.20 as none to slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect agreement [11].

Results

The groups with definite and probable SARS-CoV-2 infection were comparable for time of start of symptoms, and for age, they were comparable with the group with other diseases, although different for age on PCR− and PCR+ groups.

Clinical Performance of LFAs

Overall, the analysis of the tests showed a good sensitivity; 1-Step Wondfo and MedTest presented the best results. In addition to the high sensitivity, they were the most accurate tests (Table 1). MedTest and Wondfo presented higher sensitivity, specificity, and accuracy, with less extended 95% confidence intervals (Fig. 2a–d).

Table 1

Clinical performance characteristics of LFA in serum to detect SARS-CoV-2 antibodies

IgM and/or IgG	MedTest		EcoTest		Camtech		Wondfo
	value	95% CI	value	95% CI	value	95% CI	value	95% CI
Group PCR+/PCR–
TP	69		67		76		71
Sensitivity, %	84.15	74.42; 91.28	81.71	71.63; 89.38	92.68	84.75; 97.27	86.59	77.26; 93.11
Specificity, %	100.00	90.51; 100.00	78.38	61.79; 90.17	78.38	61.79; 90.17	100.00	90.51; 100.00
LR (+)	32.50	4.68; 225.50	3.78	2.03; 7.04	4.29	2.31; 7.94	33.43	4.82; 231.85
LR (−)	0.16	0.10; 0.26	0.23	0.14; 0.38	0.09	0.04; 0.21	0.13	0.08; 0.23
DOR	203.00	46.80; 867.00	16.43	14.5; 18.53	47.67	57.75; 37.81	257.15	60.25; 1,008
Youden's index	0.84	0.65; 0.91	0.60	0.33; 0.80	0.71	0.47; 0.87	0.87	0.68; 0.93
CUI+	0.84	0.77; 0.92	0.73	0.64; 0.83	0.84	0.77; 0.91	0.87	0.80; 0.94
CUI–	0.74	0.66; 0.82	0.52	0.40; 0.63	0.65	0.54; 0.76	0.77	0.69; 0.85
PPV, %	100.00	−	89.33	81.81; 93.98	90.48	83.68; 94.62	100.00	−
NPV, %	74.00	63.35; 82.41	65.91	54.28; 75.90	82.86	68.71; 91.41	77.08	66.00; 85.36
Accuracy, %	89.08	82.04; 94.05	80.67	72.42; 87.34	88.24	81.05; 93.42	90.76	84.06; 95.29
MCC	0.79	−	0.57	−	0.72	−	0.82	−
Group PCR+
TP	68		65		69		69
Sensitivity, %	97.14	90.06; 99.65	92.86	84.11; 97.64	98.57	92.30; 99.96	98.57	92.30; 99.96
Specificity, %	100.00	90.51; 100.00	78.38	61.79; 90.17	78.38	61.79; 90.17	100.00	90.51; 100.00
LR (+)	37.37	5.40; 258.87	4.29	2.32; 7.96	4.56	2.47; 8.43	37.92	5.48; 262.56
LR (−)	0.03	0.01; 0.11	0.09	0.04; 0.22	0.02	0.00; 0.13	0.01	0.00; 0.10
DOR	1,245	540; 2,353	47.66	58.00; 36.18	228	0.00; 64.85	3,792	0.00; 2,625
Youden's index	0.97	0.81; 1.00	0.71	0.46; 0.88	0.77	0.54; 0.90	0.99	0.83; 1.00
CUI+	0.97	0.94; 1.00	0.83	0.75; 0.91	0.88	0.82; 0.95	0.99	0.96; 1.00
CUI–	0.95	0.91; 0.99	0.67	0.56; 0.78	0.75	0.66; 0.86	0.97	0.94; 1.00
PPV, %	100.00	−	89.04	81.43; 93.77	89.61	82.35; 94.10	100.00	−
NPV, %	94.87	82.52; 98.64	85.29	71.02; 93.21	96.67	80.44; 99.51	97.37	84.09; 99.62
Accuracy, %	98.13	93.41; 99.77	87.85	80.12; 93.37	91.59	84.63; 96.08	99.07	94.90; 99.98
MCC	0.96	−	0.73	−	0.81	−	0.98	−
Group PCR–
TP	01		02		07		02
Sensitivity, %	8.33	0.21; 38.48	16.67	2.09; 48.41	58.33	27.67; 84.83	16.67	2.09; 48.41
Specificity, %	100.00	90.51; 100.00	78.38	61.79; 90.17	78.38	61.79; 90.17	100.00	90.51; 100.00
LR (+)	5.57	0.55; 56.78	0.77	0.19; 3.14	2.70	1.24; 5.87	8.36	0.95; 73.88
LR (−)	0.92	0.77; 1.09	1.06	0.78; 1.44	0.53	0.27; 1.06	0.83	0.65; 1.07
DOR	6.05	0.71; 52.02	0.73	0.24; 2.18	5.09	4.59; 5.54	10.07	1.46; 69.05
Youden's index	0.08	−0.09; 0.39	−0.05	−0.36; 0.39	0.37	−0.11; 0.75	0.17	−0.07; 0.48
CUI+	0.08	0.00; 0.55	0.03	0.00; 0.31	0.27	0.00; 0.60	0.17	0.00; 0.61
CUI–	0.77	0.69; 0.84	0.58	0.47; 0.70	0.67	0.56; 0.78	0.79	0.71; 0.87
PPV, %	100.00	−	20.00	5.77; 50.49	46.67	28.67; 65.57	100.00	−
NPV, %	77.08	73.93; 79.96	74.36	68.14; 79.72	85.29	74.41; 92.04	78.72	74.18; 82.65
Accuracy, %	77.55	63.38; 88.23	63.27	48.29; 76.58	73.47	58.92; 85.05	79.59	65.66; 89.76
MCC	0.25	−	−0.05	−	0.34	−	0.36	−

LFA, lateral flow immunochromatographic assay; TP, true positive; PPV, positive predictive value; NPV, negative predictive value; LR, likelihood ratio; DOR, diagnostic odds ratio (the higher the DOR value, the better the test); CUI, clinical utility index; MCC, Matthews correlation coefficient.

Fig. 2

Comparison of LFA diagnostic performance for IgM, IgG, or total antibodies to diagnose SARS-CoV-2 infection. MedTest (circle), IgM (black circle), IgG (empty circle), and IgM or IgG (orange circle); Eco Test (diamond), IgM (black diamond), IgG (empty diamond), and IgM or IgG (orange diamond); Camtech (square), IgM (black square), IgG (empty square), and IgM or IgG (orange square); Wondfo IgM or IgG (orange triangle). Sensitivity (a); specificity (b); accuracy (c); and Youden's index (d). LFA, lateral flow immunochromatographic assay.

Considering IgG, IgM, or total antibodies for the diagnoses of definite or probable SARS-CoV-2 infection, the 4 tests studied showed high diagnostic performance characteristics. The higher sensitivity was for Camtech; accuracy was similar for MedTest, Camtech, and Wondfo. Diagnostic specificity was 100% for the MedTest and Wondfo. LR+ was >30 for the MedTest and Wondfo, indicating a positive test, almost confirming the SARS-CoV-2 infection although it was limited for the Eco Test and Camtech. The highest DOR was seen in the Wondfo test.

For the PCR+ cases, the sensitivity and accuracy were higher for all the kits studied. Although for the probable SARS-CoV-2 infection (RT-qPCR negative) all the kits showed a limited value for total antibody, IgM, and IgG, a high specificity was seen for IgM and IgG.

The Wondfo and MedTest showed the best clinical diagnostic performance characteristics with a high LR (+), which indicates that a positive test almost confirmed the disease, and a high Youden's index, DOR, accuracy, and MCC (Table 1). Concerning the detection of IgM for the diagnoses of definite or probable SARS-CoV-2 infection, MedTest and Camtech displayed higher sensitivity and accuracy. All the kits showed high diagnostic specificity for IgM detection (Table 2). Concerning the detection of IgG for SARS-CoV-2, the sensitivity was low, although it was specific; the MedTest and Camtech showed similar diagnostic performance characteristics (Table 3).

Table 2

Clinical performance characteristics of LFA in serum to detect SARS-CoV-2 IgM antibodies

IgM	MedTest		EcoTest		Camtech
	value	95% CI	value	95% CI	value	95% CI
Group PCR+/PCR–
TP	68		48		72
Sensitivity, %	82.93	73.02; 90.34	58.54	47.12; 69.32	87.80	78.71; 93.99
Specificity, %	100.00	90.51; 100.00	83.78	67.99; 93.81	81.08	64.84; 92.04
LR (+)	32.04	4.62; 222.32	3.61	1.70; 7.68	4.64	2.37; 9.09
LR (−)	0.17	0.11; 0.28	0.49	0.37; 0.66	0.15	0.08; 0.27
DOR	188.47	42.00; 794.00	7.37	4.60; 11.64	30.93	29.63; 33.67
Youden's index	0.83	0.64; 0.90	0.42	0.15; 0.63	0.69	0.44; 0.86
CUI+	0.83	0.75; 0.91	0.52	0.39; 0.65	0.80	0.72; 0.88
CUI–	0.73	0.64; 0.81	0.40	0.29; 0.51	0.61	0.50; 0.72
PPV, %	100.00	−	88.89	79.00; 94.45	91.14	84.01; 95.27
NPV, %	72.55	62.12; 80.98	47.69	40.47; 55.02	75.00	62.18; 84.55
Accuracy, %	88.24	81.05; 93.42	66.39	57.15; 74.78	85.71	78.12; 91.45
MCC	0.78	−	0.39	−	0.68	−
Group PCR+
TP	67		46		66
Sensitivity, %	95.71	87.98; 99.11	65.71	53.40; 76.65	94.29	86.01; 98.42
Specificity, %	100.00	90.51; 100.00	83.78	67.99; 93.81	81.08	64.84; 92.04
LR+	36.83	5.32; 255.17	4.05	1.91; 8.59	4.98	2.55; 9.73
LR–	0.04	0.01; 0.13	0.41	0.29; 0.58	0.07	0.03; 0.18
DOR	921	532; 1,962	9.87	6.59; 14.81	71.14	85.00; 54.05
Youden's index	0.96	0.79; 0.99	0.50	0.21; 0.71	0.75	0.51; 0.91
CUI+	0.96	0.92; 1.00	0.58	0.45; 0.71	0.85	0.78; 0.93
CUI–	0.93	0.87; 0.98	0.47	0.36; 0.58	0.72	0.61; 0.82
PPV, %	100.00	−	88.46	78.33; 94.21	90.41	82.84; 94.85
NPV, %	92.50	80.30; 97.39	56.36	47.55; 64.79	88.24	74.09; 95.16
Accuracy, %	97.20	92.02; 99.42	71.96	62.45; 80.22	89.72	82.35; 94.76
MCC	0.94	−	0.47	−	0.77	−
Group PCR–
TP	01		02		06
Sensitivity, %	8.33	0.21; 38.48	16.67	2.09; 48.41	50.00	21.09; 78.91
Specificity, %	100.00	90.51; 100.00	83.78	67.99; 93.81	81.08	64.84; 92.04
LR+	5.57	0.55; 56.78	1.03	0.24; 4.43	2.64	1.10; 6.34
LR–	0.92	0.77; 1.09	0.99	0.74; 1.33	0.62	0.34; 1.11
DOR	6.05	0.714; 52.09	1.04	0.32; 3.33	4.26	3.33; 5.71
Youden's index	0.08	−0.09; 0.39	0.01	−0.30; 0.42	0.31	−0.14; 0.71
CUI+	0.08	0.00; 0.55	0.04	0.00; 0.35	0.23	0.0; 0.57
CUI–	0.77	0.69; 0.85	0.63	0.53; 0.74	0.68	0.57; 0.78
PPV, %	100.00	−	25.00	7.17; 58.98	46.15	26.33; 67.27
NPV, %	77.08	73.93; 79.96	75.61	69.88; 80.56	83.33	73.55; 89.99
Accuracy, %	77.55	63.38; 88.23	67.35	52.46; 80.05	73.47	58.92; 85.05
MCC	0.25	−	0.01	−	0.30	−

LFA, lateral flow immunochromatographic assay; TP, true positive; PPV, positive predictive value; NPV, negative predictive value; LR, likelihood ratio; DOR, diagnostic odds ratio (the higher the DOR value, the better the test); CUI, clinical utility index; MCC, Matthews correlation coefficient.

Table 3

Clinical performance characteristics of LFA in serum to detect SARS-CoV-2 IgG antibodies

IgG	MedTest		EcoTest		Camtech
	value	95% CI	value	95% CI	value	95% CI
Group PCR+/PCR–
TP	59		63		59
Sensitivity, %	78.67	67.68; 87.29	84.00	73.72; 91.45	78.67	67.68; 87.29
Specificity, %	100.00	90.51; 100.00	89.19	74.58; 96.97	97.30	85.84; 99.93
LR+	30.39	4.37; 211.13	7.77	3.06; 19.71	29.11	4.20; 201.92
LR–	0.21	0.14; 0.33	0.18	0.11; 0.30	0.22	0.14; 0.34
DOR	144	31.21; 639	43.17	27.82; 65.70	132.32	30; 594
Youden's index	0.79	0.58; 0.87	0.73	0.48; 0.88	0.76	0.54; 0.87
CUI+	0.79	0.70; 0.88	0.79	0.70; 0.88	0.77	0.68; 0.87
CUI–	0.70	0.61; 0.79	0.65	0.55; 0.76	0.67	0.58; 0.77
PPV, %	100.00	−	94.03	86.13; 97.56	98.33	89.48; 99.76
NPV, %	69.81	59.96; 78.12	73.33	61.80; 82.38	69.23	59.22; 77.71
Accuracy, %	85.71	77.84; 91.61	85.71	77.84; 91.61	84.82	76.81; 90.90
MCC	0.74	−	0.70	−	0.72	−
Group PCR+
TP	58		62		57
Sensitivity, %	87.88	77.51; 94.62	93.94	85.20; 98.32	86.36	75.69; 93.57
Specificity, %	100.00	90.51; 100.00	89.19	74.58; 96.97	97.30	85.84; 99.93
LR+	33.84	4.88; 234.75	8.69	3.44; 21.97	31.95	4.61; 221.41
LR–	0.12	0.06; 0.23	0.07	0.03; 0.18	0.14	0.08; 0.26
DOR	282	81.33; 1,020.00	124	115; 122	228	7.62; 850
Youden's index	0.88	0.68; 0.95	0.83	0.60; 0.95	0.84	0.62; 0.94
CUI+	0.88	0.81; 0.95	0.88	0.81; 0.95	0.85	0.77; 0.93
CUI–	0.82	0.75; 0.90	0.80	0.71; 0.88	0.78	0.70; 0.86
PPV, %	100.00	−	93.94	85.98; 97.51	98.28	89.16; 99.75
NPV, %	82.22	70.72; 89.85	89.19	76.02; 95.55	80.00	68.50; 88.04
Accuracy, %	92.23	85.27; 96.59	92.23	85.27; 96.59	90.29	82.87; 95.25
MCC	0.85	−	0.83	−	0.81	−
Group PCR–
TP	01		01		02
Sensitivity, %	11.11	0.28; 48.25	11.11	0.28; 48.25	22.22	2.81; 60.01
Specificity, %	100.00	90.51; 100.00	89.19	74.58; 96.97	97.30	85.84; 99.93
LR+	7.09	0.71; 71.11	1.03	0.13; 8.12	8.22	0.83; 80.97
LR–	0.89	0.71; 1.12	1.00	0.77; 1.29	0.80	0.56; 1.14
DOR	7.97	1.00; 63.49	1.03	0.17; 6.30	10.28	1.48; 71.03
Youden's index	0.11	−0.09; 0.48	0.003	−0.25; 0.45	0.20	−0.11; 0.60
CUI+	0.11	0.00; 0.65	0.02	0.00; 0.35	0.15	0.00; 0.64
CUI–	0.82	0.75; 0.90	0.72	0.62; 0.81	0.82	0.74; 0.89
PPV, %	100.00	−	20.00	3.07; 66.38	66.67	16.88; 95.17
NPV, %	82.22	78.59; 85.35	80.49	76.14; 84.21	83.72	78.32; 87.98
Accuracy, %	82.61	68.58; 92.18	73.91	58.87; 85.73	82.61	68.58; 92.18
MCC	0.30	−	0.003	−	0.31	−

LFA, lateral flow immunochromatographic assay; TP, true positive; PPV, positive predictive value; NPV, negative predictive value; LR, likelihood ratio; DOR, diagnostic odds ratio (the higher the DOR value, the better the test); CUI, clinical utility index; MCC, Matthews correlation coefficient.

Two subjects (2.9%) were asymptomatic and were positive for SARS-CoV-2 RT-qPCR from nasopharyngeal samples; Wondfo was positive in both patients; however, only one of them tested positive for IgM in the MedTest and Camtech. IgG was not positive in all assays performed. The other subject tested positive for both IgM and IgG in all assays, but the Camtech test was positive only for IgG.

Analytical Specificity (Cross-Reaction)

For the Wondfo and MedTest IgM and IgG and the Camtech IgG, no cross-reaction was identified, and the analytical specificity was 100%. However, for the Eco Test, the analytical specificity for IgG or IgM was 80% each. For IgM, 1 sample cross-reacted with rhinovirus and 2 samples with Tb. Concerning IgG, 1 sample cross-reacted with rhinovirus and 2 samples with sepsis. For Camtech, there was a cross-reaction of IgM with 4 samples (2 samples with rhinovirus, 1 community-acquired pneumonia, and 1 pleural Tb). The analytical specificity was 73.30%.

Analytical Interference

Analytical interference is defined as the effect of a substance present in the sample that alters the correct value of the result. There was hemolysis in 1 sample in the group with definite SARS-CoV-2 infection. This sample tested positive for IgM, IgG, and total antibodies by all the 4 kits in the group with RT-qPCR positive. The agreement between LFA kits is shown in online suppl. Table 2.

Discussion

In this study, we report the performance characteristics of the LFA for IgG, IgM, and total antibody anti-SARS-CoV-2 from 3 different manufacturers for IgG and IgM and one for total antibody anti-SARS-CoV-2. We used sera from hospitalized adult patients with definite (RT-qPCR proved) and probable (clinical, epidemiologic, and radiologic criteria, with RT-qPCR negative) SARS-CoV-2 infection. The diagnosis of SARS-CoV-2 infection involves collecting the correct specimen at the right time. IgM antibodies usually appear after the 7th day of infection, indicating acute infection. However, IgG antibodies appear from the 14th day on. However, the usefulness of monitoring IgM and discriminating them from IgG is largely debated [12].

Overall, the LFA to identify SARS-CoV-2 antibodies (IgM, IgG, or total) in the definite and probable cases showed high sensitivity, specificity, and PPV. This indicates that a positive result is often seen in those with SARS-CoV-2 infection, and a negative result is often seen in those without the infection. The very high PPV that indicates a false positive is rare, which suggests this can be a potential confirmatory test. A high CUI+ suggested LFAs are excellent for case finding and a CUI− is good for screening. The overall value of this single test for combined screening and case finding is good. The clinical performance characteristics presented higher values considering only the definite SARS-CoV-2 cases determined by positive RT-qPCR. However, for probable cases, the clinical performance characteristics showed limited values. Overall, the manufacturers' DOR and MCC of IgM were higher than IgG, and the diagnostic sensitivity and specificity were similar for IgG and IgM.

In the present study, the Wondfo test presented the best clinical performance characteristic, followed by the MedTest. Our findings are in accordance with previous studies, in which the Wondfo test was one of the 2 best-performing LFAs of the 10 evaluated, with pooled estimates of sensitivity (84.8%; 95% confidence interval [CI] = 81.4–87.8%) and specificity (99.0%; 95% CI = 97.8–99.7%) [13, 14]. However, the Wondfo test has the advantage of not differentiating between IgM and IgG. For IgM and IgG, the best results of clinical performance characteristic were with the MedTest. The Youden's index, accuracy, and MCC of IgG were similar for the 3 manufacturers.

Test performance characteristics as provided by manufacturers were similar with those observed in our study [15]. In the present study, the median time after the symptoms appeared was 11 and 17 days in the groups with definite and probable SARS-CoV-2. We observed that the positivity of immunological tests for SARS-CoV-2 diagnosis is time dependent. Patients with at least 8 days of symptoms presented a higher sensitivity [16, 17]. Antibody-mediated immunity in SARS-CoV-2-specific IgM and IgG is detectable in the serum between 7 and 14 days after the onset of the symptoms, respectively. The SARS-CoV-2 virus RNA peak occurs at 3–5 days after exposure; virus RNA inversely correlated with neutralizing antibody titers. In the acute phase of the disease, nucleic acid detection of SARS-CoV-2 in respiratory samples was greater for antibody detection in the diagnosis of CO­VID-19 [16, 18, 19, 20, 21, 22]. After that period, tests that detect the presence of specific antibodies are recommended [12, 23]. Antibody tests could play a useful role in the detection of previous SARS-CoV-2 infection if applied 15 or more days after the onset of symptoms. The persistence of antibody rises is currently unknown, as there are very little data beyond 35 days after symptoms onset [1] as well as about the extension of protection of neutralizing antibodies against subsequent infection with the virus [24]. The sensitivity of LFA was 11.1% from the 1st to the 7th day after the onset of the symptoms, 92.9% from the 8th to the 14th days, and 96.8% from day 15 after the onset of the symptoms [25].

Our study will add to the previous ones [1, 13, 25, 26, 27] as SARS-CoV-2 infection cases were diagnosed based on RT-qPCR and clinical radiologic criteria, that is, definite and probable SARS-CoV-2 cases, respectively [3]; 15% of the samples included in the SARS-CoV-2 group were probable infection. Although these cases fulfill the clinical and radiological criteria for SARS-CoV-2 infection, some cases could be miss enrolled.

The main limitation of this study is the small number of samples and the lack of blinding of the index test and reference standard; the majority patients were admitted in the hospital, which represents clinically more severe patients. Only 14% were outpatients, and of these, only 2 were asymptomatic. Therefore, it is unclear if the results could be expanded to asymptomatic or oligosymptomatic persons or even used on serological community surveys. Only adults were included; reports on the dynamics and detection of SARS-CoV-2 antibodies in children are lacking and require urgent attention.

Conclusion

Our data demonstrate excellent performance of the antibody assays studied. We observed substantial heterogeneity in sensitivities of IgM and IgG antibodies between manufacturers. The high specificity of LFAs may contribute to rapidly confirm the presence of SARS-CoV-2 infection and accelerate decision-making in emergency rooms and routing to appropriate hospital wards. Nonetheless, these LFA tests cannot replace molecular diagnostics in acute-care settings but should only be used as an additional screening tool when the improvement of hospital logistics is expected, and their limitations are carefully considered.

Statement of Ethics

This study was approved under a waiver of informed consent by the HC-UFPR Institutional Review Board, Brazil (no. 30578620.70000.0008).

Conflictof Interest Statement

The authors have no conflicts of interest to declare.

Funding Sources

The authors did not receive any funding.

Author Contributions

Conception of the study − S.M. de Almeida, M. B. Nogueira, and S.M. Raboni; study design − S.M. de Almeida, M. B. Nogueira, and S.M. Raboni; data acquisition − S. M. de Almeida, R. N. Spalanzani, M.B. Nogueira, B. Sanada, B.M. Cavalli, I. Rotta, G. R.A. Takahashi, L.A. Pereira, F.B. Moreira, C.L.T. Dino, M.E. Graf, B.M.M. de Almeida, N. R. Domino, G. Genelhoud, L.B. Bueno, and S.M. Raboni; data analysis − S.M. de Almeida, R.N. Spalanzani, M.B. Nogueira, B. Sanada, B.M. Cavalli, I. Rotta, G.R.A. Takahashi, L. A. Pereira, F.B. Moreira, C.L.T. Dino, M.E. Graf, B. M.M. de Almeida, N.R. Domino, G. Genelhoud, L. B. Bueno, and S. Raboni; data interpretation − S.M. de Almeida, M. B. Nogueira, and S.M. Raboni; drafting of the manuscript − S.M. de Almeida, M.B. Nogueira, and S.M. Raboni; final approval of the manuscript − S.M. de Almeida, R.N. Spalanzani, M.B. Nogueira, B. Sanada, B.M. Cavalli, I. Rotta, G.R.A. Takahashi, L. A. Pereira, F.B. Moreira, C.L.T. Dino, M.E. Graf, B. M.M. de Almeida, N.R. Domino, G. Genelhoud, L. B. Bueno, and S. Raboni.

Supplementary data

Click here for additional data file.