Suitability of two rapid lateral flow immunochromatographic assays for predicting SARS-CoV-2 neutralizing activity of sera.

Assessment of commercial severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) immunoassays for their capacity to provide reliable information on sera neutralizing activity is an emerging need. We evaluated the performance of two commercially available lateral flow immunochromatographic assays (LFIC; Wondfo SARS-CoV-2 Antibody test and the INNOVITA 2019-nCoV Ab test) in comparison with a SARS-CoV-2 neutralization pseudotyped assay for coronavirus disease 2019 (COVID-19) diagnosis in hospitalized patients and investigate whether the intensity of the test band in LFIC associates with neutralizing antibody (NtAb) titers. Ninety sera were included from 51 patients with moderate to severe COVID-19. A green fluorescent protein (GFP) reporter-based pseudotyped neutralization assay (vesicular stomatitis virus coated with SARS-CoV-2 spike protein) was used. Test line intensity was scored using a 4-level scale (0 to 3+). The overall sensitivity of LFIC assays was 91.1% for the Wondfo SARS-CoV-2 Antibody test, 72.2% for the INNOVITA 2019-nCoV IgG, 85.6% for the INNOVITA 2019-nCoV IgM, and 92.2% for the NtAb assay. Sensitivity increased for all assays in sera collected beyond day 14 after symptoms onset (93.9%, 79.6%, 93.9%, and 93.9%, respectively). Reactivities equal to or more intense than the positive control line (≥2+) in the Wondfo assay had a negative predictive value of 100% and a positive predictive value of 96.4% for high NtAb50 titers (≥1/160). Our findings support the use of LFIC assays evaluated herein, particularly the Wondfo test, for COVID-19 diagnosis. We also find evidence that these rapid immunoassays can be used to predict high SARS-CoV-2-S NtAb50 titers.

INTRODUCTION

Serological testing is increasingly recognized as a useful tool for control of the coronavirus disease 2019 (COVID‐19) pandemic; beyond complementing reverse‐transcription polymerase chain reaction (RT‐PCR) assays for disease diagnosis in symptomatic patients, detection of specific antibodies allows for estimating SARS‐CoV‐2 infection incidence and virus spread in a given population, inferring protection against reinfection, evaluating vaccine efficacy, and selecting appropriate plasma specimens from convalescent COVID‐19 patients for passive transfer therapies. , , Numerous SARS‐CoV‐2 serological tests have been commercialized, among which lateral flow immunochromatographic assays (LFIC) are particularly appealing because of their rapid turnaround times, simplicity of use, and suitability for point of care testing.

SARS‐CoV‐2 neutralizing antibodies (NtAb) are presumed to play a major protective role against SARS‐CoV‐2 infection. , , Unfortunately, virus neutralization assays, whether using wild‐type SARS‐CoV‐2, engineered SARS‐CoV‐2 pseudotypes, or chimeric viruses, are unsuited for routine testing, thus creating a need to assess commercial SARS‐CoV‐2 immunoassays for their capacity to provide reliable information on sera neutralizing activity. Several studies have evaluated the performance of LFIC in comparison with NtAb assays in subjects with past or ongoing SARS‐CoV‐2 infection. , , , Nevertheless, to the best of our knowledge, only one has attempted to quantitatively correlate results yielded by the two assay types by analyzing the strength of test line reactivity in LFIC devices and NtAb50 titers.

Here, we sought to evaluate the performance of two commercially available LFIC, widely used in our country, compared with a SARS‐CoV‐2 neutralization pseudotype assay for COVID‐19 diagnosis in hospitalized patients, and determine whether the intensity of the test band in LFIC was associated with the levels of NtAb, recognizing the SARS‐CoV‐2 Spike (S) protein.

MATERIALS AND METHODS

Serum specimens and patients

The current study included 90 sera from 51 patients with moderate to severe laboratory‐confirmed (RT‐PCR) COVID‐19 RT‐PCR admitted to Hospital Clínico Universitario of Valencia between March 5 and April 30, 2020. Sera were grouped according to the timing of collection after symptoms onset: 41 were obtained within < 15 days (median, 11 days; range, 5–14 days), and 49 later on (≥15 days, at a median of 23 days; range, 15‐41 days). In addition, a total of 20 prepandemic sera from healthy individuals were collected within 2019, of which 10 belonged to patients with prior endemic coronavirus infections (HCoV‐229E, n = 8; HCoV NL63, n = 1; HCoVHKU, n = 1) and were included as controls. Sera had been cryopreserved at ‐20°C and were thawed for the analyses described below. This study was approved by the Research Ethics Committee of Hospital Clínico Universitario INCLIVA (March, 2020).

SARS‐CoV‐2 neutralizing antibody assay

A green fluorescent protein (GFP) reporter‐based pseudotyped neutralization assay with a nonreplicative vesicular stomatitis virus (VSV) backbone coated with SARS‐CoV‐2 spike (S) protein was used for neutralization assays on Vero cells, using heat‐inactivated sera and a viral input of 1250 focus‐forming units, as previously described. Sera that did not reduce viral replication by 50% at 1/20 dilution were considered non‐neutralizing and were arbitrarily assigned a value of 1/10. The antibody dilution resulting in 50% virus neutralization (NtAb50) was calculated using the drc package (version 3.0‐1) in R via a two‐parameter log‐logistic regression model (LL.2 model).

Commercial SARS‐CoV‐2 IgG LFIC immunoassays

Two LFIC were evaluated: SARS‐COV‐2 Antibody test from Guangzhou Wondfo Biotech Co., Ltd. (China), which detects SARS‐CoV‐2 antibodies (IgG and IgM) in a single test band, and INNOVITA 2019‐nCoV Ab Test (Beijing Innovita Biological Technology, China), which detects IgG and IgM separately. The antigenic specificity of antibodies detected by these assays was not disclosed (to our knowledge). Both assays were performed according to the protocol provided by the respective manufacturer. Test line intensity scoring was done using a 4‐level scale (Figure 1), in which 0 corresponded to a negative result (absence of a test line), 1+   represented a weak positive result (intensity of test band lower than control band), 2+ a positive result (intensity of test band equal to control line), and 3+ a strong positive result (intensity of test band greater than control line). Four readers independently scored each test and the average of the scores was recorded as the final LFIC result. Ten sera exhibiting different strengths of reactivities (from 1+ to 3+) were tested in two different batches of each LFIC assay, these displaying identical results.

Figure 1

Test line intensity was scored using a 4‐level scale. From left to right: 0, negative result; 1+, weak positive result (intensity of test band lower than the control band); 2+, positive result (intensity of test band equal to the control line); 3+, strong positive result (intensity of test band greater than the control line)

Definition

Here, NtAb titers ≥ 1/160 were deemed as high, in line with the minimum NtAb50 titer of plasma from COVID‐19 convalescent individuals recommended by FDA for therapeutic use.

Statistical methods

Test performances were evaluated by the sensitivity with the associated 95% confidence interval (CI). Cohen's κ statistic was used to evaluate the qualitative agreement between immunoassays. Differences between medians were compared using the Mann‐Whitney U‐test. The Spearman rank test was used for the assessment of correlations between the intensity of reactivity of sera in LFIC assays and the NtAb titers. A p < .05 was considered statistically significant. The analyses were performed using SPSS version 20.0 (SPSS).

RESULTS

LFIC immunoassays performance

Categorical results obtained by the LFIC immunoassays and the NtAb assay are shown in Table 1. Out of a total of 90 sera, 83 tested positive by the NtAb assay, 82 by Wondfo SARS‐CoV‐2 Antibody test, 65 by INNOVITA 2019‐nCoV IgG, and 74 by INNOVITA 2019‐nCoV IgM. Taking RT‐PCR‐positive results as the reference, the data showed that the Wondfo SARS‐COV‐2 Antibody test had higher overall sensitivity than the INNOVITA 2019‐nCoV IgG and IgM, considered either separately or in combination, and approached that of NtAb, the most sensitive assay (92.2%) (Table 1). As expected, the sensitivity of LFIC assays increased when testing sera collected at relatively late times (≥15 days) after symptoms onset compared with sera collected earlier ( < 15 days). Notably, the Wondfo SARS‐CoV‐2 Antibody test, INNOVITA 2019‐nCoV IgM, and NtAb assay showed similar sensitivity for late sera. Considering the entire data set, the degree of agreement between qualitative results yielded by the commercial immunoassays and the NtAb assay was κ = 0.64 (p ≤ .001) for Wondfo SARS‐CoV‐2 Antibody test, κ = 0.29 (p ≤ .001) for INNOVITA 2019‐nCoV IgG, κ = 0.46 (p ≤ .001) for INNOVITA 2019‐nCov IgM, and κ = 0.56 (p ≤ .001) for the combination of INNOVITA 2019‐nCoV IgG and IgM testing.

Table 1

Clinical sensitivity of an antibody neutralization method using a green fluorescent protein (GFP) reporter‐based pseudotype (vesicular stomatitis virus‐VSV‐ backbone coated with SARS‐CoV‐2 spike protein) and two commercial SARS‐CoV‐2 lateral flow immunochromatographic antibody tests for COVID‐19 diagnosis

	No. of sera testing positive/negative (% sensitivity [% 95 CI])
Sera included in the analyses	GFP‐VSV‐ SARS‐CoV‐2 S pseudotype NtAb test	Wondfo SARS‐COV‐2 antibody test	INNOVITA 2019‐nCov IgG	INNOVITA 2019‐nCov IgM	INNOVITA 2019‐nCov IgG either IgG, IgM or both
All seraa	83/7 (92.2 [86.7–97.8])	82/8 (91.1 [83.4–95.4])	65/25 (72.2 [62.2–80.4])	74/16 (82.2 [73.1–88.8])	77/13 (85.6 [76.8–91.4])
Sera collected < 15 days after symptoms onseta	37/4 (90.2 [77.5–96.1])	36/5 (87.8 [74.5–94.7])	26/15 (63.4 [48.1–76.4])	28/13 (68.3 [53–80.4])	31/10 (75.6 [60.7–86.2])
Sera collected ≥ 15 days after symptoms onseta	46/3 (93.9 [83.5–97.9])	46/3 (93.9 [83.5–97.9])	39/10 (79.6 [66.4–88.5])	46/3 (93.9 [83.5–97.9])	46/3 (93.9 [83.5–97.9])

A total of 90 sera were included, of which 41 were collected < 15 days after symptoms onset and 49 later on (≥15 days).

As for the specificity of the LFIC assays, 20 control sera, including 10 from patients with previous endemic coronavirus infections, were tested. Only one serum returned positive results by the Wondfo SARS‐CoV‐2 antibody test, the INNOVITA 2019‐nCoV IgG and IgM assays (from a patient with a previous HCoV OC229 infection). Thus, the specificity of LFIC assays was 95% (95% CI, 76.4%–99.1%). In addition, all control sera tested negative by the NtAb assay (specificity; 100%; 95% CI, 83.9%–100%).

Prediction of high NtAb titers according to LFIC test line intensity

Median NtAb50 titers increased significantly in parallel with test line intensity (from 0 to 3+) in the Wondfo LFIC device (Figure 2). In contrast, median NtAb50 titers were not associated with the strength of reactivity of the positive test line (1+ to 3+) in the INNOVITA LFIC assay, either for IgG or IgM, although they were significantly different in sera returning negative results (0) from those yielding positive results (1+ to 3+). Of note, there were no sera exhibiting the highest reactivity (+3) when tested by the INNOVITA IgM LFIC assay.

Figure 2

Median neutralizing antibody titers (NtAb50) in sera from hospitalized COVID‐19 patients according to their strength of reactivity in lateral flow immunochromatographic assays (grading scale of test lines from 0 to 3+). p values for selected comparisons are shown

The degree of correlations between sera reactivity in the LFIC assays and NtAb titers were the following: Rho, 0.62 (95% CI, 0.47–0.73), p < .001 for the Wondfo device, Rho, 0.56 (95% CI, 0.39–0.69), p < .001 for the INNOVITA IgM assay, and Rho, 0.33 (95% CI, 0.12–0.50), p < .001, for the INNOVITA IgG assay.

High NtAb50 titers (≥1/160) were observed in 74 out of 90 sera. The Wondfo SARS‐CoV‐2 Antibody test best predicted NtAb50 titers ≥ 1/160, with reactivities ≥ 2+ having a negative predictive value (NPV) of 100% and a positive predictive value (PPV) of 96.4 (Table 2). In contrast, both the IgG and IgM INNOVITA LFIC assay exhibited good PPVs but suboptimal NPVs at a threshold of ≥ 2+. Setting the cutoff at a weaker band intensity (1+) returned slightly worse results for all LFIC.

Table 2

Prediction of neutralizing antibody titers ≥ 1/160, as measured by a green fluorescent protein (GFP) reporter‐based pseudotype (nonreplicative vesicular stomatitis virus backbone coated with SARS‐CoV‐2 spike protein), according to the intensity of reactivity of sera in two commercial Lateral Flow immunochromatographic antibody tests

Immunoassay	Parameter
	Threshold (intensity of test line)a	Area under a curve (95% CI)/p value	Specificity (95% CI)	Sensitivity (95% CI)	Positive predictive value (95% CI)	Negative predictive value (95% CI)
Wondfo SARS‐COV‐2 antibody test	≥1+ (n = 74)	0.86 (0.75–0.98)/ < .001	50 (28–72)	100 (95.1–100)	90.2 (81.9–95)	100 (67.6–100)
Wondfo SARS‐COV‐2 antibody test	≥2+ (n = 53)	0.89 (0.74–1)	80 (49–94.3)	100 (93.2–100)	96.4 (87.7–99)	100 (67.6–100)
INNOVITA 2019‐nCov IgG	≥1+ (n = 61)	0.76 (0.60–0.91)/.001	75 (50.5–89.8)	82.4 (72.2–89.4)	93.8 (85.2–97.6)	48 (30–66.5)
INNOVITA 2019‐nCov IgG	≥2+ (n = 15)	0.69 (0.52–0.86)/.05	85.7 (60.1–96)	53.6 (35.8–70.5)	88.2 (65.7–96.7)	48 (30–66.5)
INNOVITA 2019‐nCov IgM	≥1+ (n = 69)	0.79 (0.64–0.95)/.0002	68.8 (44.4–85.8)	93.2 (85.1–97.1)	93.2 (85.1–97.1)	68.8 (44.4–85.8)
INNOVITA 2019‐nCov IgM	≥2+ (n = 5)	0.71 (0.48–0.94)/.09	91.7 (64.6–98.5)	50 (23.7–76.3)	83.3 (43.6–97)	68.8 (44.4–85.8)

1 +  weak positive result (intensity of test line lower than the control line); 2, positive result (intensity of test line equal to that of the control line); 3, strong positive result (intensity of test line greater than that of the control line).

DISCUSSION

In this study, the Wondfo SARS‐CoV‐2 Antibody LFIC assay was found to exhibit excellent overall sensitivity (91.1%) and specificity (95%) in a cohort of COVID‐19 patients with moderate to severe forms of the disease. As expected, sensitivity was higher for sera collected beyond day 14 after the onset of symptoms than for those drawn earlier on. Even higher sensitivity for the Wondfo assay than that observed in the current study was reported by Martínez et al. in a mixed cohort, including SARS‐CoV‐2‐infected asymptomatic individuals and patients presenting with mild to severe COVID‐19. In turn, Guedes‐López et al. reported a sensitivity of 83% when testing sera collected between days 15–28 after symptoms onset in a cohort comprising symptomatic healthcare workers and patients admitted to the Emergency Department.

Overall, the INNOVITA 2019‐nCoV assay performed worse than the Wondfo SARS‐CoV‐2 Antibody LFIC assay in terms of sensitivity (72.2% for IgG and 82.2% for IgM), although combining the results of both test lines yielded an acceptable figure (85.6%). Nevertheless, when only sera obtained late after symptoms onset were analyzed, sensitivity was similar to the Wondfo assay (93.9%). To our knowledge, only one study has evaluated the performance of the INNOVITA LFIC assay. Yong et al. reported a sensitivity of 50% and 52% for IgM, and 87.5% and 91.3% in sera collected within 8‐15 days and ≥15 days after the onset of symptoms, respectively, in a cohort of hospitalized COVID‐19 patients. Differences in the precise timing of sera collection and the severity of COVID‐19 may account for these minor discrepancies across the abovementioned studies; as for the latter, Martínez et al. found a lower sensitivity for asymptomatic individuals (84.6%) than for the entire study group (89.9%).

A handful of studies have compared the performance of LFIC assays versus SARS‐CoV‐2 neutralization assays, , , , but none of the LFIC evaluated herein was included. These studies differed in many aspects, namely, the timing of sera collection, clinical presentation of COVID‐19, neutralization antibody assay employed, NtAb50 titer cutoff value for positive results, and the reference method for sensitivity calculations (either the NtAb assay itself or RT‐PCR results). Not surprisingly, overall sensitivities reported for these LFIC assays vary widely, ranging from 46% to 100%. , Here, we used a SARS‐CoV‐2‐S‐pseudotype as the viral input; nevertheless, NtAb levels measured by this assay have been shown to correlate strongly with those from assays using live SARS‐CoV‐2. Although the overall sensitivity of both LFIC assays was lower than the neutralization assay, it was comparable when sera collected ≥15 days were analyzed separately.

A novel contribution of the current study is that a strong association was found between the strength of test line reactivity and NtAb50 titers, notably when employing the Wondfo SARS‐CoV‐2 Antibody assay. Using a simple grading scale, we could discriminate reasonably well between sera containing high and low NtAb50 titers (≥1/160 or <1/160, respectively). In sera giving reactivities ≥ 2+ (comparable to or more intense than the control line), we identified high‐NtAb level sera with excellent PPV and NPV. Weidner and colleagues were the first to prove the suitability of this approach. By using a 4‐level intensity scale on the Wantai SARS‐CoV‐2 Ab rapid test, they were able to predict NtAb50 titers (live SARS‐CoV‐2) > 1/200 with NPV and PPV of 92%. The Wantai assay detects antibodies binding the S receptor‐binding domain, which includes several highly immunogenic epitopes eliciting potent NtAb responses within several epitopes, , making the above‐reported association somewhat unsurprising. The antigenic target/s of the LFIC assays evaluated herein are unknown to us, thus precluding speculation on that matter.

The observer‐dependent scoring of test line reactivity may be construed as a limitation of this study, although in fact, all four readers evaluating LFIC results concurred in the categorization of all sera. Moreover, readings were consistent across different rounds of testing (not shown).

In summary, our data support the use of all LFIC assays evaluated herein, particularly the Wondfo test, for COVID‐19 diagnosis, especially when testing sera collected late after symptoms onset. In addition, we have shown that these rapid immunoassays can be used to infer the neutralizing activity of sera against SARS‐CoV‐2.

CONFLICT OF INTERESTS

The authors have no conflict of interest.

AUTHOR CONTRIBUTIONS

Arantxa Valdivia: Methodology, data analysis, validation, review and editing; Ignacio Torres: Formal analysis, review, and editing; Víctor Latorre: Methodology, investigation; Clara Francés‐Gómez: Methodology, investigation; Estela Giménez: Resources, project administration, review, and editing; Roberto Gozalbo‐Rovira: Methodology, investigation, validation, funding acquisition, review, and editing; Carlos Solano de la Asunción: Methodology, data analysis, validation, review, and editing; Javier Buesa: Supervision, review, and editing; Estela Giménez: Data analysis, validation, review, and editing; Jesús Rodríguez‐Díaz: Conceptualization, supervision, funding acquisition, review, and editing; Ron Geller: Methodology, investigation, validation, funding acquisition, review, and editing; David Navarro: Conceptualization, supervision, writing the original draft, review, and editing.