Technical performance of a lateral flow immunoassay for detection of anti-SARS-CoV-2 IgG in the outpatient follow-up of non-severe cases and at different times after vaccination: comparison with enzyme and chemiluminescent immunoassays.

This study assessed the technical performance of a rapid lateral flow immunochromatographic assay (LFIA) for the detection of anti-SARS-CoV-2 IgG and compared LFIA results with chemiluminescent immunoassay (CLIA) results and an in-house enzyme immunoassay (EIA). To this end, a total of 216 whole blood or serum samples from three groups were analyzed: the first group was composed of 68 true negative cases corresponding to blood bank donors, healthy young volunteers, and eight pediatric patients diagnosed with other coronavirus infections. The serum samples from these participants were obtained and stored in a pre-COVID-19 period, thus they were not expected to have COVID-19. In the second group of true positive cases, we chose to replace natural cases of COVID-19 by 96 participants who were expected to have produced anti-SARS-CoV-2 IgG antibodies 30-60 days after the vaccine booster dose. The serum samples were collected on the same day that LFIA were tested either by EIA or CLIA. The third study group was composed of 52 participants (12 adults and 40 children) who did or did not have anti-SARS-CoV-2 IgG antibodies due to specific clinical scenarios. The 12 adults had been vaccinated more than seven months before LFIA testing, and the 40 children had non-severe COVID-19 diagnosed using RT-PCR during the acute phase of infection. They were referred for outpatient follow-up and during this period the serum samples were collected and tested by CLIA and LFIA. All tests were performed by the same healthcare operator and there was no variation of LFIA results when tests were performed on finger prick whole blood or serum samples, so that results were grouped for analysis. LFIA's sensitivity in detecting anti-SARS-CoV-2 IgG antibodies was 90%, specificity 97.6%, efficiency 93%, PPV 98.3%, NPV 86.6%, and likelihood ratio for a positive or a negative result were 37.5 and 0.01 respectively. There was a good agreement (Kappa index of 0.677) between LFIA results and serological (EIA or CLIA) results. In conclusion, LFIA analyzed in this study showed a good technical performance and agreement with reference serological assays (EIA or CLIA), therefore it can be recommended for use in the outpatient follow-up of non-severe cases of COVID-19 and to assess anti-SARS-CoV-2 IgG antibody production induced by vaccination and the antibodies decrease over time. However, LFIAs should be confirmed by using reference serological assays whenever possible.

INTRODUCTION

The etiologic agent of the coronavirus disease 2019 (COVID-19) pandemic is SARS-CoV-2, a new coronavirus whose laboratory diagnosis during the acute phase of infection is made preferentially through viral cDNA amplification by quantitative reverse transcriptase polymerase chain reaction (RT-PCR) performed on nasopharyngeal and oropharyngeal swabs . Alternatively, the rapid antigen detection assays can be applied when there is a huge demand for COVID-19 testing in people with acute respiratory symptoms .

Detection of anti-SARS-CoV-2 antibodies is of interest in scenarios different from those seen in emergency services. Although the humoral response to SARS-CoV-2 is not yet fully understood, it is estimated that approximately 60% of infected people produce IgM antibodies from day 4 after the onset of symptoms. While IgM antibodies peak between 14 and 21 days and then decrease, IgG levels begin to increase around 7-14 days after the onset of symptoms, peaking around day 25. It is unclear how long the IgG antibody production is maintained, although some infected people have IgG antibodies detectable 2-3 months after the acute phase of COVID-19 .

The anti-SARS-CoV-2 antibodies are directed against various viral proteins, such as the envelope (E), nucleocapsid (N), spike protein (S), or peptides from the S1 and S2 spike protein subunits . Antibodies directed against the receptor-binding domain (RBD) of the S1 viral subunit, as well as the N protein, to a lesser extent, seem to be partially associated with anti-SARS-CoV-2 neutralizing antibody titer . The binding of the viral RBD to the host’s cellular angiotensin-converting enzyme 2 (ACE2) receptor allows SARS-CoV-2 binding and entering human cells, making the infection effective so that the detection and quantitation of neutralizing antibodies are the preferential tests as they are directly associated with protection by preventing the virus from binding to the ACE2 receptor of human cells, carrying out the infection .

Another application for anti-SARS-CoV-2 antibody detection is to confirm COVID-19 diagnosis some days or weeks after the onset of symptoms, in symptomatic suspected cases in which RT-PCRs were persistently negative . The sensitivity of RT-PCR depends on the accuracy of respiratory secretion collection protocols, the arrival speed of respiratory samples to the laboratory, if the samples were collected at the appropriate time (when the virus is detectable) and the RT-PCR protocol (conventional RT-PCR with amplification of only one SARS-CoV-2 target or more advanced RT-PCR techniques with hydrolysis probes, various SARS-CoV-2 targets in addition to internal and external controls) has been fulfilled.

In epidemiological studies, the serological assays can provide an estimate of individuals who have already been exposed to SARS-CoV-2 in a given population, through an indistinct identification of previously naturally infected people and others vaccinated, allowing contact tracing and surveillance of those most at risk of having more serious diseases due to the presence of comorbidities .

There are currently over 350 commercially available or under development serological kits aimed at detecting antibodies, mainly for both immunoglobulin M (IgM) and immunoglobulin G (IgG) produced in response to SARS-CoV-2 infection, fully automated high-throughput enzyme immunoassays (EIA) or chemiluminescent immunoassays (CLIA) platforms in addition to rapid tests based on Lateral Flow Immunoassays (LFIA) .

During the COVID-19 pandemic, rapid serological LFIA devices containing SARS-CoV-2 antigens conjugated to colloidal gold became widely used. In LFIA, these SARS-CoV-2 specific antigens will specifically bind to IgM or IgG present in the patient’s whole blood, serum or plasma samples. LFIAs have been widely used due to the fact that they are quick and easy to perform, with no need for specialized training or equipment. However, reactive specimens for anti-SARS-CoV-2 IgM and/or IgG detected by LFIA should be confirmed by reference serological methods such as EIA and CLIA .

David et al. tested 30 commercial LFIAs using serum or plasma samples from patients with confirmed SARS-CoV-2 infections. The negative serological controls were accessed from a well-characterized biorepository of serum samples collected and stored prior to the COVID-19 pandemic. Ninety percent of the evaluated LFIAs detected both IgG and IgM and their results showed that only 4/30 (13%) of LFIAs tested, namely the Zhejiang Orient Gene COVID-19 IgG/IgM, the Genrui Novel Coronavirus (2019-nCoV) IgG/ IgM, the Biosynex COVID-19 BSS IgG/IgM and the Boson Biotech 2019-nCoV IgG/IgM) reached a sensitivity level to be submitted for the South African Health Products Regulatory Authority (SAHPRA) approval. Among these, only the Zhejiang Orient Gene COVID-19 IgG/IgM was actually recommended by SAHPRA in August 2020 for use within the approved national testing algorithm. This study highlighted the need for a thorough investigation of LFIA kits under the same clinical conditions in which they are intended to be used prior to their routine implementation.

There are many LFIAs available for the detection of anti-SARS-CoV-2 IgM and IgG antibodies and extensive literature on their performance, but few studies analyzed two specific scenarios: patients initially diagnosed with non-severe COVID-19 by RT-PCR referred to outpatient follow-up and vaccinated people tested at different times after vaccination in order to assess the production and decreasing of anti-SARS-CoV-2 IgG antibodies over time.

MATERIALS AND METHODS

Ethical issues

This study is part of a large cohort of children and adolescents with comorbidities and COVID-19 in the 2020 pandemic and was approved by the Brazilian National Research Ethics Committee (process Nº 30344420.6.0000.0008) in April 2020. We also obtained finger prick whole blood samples and serum samples from vaccinated volunteers who are health care workers, mainly biologists, medical doctors and their family members. After signing the informed consent form, they were enrolled in the study.

Study groups and reference serological tests to control LFIA results

This study tested a total of 216 whole blood or serum samples from three specific groups.

Group of true negative samples

This group consisted of 68 true negative participants: 30 blood bank donors, 30 healthy volunteers and eight children diagnosed with other coronavirus infections. The serum samples from these participants were collected and stored at -20 ºC, between 2010-2015, in a pre-COVID-19 period.

Group of true positive samples

The second group consisted of 96 “true positive” participants. Instead of using samples from COVID-19 patients, we investigated adult volunteers working in our institution, including biologists, medical doctors and their family members. We expected them to have produced anti-SARS-CoV-2 IgG antibodies as they had already received two or three doses of the vaccine, with the last shot 30-60 days before LFIA testing. To control LFIA results, serum samples were collected on the same day of LFIA and tested by an in-house EIA based on detection of SARS-CoV-2 nucleocapsid antigen, further improved by the addition of a viral spike antigen . In this EIA, positivity for anti-SARS-CoV-2 IgG antibodies is defined as a reactivity index ≥ 1.0. Due to this antigenic composition, this EIA is able to detect SARS-CoV-2 IgG antibodies in vaccinated people who received whole inactivated viruses, viral vectors or mRNA vaccines.

Group of participants with positive or negative anti-SARS-CoV-2 IgG antibodies

This third group was composed of 52 people (40 children and adolescents and 12 adults). The 40 children sought medical care when they manifested acute respiratory syndrome and received the diagnosis of COVID-19 due to a positive RT-PCR . As these 40 children were diagnosed with non-severe COVID-19, they were referred for outpatient follow-up and during this period serum samples were collected on the 2nd and 3rd weeks after the onset of symptoms in order to perform serology (CLIA – Elecsys anti-SARS-CoV-2 S protein IgG, Roche Diagnostics, Basel, Switzerland) and LFIA. In this third group, we also included 12 adults for whom LFIA was performed more than seven months after the second shot of the vaccine (booster dose). To control LFIA results, serum samples collected on the same day of LFIA underwent the aforementioned in-house EIA .

Reference serological assays

In respect to the group of true positive participants and the group with participants with positive or negative anti-SARS-CoV-2 IgG antibodies, both had LFIA results controlled by a reference serological assay. Furthermore, LFIAs were performed by one laboratory and the serological tests by another laboratory, so that one team was unaware of the results obtained by the other.

The Lateral Flow Immunoassay (LFIA) procedure

The 2019-nCoV IgG/IgM Combo Rapid Test (MP Biomedicals Germany GmbH, Eschwege, Germany) is a LFIA able to detect anti-SARS-CoV-2 IgM and/or IgG antibodies. The reaction area of this immunochromatographic device is covered with antigens (nucleocapsid and spike SARS-CoV-2 proteins) immobilized and conjugated to colloidal gold. The SARS-CoV-2 antibodies (IgM and/or IgG) present in the participant’s five microliters of whole blood, serum or plasma sample bind to their corresponding antigens and after the antigen-antibody complex formation, the conjugated colloidal gold precipitates and one or two red lines come up, indicating the presence of anti-SARS-CoV-2 IgM and IgG antibodies. LFIA’s control line indicates whether the reaction was performed properly or not. The reaction area of this control line is covered with rabbit IgG antigen conjugated to colloidal gold and the reaction buffer provides the anti-rabbit IgG needed to trigger the reaction and induce the colloidal gold precipitation that, in turn, forms the reaction control red line. The optimal time for reading the test is 15 min.

LFIA’s sensitivity, specificity and accuracy

According to the manufacturer, the 2019-nCoV IgG/IgM Combo Rapid Test (MP Biomedicals Germany GmbH, Eschwege, Germany) has clinical sensitivity and specificity of 94.74% and 97.06%, respectively, with 95.87% accuracy. Regarding the sensitivity and specificity in detecting anti-SARS-CoV-2 IgM, the results were 70.59% and 99.67%, respectively, with 84.74% accuracy, while sensitivity and specificity for the detection of anti-SARS-CoV-2 IgG were 90.40% and 97.39%, respectively, with 93.80% accuracy.

Interference caused by biological substances and commonly used drugs

The manufacturer claims that the 2019-nCoV IgG/IgM Combo Rapid Test result is not influenced by bilirubin, triglycerides and hemoglobin at concentrations up to 342 μmol/L, 37 mmol/L and 10 ng/mL, respectively, nor by drugs commonly used in patients suffering from Severe Acute Respiratory Syndrome such as antivirals (zanamivir 426 ng/mL, ribavirin 6 mg/L, oseltamivir 46.9 mg/L, peramivir 132.7 μg/mL, lopinavir 3.2 mg/mL, ritonavir 159 μg/mL and arbidol 2.0 μg/mL), as well as antibiotics (levofloxacin 9.2 mg/L, azithromycin 1.2 μg/mL, ceftriaxone 240 mg/L, meropenem 200 mg/L and tobramycin 12 mg/L).

Statistical analysis

A minimum total sample size of 100 was determined based on the minimum number of samples to reach a sensitivity of 90%, with a testing power of 80% . The sensitivity and specificity for detecting specific anti-SARS-CoV-2 IgG antibodies were calculated under the following assumptions: all samples obtained prior to the COVID-19 pandemic were considered as true negative (TN); all samples from recently vaccinated participants were considered as true positive. Sensitivity was calculated as TP/TP+FN (FN-false negative); specificity was calculated as TN/TN+FP (FP-false positive); efficiency was calculated as TP+TN/TP+FP+TN+FN; positive predictive value (PPV) was calculated as TP/TP+FP; negative predictive value (NPV) was calculated as TN/TN+FN; the likelihood ratio for a positive result was calculated as sensitivity/1-specificity; the likelihood ratio for a negative result was calculated as 1-sensitivity/specificity. The agreement between results generated by LFIA and the reference serological methods, i.e. in-house EIA or commercial CLIA was assessed by the Kappa index and interpreted as: bad (< 0); poor (0 – 0.2); unsatisfactory (0.21 – 0.40); regular (0.41 – 0.60); substantial or good (0.61 – 0.80) and almost perfect agreement (0.81 – 1.0) .

RESULTS

This study analyzed a total of 216 whole blood or serum samples, and Figure 1 summarizes the groups and tests performed.

Figure 1

The total number of participants and samples analyzed in the study (n=216), their division into three groups of true negatives (n=68), true positives (n=96) and samples with unknown results (n=52). In the groups of true positives and the group with unknown results for COVID-19, LFIA results were controlled by a reference serological method (EIA or CLIA).

To assess LFIA’s specificity, 68 participants (30 blood bank donors, 30 healthy young volunteers and eight children with other coronavirus infections) were tested (group 1). Among the 60 adult participants (30 blood donors and 30 healthy volunteers), 50% were males. Regarding the 30 blood donors, 28 serum samples were negative by LFIA and two samples were positive (only for IgG). Analyzing the database of these 30 blood donors, there were 28 non-reactive donors for all blood bank mandatory serological tests (syphilis, HIV, HCV, HBV and Chagas disease) and in the two blood donors whose serum samples were positive for anti-SARS-CoV-2 IgG by LFIA, we also found positive results for syphilis, as their serum samples tested undiluted and at a 1:10 dilution were positive by VDRL (Venereal Disease Research Laboratory). Therefore, we considered that 28 blood bank donors were true negative for anti-SARS-CoV-2 IgG antibodies when tested by LFIA and two blood donors showed false-positive results. Regarding the 30 healthy young volunteers (18-45 years old), all 30 serum samples were IgG-negative by LFIA. Then, we further evaluated LFIA’s specificity by testing eight serum samples from children (6 boys and 2 girls) with ages varying from 6 months to 11 years old, diagnosed with other coronavirus infections (OC43, HKU1, NL63 and 229E) by RT-PCR performed on respiratory secretions at the time of acute infections. Their serum samples collected on the 2nd or 3rd weeks after the onset of symptoms were tested by LFIA and they were all negative for anti-SARS-CoV-2 IgG antibodies. The ultimate goal for testing these eight children was to assess cross-reactions of LFIA with phylogenetically related coronaviruses.

The second study group (true positive cases) comprised 96 vaccinated participants tested 30-60 days after the second dose (most cases) or third dose (only those older than 80 years old) of COVID-19 vaccines (Coronavac from Sinovac Biotech, China; Vaxzevria, formerly named Oxford-AstraZeneca COVID-19 vaccine, now AstraZeneca PLC, Cambridge, UK); and Comirnaty (Tozinameran), the Pfizer-BioNTech COVID-19 vaccine, manufactured in Brussel, Belgium). This group was composed of 88 women and eight men, ranging from 18-93 years old, 15 of 96 (15.6%) were > 60 years old and eight of 96 (8.3%) reported one or more comorbidities, mainly cardiovascular disease (high blood pressure as the leading cause), type II diabetes mellitus and chronic obstructive pulmonary disease. Five relatives of workers of our institution were > 75 years old and all of them reported at least one comorbidity. Our LFIA found 95 positives out of 96 (98.95%) tests performed through finger prick. Although these 96 participants were all asymptomatic prior to LFIA testing, it is not possible to rule out that some of them may have had natural infections. In any case, this was our group of participants “previously exposed to SARS-CoV-2”, either by vaccination or eventual natural asymptomatic infections leading to the production of anti-SARS-CoV-2 IgG. The only sample (1.05%) with a negative LFIA result came from a 93-year-old man who was a health care worker relative and reported to have severe chronic obstructive pulmonary disease and high blood pressure. He received the third dose of the vaccine 45 days before his LFIA testing. A serum sample collected on the same day of LFIA was sent to a private laboratory and was investigated through a highly sensitive commercial CLIA serology for COVID-19 (Architect anti-SARS-CoV-2 protein S IgG detection by Abbott Laboratories, Chicago, Illinois, USA). According to the manufacturer, this serological assay has sensitivity of 43% (3-7 days after the onset of symptoms), 78.9% (8-14 days) and 100% (≥ 15 days), and specificity of 99.60% regardless of the period. In this elderly man, the absence of IgG directed against SARS-CoV-2 revealed by LFIA was confirmed by CLIA. In addition, all the 96 participants of this group had their LFIA results controlled by an in-house EIA which also confirmed the 95 IgG-positive and the one IgG-negative result in the same 93-year-old man with comorbidities that had previously tested negative by LFIA and CLIA. As we did not find variations between results produced by finger prick whole blood (LFIA) and serum samples (EIA and CLIA) performed in parallel, the results produced by the two types of biological material were analyzed together.

In the third group composed of 52 participants (40 children and adolescents and 12 adults), they did or did not have positive or negative results for COVID-19 due to specific scenarios. The 40 children and adolescents with suspected COVID-19 due to acute respiratory symptoms, underwent RT-PCR on admission to the emergency room and the molecular test was positive in all 40 children. As these 40 children were diagnosed with non-severe COVID-19, they were referred for outpatient follow-up and during this period the serum samples were collected on the 2nd and 3rd weeks after the onset of symptoms and tested by a commercial CLIA (Elecsys anti-SARS-CoV-2 protein S- CLIA, Roche Diagnostics GmbH, Mannheim, Germany). According to the manufacturer, this CLIA has sensitivity of 90.6% (0-7 days), 87% (8-14 days) and 96.6% (≥ 15 days), and specificity of 100% regardless of the period. However, it is known that not all COVID-19 patients, especially the ones with non-severe disease, produce detectable amounts of antibodies against SARS-CoV-2 and maintain these antibody titers for longer periods of time . The 40 serum samples from the follow-up were analyzed by CLIA and revealed the presence of 22 positive and 18 negative anti-SARS-CoV-2 IgG results. Afterward, they were tested by LFIA resulting in 20 positive and 20 negative anti-SARS-CoV-2 IgG samples. In terms of agreement, there were 38 (38/40 or 95%) concordant results between CLIA and LFIA, taking the CLIA results as the gold standard. In other words, two samples (5%) had discordant CLIA and LFIA results and both samples were IgG-positive by CLIA and negative by LFIA, thus LFIA failed to detect IgG antibodies in two samples (two false-negative results).

In the remaining 12 adults of this third group who work in our healthcare institution, they were 31-74 years old and had received the second dose (booster dose) of the vaccine more than seven months before LFIA was performed on whole blood (prick test). Therefore, they could still have detectable anti-SARS-CoV-2 IgG antibodies, although a reduction of anti-SARS-CoV-2 antibodies induced by vaccination is known to happen . Ten of the 12 participants tested negative for anti-SARS-CoV-2 IgG antibodies by LFIA and two participants tested positive, and these results were all confirmed (total agreement) by the in-house EIA-IgG .

According to the results obtained in the three groups analyzed in this study (true negative, true positive, and a third group of cases with uncertain results for COVID-19), and considering that the true positive group and the group with uncertain results were controlled by serological tests (EIA or CLIA), LFIA and the serological results were used to construct a contingency table (Table 1).

Table 1

Contingency table comparing the proportion of positive and negative results in the two groups (reference laboratory methods for the diagnosis of COVID-19 vs. LFIA – IgG by the MP 2019-nCoV IgG/IgM Combo rapid test).

COVID-19 reference methods (RT-PCR/EIA/CLIA)
LFIA-IgG (MP 2019-nCoV IgG/IgM)	Positive	Negative	Total
Positive	117	2	119
Negative	13	84	97
Total	130	86	216

Fisher’s exact test p<0.0001

Table 1 illustrates the comparison of proportions (positive and negative results) evidenced by COVID-19 reference serological methods (EIA/CLIA) and LFIA (MP 2019-nCoV IgG/IgM Combo rapid test). The Fisher’s exact test was applied and found a statistically significant difference (p<0.0001).

Thereafter, data from Table 1 were used to calculate the technical performance of LFIA-IgG detection by the MP Rapid 2019-nCoV IgG/IgM Combo test: sensitivity, specificity, positive and negative predictive values, efficiency, likelihood ratio for a positive result and likelihood ratio for a negative result (Table 2).

Table 2

Clinical performance of LFIA-IgG (MP 2019-nCoV IgG/IgM Combo rapid test).

LFIA-IgG Result		95% CI
Sensitivity (%)	90.0	0.8351 – 0.9457
Specificity (%)	97.6	0.9185 – 0.9972
Efficiency (%)	93.0	–
Positive predictive value (%)	98.3	0.9406 – 0.9980
Negative predictive value (%)	86.6	0.7817 – 0.9267
Likelihood ratio for a positive	37.5	–
Likelihood ratio for a negative	0.01	–

95% CI = 95% Confidence Interval

Then, the agreement between the groups of assessments (serology vs. LFIA) was calculated by means of the Kappa index, resulting in 0.677 (95% CI: 0.576 to 0.778). Please refer to Table 3.

Table 3

Agreement between the results of the reference laboratory methods for the diagnosis of COVID-19 (RT-PCR/EIA/CLIA) and LFIA tested (MP 2019-nCoV IgG/IgM Combo test) according to the Kappa index.

COVID-19 reference methods (RT-PCR/EIA/CLIA)
LFIA-IgG (MP 2019-nCoV IgG/IgM)	Positive	Negative	Total
Positive	117	20	137
Negative	13	66	79
Total	130	86	216

Kappa index = 0.677 (95% CI: 0.576 to 0.778); Kappa index interpretation = between 0.61 and 0.80 the Kappa index is considered a substantial (good) agreement

DISCUSSION

In this study, we performed a technical evaluation of a commercially available LFIA (MP 2019-nCoV IgG/IgM Combo rapid test, manufactured by MP Biomedicals Germany GmbH, Eschwege, Germany) and the results were compared with the results of an in-house EIA anti-SARS-CoV-2 nucleocapsid/spike IgG antibodies or automated CLIA platforms from Roche Diagnostics and Abbott Laboratories.

Although a statistically significant difference was found between the proportions of positive and negative results (Table 1) obtained by LFIA and reference serological methods (Fisher’s exact test p< 0.0001), showing that they were not equivalent when the agreement between the two assessments (serology vs. LFIA) was calculated, the resultant Kappa index was 0.677, i.e. there was a substantial (good) agreement between the two laboratory techniques, as shown in Table 3.

Table 2 shows that LFIA tested in this study reached a 90% sensitivity in detecting anti-SARS-CoV-2 IgG antibodies, 97.6% specificity, 93% efficiency, 98.3% PPV, 86.6% NPV and likelihood ratio for a positive or a negative result of 37.5 and 0.01, respectively. When these rates are compared with those provided by the manufacturer (90.4% sensitivity, 97.39% specificity and 93.80% accuracy), one can observe that our LFIA results are very similar to the manufacturer’s ones.

Due to our study design, we did not evaluate the LFIA for its capacity to detect IgM antibodies raised to SARS-CoV-2. Our results are based only on IgG detection, as we did not test serum samples from the acute phase of infections that would likely be negative for anti-SARS-CoV-2 IgM antibodies if collected in the emergency room . Moreover, our aim was not to use LFIA as a diagnostic method for COVID-19, as RT-PCR is the gold standard for the early diagnosis of COVID-19 .

In addition to testing the traditional groups of true negative and true positive samples, we aimed at investigating two specific scenarios in which LFIA could be useful: at different times after vaccination, to verify if participants have already been exposed to SARS-CoV-2 and if IgG antibodies production was induced by vaccination (or eventually by a non-diagnosed asymptomatic natural infection), and to observe the decrease of vaccine-induced IgG antibodies over time . The second scenario of interest was to evaluate the immune response during outpatient follow-up of non-severe cases of COVID-19 as this group of patients did or did not produce detectable antibodies, depending on the sensitivity of serological techniques implemented and on the time elapsed after the symptomatic phase .

We evaluated a group of true negative samples from a pre-pandemic period and this was the only group in which LFIA results were not controlled by reference serological assays, as we did not expect to find positive samples among these participants. In the group of true positive samples, LFIA results were confirmed by EIA, and in the only case of negative anti-SARS-CoV-2 IgG by LFIA, this result was corroborated by CLIA and EIA. This was the reason not to consider the IgG-negative result as a false-negative of LFIA. In this particular case, we raised the hypothesis of immune senescence to explain the failure to produce anti-SARS-CoV-2 IgG antibodies after three doses of vaccine, a fact that has been highlighted in the literature .

In the third group of 52 participants that did or did not have positive or negative for SARS-CoV-2 IgG antibodies, among the 40 pediatric participants, although a high (95%) concordance of results was found between LFIA results and reference serological methods, there were two false-negative results by LFIA, reinforcing that CLIA results are the gold standard. As for the 12 vaccinated adults, LFIA and serology (EIA) showed a total agreement of results, revealing that most participants had no longer detectable anti-SARS-CoV-2 IgG antibodies after more than seven months following the vaccine booster dose. This waning of antibodies in natural infections and after vaccination has already been described. Although the ideal test to investigate the effectiveness of vaccination is the one that targets the neutralizing antibodies as they prevent the binding of the viral spike RBD region to the human cell receptor ACE2 . In the context of epidemiological field studies in which performing reference serological methods is not feasible, the detection of anti-SARS-CoV-2 IgG antibodies by LFIAs fulfills the role of characterizing previous exposure to SARS-CoV-2 that happened through natural infections and/or vaccination. More recently, a new LFIA has been developed with recombinant viral spikes containing the RBD region as the immobilized antigen conjugated to colloidal gold, so that this more advanced LFIA allows the timely detection of neutralizing anti-SARS-CoV-2 antibodies .

Concklin et al. assessed 15 different LFIAs for the detection of SARS-CoV-2 antibodies and analyzed 100 well-characterized samples: 40 samples from COVID-19 convalescent patients (collected on average 45 days after the onset of symptoms) to assess sensitivity; a negative control group of 60 samples collected in a pre-pandemic period to assess specificity. Among the five LFIAs, sensitivity and specificity ranged from 55-97% and 78-100%, respectively. When the performance of IgM and IgG detection was evaluated, sensitivity and specificity ranged from 0-88% and 80-100% for IgM, and 25-95% and 90-100% for IgG. The authors concluded that the performances of the 15 LFIAs differed widely and the greatest variation was found in sensitivity. When these results were compared with our results, we found 90% sensitivity (95% CI, 0.8351 to 0.9457) and 97.6% specificity (95% CI, 0.9185-0.9972), thus placing the LFIA tested by us among those with the highest sensitivity and specificity.

Flower et al. assessed the performance of 11 commercial LFIAs in parallel with two in-house EIAs using 276 serum samples from COVID-19 RT-PCR-positive patients in addition to their serum samples collected ≥ 21 days after the acute phase. The specificity analysis tested 500 pre-pandemic serum samples. The authors began by evaluating self-administered whole blood LFIA compared to LFIAs performed by laboratory personnel and to EIA results. The selection of the best kits was based on the performance in whole blood and serum samples (EIA). The concordance between LFIA vs. clinical suspicion and between whole blood vs. serum results was also estimated. Based on the combination of LFIA’s usability, high specificity (98.6%), 84% sensitivity with whole blood and 88% with serum, 87% PPV, moderate (0.56) Kappa agreement, and availability for testing at scale, the Fortress rapid test (IgM and IgG detection LFIA kit manufactured by Fortress Diagnostics Ltd, Co. Antrim, Northern Ireland UK) was the best to perform LFIA. When these results were compared to our results, we found a slightly better sensitivity (90%), an equivalent excellent specificity (97.6%) and a Kappa index of 0.67 (good or significant) while they found only a regular agreement. The authors concluded that LFIA’s sensitivity and the concordance of results were variable, highlighting the importance of evaluating each LFIA in the specific setting of intended use, as we did in our study by evaluating people at different times after vaccination or children diagnosed with non-severe COVID-19 on the 2nd or 3rd weeks after the onset of symptoms.

In an interesting survey , one LFIA (SARS-CoV-2 IgM/IgG) and one EIA (SARS-CoV-2 for IgM and IgG detection) assessed anti-SARS-CoV-2 IgM and IgG antibodies in 1,150 serum samples stored before the COVID-19 pandemic, in addition to 15 samples from patients with suspected SARS-CoV-2 infections. The urea avidity test was performed to rule out false-positive reactivity. LFIA revealed 21 (1.8%) positive results among 1,150 samples tested: 12 for IgM, four for IgG and five for both (IgM and IgG). After treatment with urea, only two IgM-positive results have persisted. In the group of 15 patients (nine with and six without COVID-19), LFIA found nine positives (six for IgG and three for IgM and IgG) and 65 negative samples. After treatment with urea, there were eight IgG-positive and one IgM/IgG-positive sample. When the nine COVID-19-positive samples were tested by EIA, there were seven IgG-positive and two IgM/IgG-positive samples. After treatment with urea, the results remained unchanged. When the six IgM/IgG-negative samples by LFIA were tested by EIA, the results remained unchanged. The authors concluded that the use of pre-COVID-19 samples as negative controls has a limited utility, and this reasoning may also be true in this study, as we found two false-positive results by LFIA in pre-pandemic samples. Furthermore, the authors argued that immunoassays are useful to confirm LFIA results and more importantly, that the urea dissociation assay is effective to avoid misdiagnosis .

In this study, most LFIA interfering factors were listed in the Material & Methods section according to information provided by the manufacturer (MP Biomedicals Germany GmbH, Eschwege, Germany), including a number of antiviral drugs and antibiotics, interference triggered by excess bilirubin, hemoglobin and cholesterol. Interference factors such as excess bilirubin, hemoglobin and cholesterol do not seem to be the case in our two blood donors that were LFIA IgG-positive. Although the presence of the rheumatoid factor was not investigated by us, it is known to interfere with IgM detection, justifying the testing of 10 samples from cases of rheumatoid arthritis in another study . Our study focused on anti-SARS-CoV-2 IgG detection and to test cross-reactivity we evaluated eight samples from proven infections caused by other seasonal coronaviruses (OC43, HKU1, NL63 and 229E) and none of them was positive for SARS-CoV-2 IgG by LFIA.

Surprisingly, two blood donors belonging to the group of true negative samples showed positive IgG results by LFIA, and they were also positive by VDRL, a screening test for syphilis. To further investigate these blood donors, immunoassays to detect anti-T. pallidum antibodies should have been performed to confirm or rule out the diagnosis of syphilis. Unfortunately, we no longer had access to these medical records, but we presume that these two blood donors did not have syphilis.

The VDRL is a non-treponemal test based on the detection of anticardiolipin antibodies. More recently, anticardiolipin antibodies were recognized as part of the antiphospholipid syndrome, as well as anti-nuclear antibodies, lupus anticoagulants, anti-prothrombin antibodies, anti-β2-glycoprotein I antibodies of immunoglobulin G and M classes, anti-β2 glycoprotein I domain 1, anti-phosphatidyl serine antibodies and anti-phosphatidyl serine/prothrombin complex antibodies . Therefore, the antiphospholipid syndrome can be part of autoimmune diseases and has also been described in COVID-19, but in the case of our two healthy blood donors with positive VDRL, i.e., presenting with anticardiolipin antibodies, this is more likely due to cross-reactions and a true detection of anticardiolipin antibodies, rather than a diagnosis of syphilis or COVID-19. However, the real significance of these anticardiolipin antibodies remains unclear as the two blood donors did not have any chronic disease. The possibility of having anticardiolipin antibodies associated with asymptomatic COVID-19 was not plausible since COVID-19 did not exist at the time these serum samples were collected.

Recently, Jassam et al. evaluated the diagnostic accuracy of the same commercial LFIA evaluated by us. They compared LFIA results with those from a serological platform (Elecsys anti-SARS-CoV-2 protein S – CLIA, Roche Diagnostics GmbH, Mannheim, Germany). Serum samples from 144 cases of COVID-19 confirmed by RT-PCR and 130 pre-pandemic controls were tested in parallel by the two methods. During the first week (days 1-7) both methods had comparable sensitivities of 74% for LFIA and 67% for Elecsys (p=0.3947). The sensitivity 15 days after the onset of symptoms onwards was 100% for both methods. LFIA’s specificity was 100% for IgG and 98.5% for IgM vs. 100% for both antibody isotypes by Roche Elecsys. The concordance between MP Biomedicals LFIA and Roche Elecsys CLIA was almost perfect (Kappa index of 0.96). When the results of the study by Jassam et al. were compared to our results, we obtained 90% sensitivity, but our samples were representative of children with non-severe COVID-19 evaluated with samples collected earlier, starting from day 8 to day 21 after the onset of symptoms. This may justify our lower sensitivity (90% vs. 100%). Moreover, their LFIA’s specificity in detecting IgG was 100%, while we had 97.6, although rates > 95% are adequate. In respect to the agreement of results between immunoassay vs. LFIA, Jassam et al. reported a Kappa index of 0.96 (almost perfect), while we found a Kappa index of 0.677 (significant or good). A possible explanation for this difference is the more heterogeneous nature of our groups, highlighting the test of vaccinated people at different times after vaccination and non-severe COVID-19 cases during the outpatient follow-up.

Our study has strengths and weaknesses. The strengths are related to the reasonable size of the sample tested when compared to the literature, our LFIA results were controlled by reference serological methods (EIA, CLIA) and we investigated the same clinical conditions in which we intended to use LFIA. As for weaknesses, we should have performed serological assays in the true negative group and tested freshly obtained COVID-19-negative samples instead of frozen ones dating from before the pandemic. The test of a larger number of vaccinated people at different times after vaccination in addition to different vaccination schedules could also have added value to the study.

CONCLUSION

In conclusion, the LFIA 2019-nCoV IgG/IgM Combo Rapid Test showed a 90% sensitivity, 97.6% specificity and good agreement (Kappa index of 0.677) with reference serological assays (EIA/CLIA). Therefore, this LFIA showed a good technical performance and agreement with reference serological assays (EIA or CLIA) and it can be recommended for use during outpatient follow-up of non-severe cases of COVID-19 and to assess vaccination-induced anti-SARS-CoV-2 IgG antibody production and decline over time. In any case, LFIAs should be confirmed by reference serological assays, whenever possible.