Performance Evaluation of Lateral Flow Assays for Coronavirus Disease-19 Serology.

The coronavirus disease of 2019 (COVID-19) pandemic, caused by infection with the severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2), has undoubtedly resulted in significant morbidities, mortalities, and economic disruptions across the globe. Affordable and scalable tools to monitor the transmission dynamics of the SARS-CoV-2 virus and the longevity of induced antibodies will be paramount to monitor and control the pandemic as multiple waves continue to rage in many countries. Serologic assays detect humoral responses to the virus, to determine seroprevalence in target populations, or induction of antibodies at the individual level following either natural infection or vaccination. With multiple vaccines rolling out globally, serologic assays to detect anti-SARS-CoV-2 antibodies will be important tools to monitor the development of herd immunity. To address this need, serologic lateral flow assays (LFAs), which can be easily implemented for both population surveillance and home use, will be vital to monitor the evolution of the pandemic and inform containment measures. Such assays are particularly important for monitoring the transmission dynamics and durability of immunity generated by natural infections and vaccination, particularly in resource-limited settings. In this review, we discuss considerations for evaluating the accuracy of these LFAs, their suitability for different use cases, and implementation opportunities.

Key points

Lateral flow assays (LFAs) are affordable and easy-to-use serologic assays for SARS-CoV-2.

LFAs are amenable for home testing and community seroprevalence monitoring efforts.

Evaluation of LFAs includes both laboratory assessment of performance characteristics and fitness for implementation.

The utility of LFAs should adapt to vaccine rollouts and emergence of new SARS-CoV-2 variant strains.

Introduction

The coronavirus disease of 2019 (COVID-19), caused by infection with the severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) has undoubtedly been the most disruptive pandemic of the last century. Despite global advances in testing, the true burden of COVID-19 in most countries still remains unclear and is continuously evolving. Reports of prevalence rates thus far have relied on positive SARS-CoV-2 diagnosis using gold standard molecular diagnostics and rapid antigen tests. On the other hand, seroprevalence studies estimate the rates of prior exposure to the virus in each population by gauging the proportion of individuals with antibodies against the virus. , These estimates of the true extent of herd immunity in different communities , could inform public health action and unveil disparities in the susceptibilities of diverse communities to infection with SARS-CoV-2. , As several vaccines are administered globally, monitoring longevity of immune responses induced by vaccination or natural infection with SARS-CoV-2 should inform public health measures to prioritize high-risk populations, such as informal settlements with lower socioeconomic statuses, for vaccinations or to implement containment measures, such as lockdowns and travel restrictions. Serologic lateral flow assays (LFAs) provide an affordable and scalable solution to rapidly monitor seroprevalence and attainment of herd immunity. ,

Here, we review the global context and use cases in which serologic tests are deployed, with a specific focus on LFAs. We review considerations for designing studies to evaluate LFAs, particularly in the context of COVID-19 vaccinations and emerging SARS-CoV-2 variants and provide guidance for implementation of LFAs for both home use and population surveillance.

SARS-CoV-2 Diagnostics

To date, diagnosis has played an important role in monitoring and managing SARS-CoV-2 infections. COVID-19 tests can be broadly classified into molecular diagnostics, antigen-detection tests (rapid tests), and serologic diagnostics, which detect anti-SARS-CoV-2 antibodies. , Molecular and antigen tests detect active viral infections, whereas serologic tests indicate prior exposure to the virus by measuring SARS-CoV-2-specific antibodies. , Gold-standard point-of-care molecular tests currently rely on the detection of ribonucleic acid (RNA) from SARS-CoV-2 by reverse transcriptase-quantitative polymerase chain reactions (RT-qPCR). Rapid antigen tests detect viral antigens, and offer an attractive option for affordable and scalable diagnostics, especially for mass community surveillance.19, 20, 21 However, both molecular and rapid antigen tests only detect active infections, and do not assess prior exposure to SARS-CoV-2, the extent of transmission that had already occurred in a population, or immune status and durability of antibody responses. Serologic tests can be useful epidemiologic tools for monitoring the infection prevalence and herd immunity in diverse populations. As LFAs are cheap and scalable, they are the most amenable form of serologic assays to fulfill these individual and epidemiologic needs.

Need for Validated Serologic Tests for Coronavirus Disease of 2019

Since the beginning of the pandemic, diagnostic tests and serologic assays have flooded the market. Test developers took advantage of the emergency use authorization (EUA) process by the Food and Drug Administration (FDA) locally, and regulatory bodies internationally, including the European Commission, Ministry of Health in Canada, Medicines and Healthcare products Regulatory Agency (MHRA) in the United Kingdom, and the World Health Organization (WHO), to release their products to the market before completing detailed evaluations. Many serologic tests obtained EUA by the FDA, or equivalent regulatory approvals, for example, interim order (IO) authorizations or Conformité Européenne (CE) marks, with evaluations that were often based on samples from a small number of patients, which were not always representative of the entire susceptible population (e.g., symptomatic patients only). Therefore, these evaluations limited the reliability and generalizability of tests to estimate the true extent of SARS-CoV-2 transmission in diverse community settings. Hence, standardized protocols for rigorous evaluations of these tests by manufacturer-independent third parties became crucial to determine their accuracy and usability in an unbiased way. Importantly, the increased reliance on antibody tests as “immunity passports” demands their careful evaluation, as well as community education on the interpretation of the test results, to prevent premature assumptions of immunity against SARS-CoV-2.27, 28, 29

World Health Organization Guidance on Serologic Testing

Since mid-2020, the WHO has advocated for countrywide serosurveys to determine the extent of SARS-CoV-2 spread globally. To guide this process, the WHO developed an interim guidance policy document stating that serologic assays would be crucial to support serosurveillance efforts aimed at estimating transmission to inform public health responses. However, in this document, the WHO cautioned against using serologic assays to determine antibody titers as surrogates for protective immunity, or as tools for contact tracing or diagnosis of active infections. ,

To support country-wide serosurveillance efforts, the WHO partnered with the Centers for Disease Control (CDC), the Foundation for Innovative New Diagnostics (FINDdx), African Society for Laboratory Medicine (ASLM), and others, to evaluate and roll out COVID-19 diagnostics. As a result, FIND created a centralized repository of available SARS-CoV-2 serologic assays, which measures both performance accuracy and feasibility for scale-up in low and middle-income countries. This effort resulted in standardized protocols to evaluate the accuracy and suitability of serologic assays to achieve the following: triaging suspected patients with COVID-19, assessing recovery of convalescent patients with COVID-19, and implementation of these assays in broader seroprevalence initiatives to inform public health actions, such as prioritizing regions of high transmission, for COVID-19 vaccination. Easy-to-use serologic assays, such as LFAs, which are also affordable and scalable, will be key to decentralizing access to these tests.

Seroprevalence of Severe Acute Respiratory Syndrome Coronavirus 2 Globally

Several reports conducted in different populations with varied demographics showed a wide range of seroprevalence estimates of antibodies against SARS-CoV-2, as highlighted herein. In Wuhan, China, a study on samples from 18,712 asymptomatic participants collected between January and February 2020 found a seroprevalence of 3%-8% for IgG titers, whereas another study in the same area from March to April 2020 described rates of 0.3% in 9442 community resident men. In the United States, one study had 4675 outpatients, another 177,919 community samples, and in the United Kingdom, 365,000 community samples yielded rates that varied from 0% to 20%. In a Spanish teaching hospital in Madrid, seroprevalence estimates ranged from 25% to 33% among 2919 health care workers. In a slum in India, the seroprevalence was as high as 57.9% in 470 individuals. In Pakistan, the estimates in Karachi ranged from 8.7% to 15.1% for 3005 community samples.

In sub-Saharan Africa (SSA), most economies adopted systematic lockdowns, social distancing, and donning of masks to reduce transmission. , As a result, SSA countries saw overall lower rates of severe disease in the early stages of the pandemic.43, 44, 45 However, following the economic pressure to reopen and relaxation of social distancing measures, infection rates have risen, with seroprevalence estimates in Kenya between 5% for 3174 blood donor samples and 50% for 196 antenatal clinical samples, 12.3% among 500 asymptomatic health care workers in Malawi, 3% of 99 asymptomatic individuals sampled in Ethiopia, 45% in 133 health care workers in Nigeria, Guinea Bissau 18% in 140 health care workers, and 38.5% among 2214 individuals in households in South Sudan. However, in most cases, these estimates are based on studies of target groups, such as health care workers, truck drivers, and small populations of less than 3,000 individuals. , Therefore, the number of participants in SARS-CoV-2 serosurveys in low- and middle-income countries has been generally lower than those of wealthier counterparts. The true extent of COVID-19 spread, particularly in rural settings with little active case finding and surveillance remains undetermined, especially whereby social distancing measures are more difficult to enforce. These seroprevalence studies collectively demonstrate that SARS-CoV-2 spread, estimated by molecular test positivity rates, severely underestimate true transmission rates. , Therefore, there is a need for more systematic sampling to determine the evolving seroprevalence of COVID-19 across various communities.

Types of Serologic Tests for Severe Acute Respiratory Syndrome Coronavirus 2

Serologic tests that detect antibodies against SARS-CoV-2 include enzyme-linked immunosorbent assays (ELISAs), chemiluminescence assays, and LFAs.56, 57, 58, 59 ELISAs are plate-based assays to detect an analyte, such as an antibody against a SARS-CoV-2 antigen. Several commercial and noncommercial tests have been developed to measure antibodies to SARS-CoV-2, which include both ELISA and chemiluminescence immunoassays. These assays generally target antibodies against the receptor-binding domain (RBD), spike (S), or nucleocapsid (N) proteins. , The commercially developed ELISA EUROIMMUN assay detects IgA/IgG antibodies that bind to spike antigens.62, 63, 64 This ELISA has been evaluated using 103 clinical samples, whereby they observed a sensitivity of 21.6% within a week of symptoms onset, 55.1% on the second week and 89.5% after 2 weeks for IgG, with an overall specificity of 96.1%. Similar results were obtained for NovaLisa ELISA kit at 2-weeks post-infection for IgG (sensitivity: 94.9%, specificity: 96.2%), IgM (sensitivity: 89.7%, specificity: 98.7%), and IgA (sensitivity: 48.7%, specificity: 98.7%) in 287 patients. The Platelia ELISA kit yielded 97.4% and 94.9% sensitivity and specificity, respectively, using the same 287 patient samples. Several in-house noncommercial ELISAs have also been developed. A recent study evaluated inactivated SARS-CoV-2 virus antigen by ELISA using 513 clinical samples at 2-weeks postinfection and found that it demonstrated 92.3% sensitivity and 97.9% specificity. An alternative indirect ELISA method used S protein to measure IgG to SARS-CoV-2 in 418 healthy persons, patients with COVID-19 and health care workers, yielding 100% sensitivity and 98.4% specificity, with no cross-reactivity to other human coronaviruses. In another study, 30 inpatients with SARS-CoV-2-positive were subdivided into severe and mild, based on whether they needed intensive care or not, respectively, and a total of 151 samples were collected. In these samples, evaluation of IgG titers of RBD, S, and N proteins showed that antibodies against RBD and N proteins more accurately reflected disease status, and were higher in samples from inpatients with severe than mild COVID-19. For chemiluminescent assays, the sensitivity was 96% in 1338 clinical samples collected at a median of 47 days. Although ELISAs and chemiluminescent assays can quantify antibodies, they remain primarily a research tool, particularly in resource-limited areas, since they require expensive equipment, trained personnel, and central laboratories that preclude their use in decentralized community testing programs.

Serologic LFAs are best suited as point-of-care tests for assessing prior exposure to SARS-CoV-2. , LFAs were thus developed as tools to detect SARS-CoV-2-specific antibodies in patient sera, plasma, or whole blood. Earlier in the pandemic, serologic LFAs were proposed as alternatives to the expensive and time-intensive RT-qPCR, to complement COVID-19 diagnosis. , However, molecular and rapid antigen tests remain the gold standard for diagnosing active infection. LFAs are simple devices that usually show a qualitative band to indicate the presence of antibodies targeting different SARS-CoV-2 antigens, and usually a second control band to indicate the validity of the test. SARS-CoV-2 serologic LFAs are effective in detecting antibodies between 15 and 30 days after the onset of disease. , , , However, data on the sensitivity of the LFAs more than 30-days postinfection are limited. There are currently more than 448 tests available or in development (SARS-CoV-2 diagnostic pipeline - FIND (finddx.org)). The FINDdx repository continues to be updated with new SARS-CoV-2 serologic tests and their performance characteristics, as evaluated by multiple partner institutions, (Fig. 1 , Table 1 ). The performance of these assays relies on the ability of SARS-CoV-2-infected individuals to mount antibodies against the virus as described later in discussion.

Fig. 1

Regulatory Authorizations for COVID-19 Serology LFAs: The percentage of serology lateral flow kits (x-axis) that have been approved by different regulatory bodies across the world (y-axis). Others§: combination of regulatory authorities that have approved less than 1% of the kits (n = 269), including the Philippines FDA and Korea Export (0.7% each), COFEPRIS (Comisión Federal para la Protección contra Riesgos Sanitarios; Mexico), In Vitro Diagnostics class D (IVD-D), Ministry of Health, Labor and Welfare-In Vitro Diagnostics (MHLW-IVD), Medicines and Healthcare Products Regulatory Agency (MHRA; UK), Medical Device Authority (MDA; Malaysia), Roszdravnadzor (RZN; Russia), Swiss Medic and Taiwan FDA (0.4% each). CE-IVD: Conformité Européene In vitro diagnostics (approval by the EU). RUO: Research Use Only. EUA: Emergency Use Authorization. Data is accessed from the Foundation for Innovative New Diagnostics (FINDdx).

Table 1

COVID-19 Lateral Flow Serology Assays Reported by Foundation for Innovative New Diagnostics (FIND), accessed on 02 April 2021

Feature/Characteristic	Total: n (%)
Target antibody	269 (100%)
IgG	269 (100%)
IgM	269 (100%)
Type of sample to test	269 (100%)
Serum	269 (100%)
Plasma	269 (100%)
Whole Blood	269 (100%)
Phase of development	269 (100%)
Commercialized	250 (92.9%)
In development	19 (7.1%)
Use authorization	269 (100%)
Emergency Use Authorization	29 (10.8%)
Research Use Only	29 (10.8%)
No restricted use	211 (78.4%)

Induction of Antibodies Against Severe Acute Respiratory Syndrome Coronavirus 2

Innate and adaptive immunity play an important role in controlling SARS-CoV-2 infection. Adaptive immunity creates durable memory responses to reinfection with SARS-CoV-2, through T cell-mediated cellular immunity,75, 76, 77 and B cell-mediated humoral immunity. , B cells differentiate into plasma cells, which produce antibodies that target viral antigens. Binding of antibodies to the virus can neutralize it and block its replication in host cells, which forms the basis for proposed antibody therapeutics against SARS-CoV-2. , Antibodies against SARS-CoV-2 include multiple isotypes: immunoglobulin-M (IgM), IgG, IgA, which start to appear in patients with COVID-19 around 7 to 14 days post-infection and persist for weeks after virus clearance. The most detected antibodies recognize either the internal N protein or the highly immunogenic external S protein. The RBD is the component of the spike protein, which binds to the human angiotensin-converting enzyme-2 (ACE2) receptor to enter and replicate in the host cell. , Therefore, neutralizing antibodies against the RBD of SARS-CoV-2 are particularly important to block entry and replication in host cells.

Given the integral role of the S protein and RBD in facilitating viral entry, these antigens form the basis of many immunoassays described to date , and inform rational COVID-19 vaccine design. Recent data from immunoassays based on the SARS-CoV-2 nucleocapsid protein show high sensitivity. , , , However, the higher sequence homology of the SARS-CoV-2 N protein to other coronaviruses, than the S protein, could increase the possibility of cross-reactivity against N proteins from related coronaviruses.92, 93, 94

IgM antibodies are usually the first humoral response on SARS-CoV-2 infection. Travel requirements in China have required a negative IgM test to permit travel (http://www.china-embassy.org/eng/notices/t1841416.htm). However, using IgM as an indicator of early infection is still likely to miss individuals within 5 days of exposure. Serum levels of SARS-CoV-2-specific IgM antibodies decrease precipitously over time, than IgG response, as shown in longitudinal serosurveys of households in Wuhan, China, and longitudinal studies of convalescent patients after discharge. In contrast, SARS-CoV-2 RBD-specific IgG antibodies were durable in convalescent patients with COVID-19 and showed minimal cross-reactivity against other widely circulating coronaviruses (HKU1, 229E, OC43, NL63).

Utility of Lateral Flow Assays for Coronavirus Disease of 2019 Antibody Testing

LFAs are effective point-of-care tools to detect immune responses to widely transmitted infections like SARS-CoV-2. , Serologic LFAs measure pathogen-specific antibodies in accessible biological specimens using simple platforms, whereby gold or other material-based nanoparticles are often used to label secondary antibodies. , LFAs are ideal for mass population surveillance for antibody responses, induced by either natural infection or vaccination, because they are cost-effective, portable, rapid, can be designed to measure more than one antibody isotype in the same sample, and do not require sophisticated equipment to produce results. , Furthermore, LFAs are easy-to-use and do not require specialized training for implementation. Additional developments to improve their sensitivity include the use of smartphone apps to detect positive LFA results, which could enable aggregation of data in centralized databases to report disease exposure and inform public health intervention. , Therefore, LFAs are useful candidates for population serosurveillance and to monitor longevity of vaccine and SARS-CoV-2-induced antibodies to understand the real extent of herd immunity in a population. , However, before implementation of LFAs, the performance of these assays must be systematically evaluated (Table 2 ), including the impact of factors such as temperature and humidity. , ,

Table 2

Key considerations for LFA evaluation studies

	Issues and Questions to Address in the Evaluation
Target population	• Will the study include both symptomatic and asymptomatic individuals? • Inclusion of vulnerable and high-risk populations (e.g., immunocompromised individuals and those with comorbidities)? • Diverse ethnic and socio-economic participants • Different age groups (children and the elderly) • Implementation in occupational settings: for example, for testing healthcare workers and education staff • Inclusion of travelers (e.g., for border crossing restrictions)
Sampling scheme	• Cross-sectional schemes for direct evaluation of LFA performance characteristics (e.g., sensitivity and specificity) • Longitudinal schemes particularly of highly exposed individuals to allow the analysis of seroconversion, durability of vaccine, and infection-induced antibody responses
Type of sample	• Are samples easy to collect? (e.g., finger prick whole blood, urine, saliva)? Invasiveness? • Does the sample collection require trained personnel? • Access to storage and transport conditions to preserve the sample quality • Infection control: Does the sample expose the “collector” to SARS-CoV-2 or other pathogens? • Can the end-user collect the samples themselves?
Study case definition	Confirmed SARS-CoV-2 exposure and time between confirmed RT-qPCR test and sample collection for serology.
Study control definition	1. Historic pre-pandemic samples 2. Populations that are routinely tested: For example, healthcare workers without any documented positive test
Performance characteristics	Test sensitivityTest specificityPositive predictive values (PPV)Negative predictive values (NPV)
Prevalence in the target population	• The impact of prevalence on PPV and NPV? • Would the test overestimate or underestimate the test results?
Specificity controls	• Will the evaluation determine analytical specificity by measuring cross-reactivity against other seasonal coronaviruses: HKU1, OC43, NL63, and 229E, or coronaviruses from previous outbreaks: SARS-CoV and MERS?
Reference standard	• Will the evaluation include reference serology standards: for example, pooled samples from known positives with high, mid, and low antibody titers.
Target antigen	• What is the target antigen in the LFA?1. Nucleocapsid 2. Spike 3. Other antigens: for example, RBD
Isotype of interest	• Will the test target IgM, IgG, or IgA isotypes? • What is the definition of a positive and negative test result if multiple antibody isotypes are included?
Conservation of antigen	• Is the target antigen from a conserved region of the SARS-CoV-2 genomic sequence? • How similar is the antigen to other coronaviruses to allow discrimination of SARS-CoV-2? • Is the LFA performance impacted by mutations in the SARS-CoV-2 antigens?
Variants	• What autologous SARS-CoV-2 strain was the “case” infected with? • Is the LFA intended to specifically detect SARS-CoV-2 variants?
Limit of detection	• What is the analytical sensitivity of the LFA: at which antibody concentration does the LFA lose sensitivity?
Quantitative utility	• Is the kit used for qualitative test results only? • Does the band intensity correlate with antibody titers?
Vaccination Status	• Is this LFA intended for a vaccinated population?
Use cases	• Individual vs population? • Vaccinated vs unvaccinated? • Epidemiologic understanding of seroprevalence and transmission? • Durability of responses?
Financial effectiveness	• How affordable is the test? • Will the cost allow the LFA to be subsidized by a healthcare system or individuals will cover the cost? • How does the cost impact the community uptake?
Utility of implementation	• Does the LFA fulfill a critical public health implementation need? • Is the LFA the most suitable testing modality for the use case? • Do you foresee barriers to social acceptability to implementation?
Supply chain (manufacturer)	• Can manufacturing be scaled up? • Who is funding the manufacturing? • What is the availability of consumables in the region? • Will the LFA kits require assembly in the user laboratories, or is the assembly centralized? • Are the locally available consumables compatible with the LFA?
Impact on clinical decision making	• Does the result impact clinical practice? • Is there evidence supporting the implementation of the LFA in clinical care settings?
Provider/health care system acceptance	• Are the LFA vendor and/or developer considered credible for local public health authorities?
Utility for local public health systems?	• What is the demand landscape for the LFA? • Does the LFA inform social distancing guidelines? • Can the evaluation protocol determine fitness for implementation? • Is the LFA high on the priority list for tools in the fight against the COVID-19 pandemic? • What are the cold chain requirements for storage and distribution? • Can the LFA adapt to different temperatures/climates?
Feasibility and adoption	• Is there a political will to adopt LFAs? • What is the available infrastructure for rolling out LFAs? • Are they fit for the proposed use cases? • What is the balance between feasibility, practicality, and actual fit that ensure the utility of adoption? • Will the evaluation assess adoption-uptake (decision to use the LFA and trialability (ability to attract the utilization and ease of use-for direct-to-consumer testing)160 , 161?

What is the target antigen in the LFA?

Will the evaluation assess adoption-uptake (decision to use the LFA and trialability (ability to attract the utilization and ease of use-for direct-to-consumer testing) , ?

Band Strength and Sensitivity of Lateral Flow Assays

As LFAs are designed to be qualitative tests, an important question is whether the band strength (that is, the color intensity of the bands) should be evaluated. The interpretation of band strength can be subjective, but perhaps can be improved by incorporating smartphone apps, as conducted recently for a rapid antigen test. Variation in band strength across multiple samples raises the question of whether band strength correlates with titers of antibody titers. Antibody levels, determined by optical density (OD) ratios, were initially low following symptom onset, then increased over time whereby IgM, IgG, and IgA levels correlated with clinical disease severity. A rapid decay of anti-SARS-CoV-2 antibodies, particularly for patients with mild symptoms implies that the band intensity could serve as a biomarker for disease severity. As suboptimal antibody titers may promote pathology through antibody dependent-enhancement, correlating LFA band strength with symptom severity could provide a use case for LFAs to inform clinical management. Because LFAs are best suited for population surveillance, the importance of the analytical sensitivity, also known as the limit of detection (LoD) of LFA, that is, the lowest antibody titers in each sample to give a positive LFA result, cannot be understated. High analytical sensitivity is important in cases that present late with milder symptoms and in patients suspected of COVID-19 despite a negative SARS-CoV-2 RT-qPCR test result.

Use Cases of Serologic Lateral Flow Assays

LFAs have the potential of deployment outside of clinical care settings due to their affordability and ease of use. The number of people infected with SARS-CoV-2 is known to be underestimated, especially in low- and middle-income countries, due to the high rate of unreported and asymptomatic cases which can spread the infection within the community. The availability of molecular testing and public health restrictions that follow especially for the informal labor sector and rural communities, pose real barriers to testing. Thus, LFAs provide a cheap and scalable alternative to estimate the spread in diverse communities. The presence of anti-SARS-CoV-2 antibodies can identify presumably immune individuals and could thus serve as a tool to release individuals from isolation or lockdown. However, it is important to note that LFAs do not quantify antibody titers or their neutralizing potential. Hence, LFAs are not ideal surrogates for herd immunity, , but are better suited for estimating SARS-CoV-2 transmission in diverse communities.

On the individual level, LFAs can complement efforts for retrospective diagnosis of presumably exposed individuals. Positive LFA results can confirm exposure to a SARS-CoV-2-infected individual, and so LFAs can complement contact-tracing tools, but cannot replace molecular or antigen tests. LFAs are also ideal as direct-to-consumer at-home serologic tests that empower individuals to test for anti-SARS-CoV-2 antibodies. The Food and Drug Administration (FDA) has already approved several LFAs, such as Cellex qSARS-CoV-2 IgG/IgM Rapid Test and others for home use. Interestingly, Cellex partnered with Gauss to launch a parallel rapid SARS-CoV-2 antigen test, which was the first to be approved by the FDA for home use. It is very likely that serologic LFAs will follow suit. Although home use of serologic tests can be a vital instrument in empowering users, the risk of result misinterpretation is very high, and may result in premature behavioral changes that could increase the risk of SARS-CoV-2 transmission. More dangerously, ineffective immunity has the chance of exerting selection pressure to increase spontaneous mutations of SARS-CoV-2, and transmission of SARS-CoV-2 variants of conern. , A positive result is prone to be false when the prevalence of the disease is low, or if the specificity of the assay is suboptimal for reasons such as cross-reactivity with related coronaviruses. Therefore, deployment of LFAs for home-use requires the inclusion of educational materials that facilitate interpretation as explained later in discussion.

Lateral Flow Assays to Distinguish Antibodies Induced by Infection or Vaccination

The identity of target SARS-CoV-2 antigens in the LFAs is critical. , , , Some LFA kits target the N protein, others the RBD and some the S protein, which is displayed all around the surface of the virus. Additionally, the N-terminal domain of the N protein is highly conserved in all beta-coronaviruses and may cause false-positive results and/or fail to detect true early sensitization. Several widely used SARS-CoV-2 vaccines use the S antigen, including mRNA-1273 by Moderna, AZD1222 by AstraZeneca, the Ad26.COV2.S from Johnson and Johnson, or more specifically the RBD of BTN162b2 by Pfizer-BioNTech. Therefore, in populations that receive Spike-based vaccines, LFAs targeting the N and S antigens can be used to distinguish natural SARS-CoV-2 infection only, or vaccine and infection-induced antibodies, respectively (see Table 2). Vaccines that are based on the complete inactivated virus, such as BBV152/COVAXIN or N antigen only will not allow this use case. The variety of antigenic targets for the LFAs, as well as more complex serologic assays, allow for this application. , LFAs targeting the S protein only include COVID-19 IgM/IgG tests from: Camtech, Oranoxis, and Ozo, and N-specific LFAs include CareHealth, KHB, Phamatech, and Ray Biotech, whereas several LFAs target both and would not be suitable for this use case. , The tests overall show high sensitivity and specificity for IgG antibodies in samples collected 10 days or more following a positive SARS-CoV-2 RT-qPCR result. The sensitivity was generally higher for IgG than IgM, which motivates for using IgG LFA readouts for serosurveys or home use. Overall, FINDdx reports that most LFAs target the N antigen (see Table 1), making them more appropriate for testing breakthrough SARS-CoV-2 infections in individuals who received S-based vaccines.

Study Design to Evaluate Serologic Lateral Flow Assays

Decentralized administration of serologic tests raises important concerns about the accuracy of these platforms, subsequent interpretation of test results by both providers and end-users, and their suitability for different implementation scenarios. These considerations are summarized in Table 2.

Study population

It is important that cohorts used for LFA evaluations reflect the characteristics of the intended populations for implementation. For example, if the intended application is testing the longevity of vaccine-induced response, the study design should include control pre-vaccination samples, proximal post-vaccination samples to assess seroconversion (e.g., 1- and 2-week post-vaccination) and remote samples (e.g., 6 months or 1-year post-vaccination). In this situation, quantitative serologic assays such as ELISAs should be used as a reference to benchmark the LFA performance. In contrast, if LFAs are intended to test the induction of antibodies in specific subgroups, such as HIV-positive or immunocompromised individuals, the cohorts need to include individuals with these clinical characteristics and controls. In contrast, evaluation of LFA’s analytical specificity against other related viruses will require inclusion of populations with a known history of exposure to other coronaviruses, such as historical samples from convalescent individuals from the first SARS-CoV epidemic in 2003, as SARS-CoV and SARS-CoV-2 share 76.5% amino acid sequence similarity, and share tropism for the ACE2 receptors for entry into mammalian cells.

Symptomatic SARS-CoV-2 infection increases the pretest probability that someone was exposed to SARS-CoV-2. However, as asymptomatic SARS-CoV-2 carriers are estimated to comprise at least 40% to 45% of all SARS-CoV-2 infected individuals, evaluation studies should include both symptomatic, as well as asymptomatic individuals, with a positive SARS-CoV-2 result on a highly sensitive and specific molecular test. These studies were difficult at the beginning of the pandemic as testing was generally restricted to hospitalized and severely ill patients with COVID-19. However, with expanded access to community testing using sensitive RT-qPCR tests, inclusion of SARS-CoV-2-positive individuals with mild or no symptoms for sampling to evaluate serologic LFAs is important. The cases should ideally span diverse demographics, clinical presentations (from asymptomatic, mildly, and severely symptomatic to those in intensive care), and comorbidities, which may compromise seroconversion following SARS-CoV-2 infection, especially that some of these populations may be at even higher risk of SARS-CoV-2 infection and COVID-19 disease. , The parallel uninfected controls should preferably be sampled from the same population as cases to reduce systematic biases in the evaluation.

The selection of SARS-CoV-2 unexposed controls is more difficult considering the wide-spread transmission of the virus and high seroprevalence globally. , The WHO only declared the pandemic a global emergency in March of 2020, whereas seroprevalence at these months indicated higher rates of infections that reflect earlier transmission. Several communities outside of Wuhan already documented seropositive patients in January and February of 2020, which may be due to cross-reactivity to other related coronaviruses, or real transmission of SARS-CoV-2 before molecular testing was widely implemented. Therefore, controls should be collected from earlier samples, even before October or November of 2019, to rule out unreported SARS-CoV-2 infection. One possible way to avoid including SARS-CoV-2 exposed individuals as negative controls is to use prepandemic bio-banked samples. Alternatively, individuals who are routinely tested for SARS-CoV-2, who have never had a positive test result would be the suitable “matched” uninfected group. This prospective evaluation of LFA effectiveness is especially critical as new variants circulate and may compromise the performance accuracy of the LFAs under evaluation.134, 135, 136 However, in situations whereby controls are enrolled from the same SARS-CoV-2 exposed communities, repeat testing with highly sensitive molecular tests as well as complementary serologic tests that are more sensitive, would be important to rule out prior exposure to the virus. This is particularly critical in prospective studies whereby LFAs are evaluated using freshly collected samples, such as whole blood from finger pricks or saliva.

Sampling schemes

In addition to the choice of the population of interest for LFA evaluation, the samples can either be collected cross-sectionally or longitudinally or using a hybrid of the 2 designs. , As above, the sampling scheme should address the intended use case. Evaluating LFAs to measure the durability of vaccine-induced antibodies will require longitudinal sampling, whereas cross-sectional samples from confirmed SARS-CoV-2 exposed and unexposed individuals would suffice for the evaluation of LFAs for implementation in seroprevalence studies. For positive cases, samples should be collected at least 10 days to 3 or more weeks after symptom onset, for those with clear COVID-19 symptoms, to allow sufficient time for seroconversion. Asymptomatic study participants should be diagnosed by positive RT-qPCR results using a sensitive molecular test. In general, very low sensitivity and higher variability in accuracy were reported for LFAs measuring IgM and IgG from samples collected within a week postsymptoms onset. This is consistent with the often-delayed seroconversion in patients with COVID-19 which occurs around day 11 to 19 postsymptoms onset. Consequently, additional effort is required to improve the sensitivity of these assays for early detection of antibodies following symptoms onset.

Sample size determination

One of the biggest limitations with the initial FDA EUA process for the evaluation of COVID-19 diagnostics was the small number of clinical samples needed from confirmed SARS-CoV-2-infected individuals. Initial evaluations included fewer than 100 SARS-CoV-2 positive cases, which would only detect extreme differences in the accuracy of diagnostic platforms. , This is particularly critical whereby the prevalence of SARS-CoV-2 infections in various communities is still relatively low, as lower prevalence reduces the positive predictive value (PPV) of these tests. Sample sizes to ensure adequate power are inversely correlated with the effect size differences to be detected at a prespecified significance level. Consequently, a larger sample size will be required to compare the performance of 2 LFAs with close sensitivity levels (ie, small effect size), than comparing 2 LFAs with poor and excellent sensitivities (ie, large effect size). Considering the initially limited sample sizes for LFA evaluations, it is critical to expand sample sizes to validate the performance of LFAs to increase the confidence of assay performance before rollout.

Sample choice

Unlike nasopharyngeal swab samples that are hard to collect and have variable quality, serology assays rely on serum and/or plasma samples collected from whole blood that is drawn by widely standardized procedures. Therefore, it is conceivable that some samples tested by RT-qPCR turn negative or indeterminate because of the quality of the sample tested or RNA degradation, leading to false-negative classifications. Serology is less likely to be impacted by sample quality. Furthermore, saliva samples have been evaluated for serology, particularly for the induction of IgA responses, but they are not the norm for LFA evaluations. As the success of a diagnostic test depends on the quality of the biological specimen tested, serologic assays are appealing alternative tests because of the reliability of samples needed.

Performance characteristics

To evaluate the accuracy of LFAs, several performance metrics need to be assessed based on the intended use cases. These characteristics include sensitivity, specificity, PPV or precision, and negative predictive values (NPVs), inter and intra-operator reproducibility and finally, analytical sensitivity, also known as LoD.

Sensitivity refers to the proportion of positive cases, defined by a gold standard test like a SARS-CoV-2 RT-qPCR, that are detected accurately by the test. A highly sensitive test will detect most cases, usually at the expense of inaccurately over-diagnosing uninfected individuals as false positives, and hence can be used to rule out disease when negative. PCR tests generally fall in this category, as they are prone to detect very low concentrations of residual SARS-CoV-2 RNA molecules weeks after infection. Highly sensitive PCR tests may also detect contaminating templates from the environment or in the test reagents, as reported for the Cepheid Xpert Xpress SARS-CoV-2 test. In contrast, the sensitivity of serologic tests is confounded by other factors, including time since symptom onset, the immunocompetence of study participants, the reactivity of the antibodies from a given sample to the antigen, and the emerging variants of SARS-CoV-2. The sensitivity of some LFAs was evaluated in samples from hospitalized patients with COVID-19 in a case-control study design, which likely overestimated the sensitivity of these compared with the general population, leading to spectrum bias, that is, reporting different accuracies in the evaluation cohort and target population.

Specificity is the proportion of SARS-CoV-2-negative samples, which are correctly detected as negative by the LFA. The specificity of LFAs is expected to be generally high and close to 100%. , , , , False-positives results could be caused by SARS-CoV-2 LFA cross-reactivity of antibodies against other circulating coronaviruses, or inaccurate definitions of SARS-CoV-2 negative samples with a false-negative RT-qPCR or rapid antigen test result.

PPV refers to the proportion of positive tests that are likely to correspond to a SARS-CoV-2-positive sample. Conversely, NPV is the proportion of negative tests that are likely to come from true SARS-CoV-2-negative samples. It is important to note that PPV and NPV are a function of both the accuracy of the test and the seroprevalence in each population. Low prevalence penalizes the PPV of diagnostic tests, whereby a positive result is more likely to be a false positive the lower the infection rates are in a given population. Thus, test outcomes, especially in nonhealth care settings have to be interpreted with caution and with an understanding of the community transmission dynamics and test limitations.

Analytical sensitivity or LoD refers to the minimum SARS-CoV-2-specific antibody titers that are detectable by the LFA. Quantitative platforms such as ELISAs can be used to establish the LoD for LFAs, by adding titrated amounts of serologic standards with known antibody titers, and running them concurrently on the ELISA, or other quantitative platforms, and the LFAs under evaluation. Since LFAs are intended to be qualitative, it is worth considering whether a positive band needs to be detected by the naked eye, or whether additional smartphone apps or instruments can detect faint bands that correspond to low antibody concentrations in the sample. The LoD of LFAs is higher than known sensitive quantitative methods such as the ultrasensitive Single molecule array (SIMOA) platforms. , , However, a high LoD reduces the chance of misusing LFAs to ascribe immunity passports to individuals with low antibody titers to conservatively prevent overestimation of seroprevalence, and herd immunity. , Finally, testing the reproducibility of test results run by the same operator multiple times (intraoperator reproducibility), or between different operators (interoperator reproducibility), as well as reproducibility across different reagent lots would instill confidence in the reliability of the manufacturing quality of the LFAs.

Impact of emerging variants on the performance of lateral flow assays

The emergence of SARS-CoV-2 variants is an important consideration in the evaluation of SARS-CoV-2 diagnostic tests, since SARS-CoV-2 antigenic drift may reduce the sensitivity of these tests. So far, at least 3 known variants of SARS-CoV-2 have been described that are characterized by novel genetic mutations. These include B.1.1.7, B.1.351, and P.1 and P.2. B.1.1.7 has 23 mutations located in the open reading frame (ORF)1ab, ORF8, and N regions. Out of the 23 mutations, 17 are of concern whereby 13 are nonsynonymous, resulting in amino acid substitutions, and 4 are deletions. B.1.351 has 21 mutations including 9 amino acid changes in the S gene. The other mutations are in ORF1ab, ORF3a, N, and E genes. The P.1 variant has 10 mutations in the S gene, with additional mutations in ORF1ab, ORF8, and N genes. These emerging SARS-CoV-2 variant strains have compromised the ability of naturally induced antibodies to neutralize SARS-CoV-2. For example, a new SARS-CoV-2 variant, 501Y.V2, substantially or completely escapes from neutralizing antibodies in COVID-19 convalescent plasma.

Variants of SARS-CoV-2 with the D614G mutation in the spike (S) protein that increases receptor-binding avidity have also been reported globally (Table 3 ). The B.1.351 and B.1.1.28 (P.1) variants are known to affect the performance of real-time RT-qPCR tests. Patients infected with the H69del/V70del SARS-CoV-2 variant have an increased Spike (S) protein gene amplification drop-out rate, which leads to RT-qPCR target failure.

Table 3

Summary of emerging SARS-CoV-2 variants

	Variant Designation	Characteristic Mutations (Protein: Mutation) and Location
1	B.1.1.7 (20I/501Y.V1)	ORF1ab: T1001I, A1708D, I2230 T, del3675–3677 SGF
		S: del69–70 HV, del144Y, N501Y, A570D, D614G, P681H, T761I, S982A, D1118H
		ORF8: Q27stop, R52I, Y73C N: D3L, S235F
2	B.1.351 (20H/501Y.V2)	ORF1ab: K1655N
		E: P71L
		N: T205I
		S: K417N, E484K, N501Y, D614G, A701V
3	P.1 (20 J/501Y.V3)	ORF1ab: F681L, I760T, S1188L, K1795Q, del3675–3677 SGF, E5662D
		S: L18F, T20N, P26S, D138Y, R190S, K417T, E484K, N501Y, D614G, H655Y, T1027I
		ORF3a: C174G ORF8: E92K ORF9: Q77E ORF14: V49L N: P80R
		ORF1ab: F681L, I760T, S1188L, K1795Q, del3675–3677SGF, E5662D
		S: L18F, T20N, P26S, D138Y, R190S, K417T, E484K, N501Y, D614G, H655Y, T1027I
		ORF3a: C174G ORF8: E92K ORF9: Q77E ORF14: V49L N: P80R
		ORF1ab: F681L, I760T, S1188L, K1795Q, del3675–3677SGF, E5662D

The mature SARS-CoV-2 Spike trimer is composed of the exterior S1 and transmembrane S2 subunits. The S1 subunit uses the RBD to interact with the ACE2 receptor, whereas the S2 subunit governs the fusion between the viral and cellular membranes. Spike is considered the major target of the cellular and humoral responses against SARS-CoV-2 in natural infection. , Of all SARS-CoV-2 variants, the D614G mutant accounts for 75.7% of all circulating strains and is associated with severe clinical presentation. SARS-CoV-2 Spike D614G had a more severe impact on antibody binding than the wild-type strain. Studies using monoclonal antibodies (mAbs) have shown that V483A in the receptor-binding domain has a mutation frequency of more than 0.1%. It showed decreased reactivity to the 2 mAbs (P2B-2F6 and X593) and the A475V is significantly resistant to several neutralizing antibodies. Strains with combined D614G and I472V mutations have shown increased infectivity and more resistance to neutralizing antibodies. Some variants, including N439K, L452R, A475V, V483A, F490L, and Y508H, do have decreased sensitivity to neutralizing mAbs.

Most LFAs target the C-terminus of viral nucleocapsid (N) protein. B.1.1.7 mutations on the N gene are located at the N-terminus. Hence, this variant is unlikely to show an impact on LFA performance as the epitope for antibody recognition likely remained intact despite the mutation. Other LFAs target S protein coded by S gene, which recent data show has a majority of mutations, including spike mutation E484K that affect antibody response, and hence could affect the LFA performance. Collectively, whether these mutations reduce the sensitivity of LFAs needs to be systematically evaluated (Table 4 ).

Table 4

The possible consequences of emerging SARS-CoV-2 mutations on LFA performance

Variant Designation	Impact on Performance of Rapid Lateral Flow Assays
B.1.1.7 (501Y.V1)	The N gene mutations in this variant are located at the N-terminal. An assessment by Public Health England found that five SARS-CoV-2 rapid antigen tests evaluated were all able to successfully detect the variant.162 No evaluations were performed for serology LFAs.
B.1.351 (501Y.V2)	To date, no evaluation studies have been carried out to confirm that performance of serology LFAs is not affected, but no major performance deficits are anticipated.
P.1 (501Y.V3) and P.2	To date, no evaluation studies have been carried out to confirm that test performance is not affected, but no major performance deficits are anticipated.

Prospects for Next-Generation Lateral Flow Assays to Detect Severe Acute Respiratory Syndrome Coronavirus 2 Variants

To evaluate the detection of SARS-CoV-2 variant-specific antibodies, the mutated antigen from the variants should be included in the kit, especially when amino acid changes in the SARS-CoV-2 antigens are sufficient to alter antibody binding. Hence, recombinant antigens reflecting the pseudotypes of the emerging variants should be incorporated in the next generations of LFAs. Subsequent evaluation efforts for LFAs should perhaps analyze both the conserved and mutated antigens to distinguish whether an infection has occurred and whether antibodies were generated in response to a mutant strain. It is important to note that the difference in antigenicity may be too subtle to influence the detection of antibody responses. However, as new SARS-CoV-2 variants are still emerging, it is imperative to iteratively develop and improve LFA assays to detect variant-specific serologic responses.

Evaluation of Implementation Feasibility and Fitness for Use

Following the evaluation of the accuracy of LFAs, they need to be assessed for implementation effectiveness and fitness for use. , Effectiveness reflects whether the LFA is fit for implementation in the intended population and settings by evaluating relevant factors, including required storage conditions and affordability, particularly in resource-limited countries and communities. For instance, if LFAs require refrigeration in hot regions with little access to stable electricity or testing in temperature-controlled settings as reported for rapid antigen LFAs, they may not be fit for implementation in those contexts. It is also important to evaluate whether the kit manufacturers or governments have assumed the financial responsibility to ramp up the supply chain to avail the LFAs to communities. If communities assume the financial burden of evaluation and cost for large-scale implementation, it is unlikely that results would meaningfully improve the public health outcomes of these communities.

The WHO’s standard for point-of-care tests, including LFAs, need to be ASSURED- “Affordable, Sensitive, Specific, User-friendly, Rapid and robust, Equipment-free and Deliverable to end-users”. Gaps in any of these criteria compromise the successful implementation of the evaluated LFAs, as previously reported for the diagnostics of sexually-transmitted infections. Hence, if sustainable scale-up of LFAs is intended, then pilots for LFA design and implementation should consider “beginning with the end in mind” framework that enhances its potential for future large-scale impact. For the successful programmatic implementation of LFAs in routine serosurveillance, the 13-step recommendation guide should be used:

Participatory stakeholder engagement to build ownership, generate political commitment, and create champions of LFAs.

Ensuring the product addresses relevant public health needs and that implementation is feasible.

Building stakeholder consensus on the contextual implication of scale-up.

Tailoring LFAs to diverse sociocultural and institutional settings to ensure early identification of both barriers and opportunities for scale-up.

Ensuring LFAs be as simple as possible for the ease of future scale-up in diverse populations.

The LFA should be tested in a variety of settings whereby scale-up is intended.

Testing of implementation appropriateness should include day-to-day situations, and resource-constrained health care settings.

The process of early implementation should be evaluated and documented using implementation research.

Advocacy for financial support from governments, donors, and funding agencies for scale-up and funding for transition from pilot to large-scale rollout.

Advocacy for review of policies, laws, and regulations to institutionalize LFAs at the national level and subsequent governance structures in countries.

Laying down structures that promote learning and dissemination of information.

Cautious, incremental, initial scale-up with appropriate documentation of the implementation pathway is crucial.

Compare the LFA to other published methods.

LFA evaluation studies should consider appropriate theoretic frameworks toward achieving adoption and sustainability. These should ideally guide evidence generation, contextualize implementation and facilitate iteration, adoption, and sustainability.

Summary

In conclusion, serologic LFAs can be useful tools for estimating the true extent of SARS-CoV-2 globally, which is estimated considering inaccuracies in reporting, limited availability of molecular tests, and asymptomatic transmission. Due to their affordability and ease of implementation, LFAs can be crucial tools in determining appropriate public health mitigation responses against the COVID-19 pandemic. However, their deployment should be coupled by rigorous evaluation both for their accuracy and their fitness for implementation in a variety of health care and community settings and can guide critical decisions such as the opening of economies from the socially and economically disruptive nation-wide lockdown measures. LFAs should also be evaluated in the context of vaccine rollouts and emerging variants.

Clinics care points

Serological tests to monitor anti-SARS-CoV-2 antibodies, including LFAs, are better suited for population surveillance and research studies to monitor the longevity of infection- or vaccine-induced antibody responses. However, they are not well-suited to inform clincial decisions at the individual level, since their performance relies on several characteristics, which impact their positive and negative predictive values.