An international comparison of anti-SARS-COV-2 assays used for seroprevalence surveys from blood component providers.

BACKGROUND AND OBJECTIVES: Access to large pools of healthy adult donors advantageously positions blood component providers to undertake anti-SARS-CoV-2 seroprevalence studies. While numerous seroprevalence reports have been published by blood operators during the COVID-19 pandemic, details on the assay used has not been well documented. The objectives of this study were to evaluate the diversity of assays being used by blood operators and assess how this may affect seroprevalence estimates.
MATERIALS AND METHODS: We surveyed 49 blood component providers from 39 countries. Questionnaire included information on the number and identity of assays used, the detected immunoglobulin(s) and target antigen, and performance characteristics (sensitivity, specificity).
RESULTS: Thirty-eight of the 49 contacted blood suppliers provided at least partial responses. The results indicate that 19 commercial and five in-house serology assays have been used by surveyed blood operators. The Abbott SARS-CoV-2 IgG assay was the most commonly used kit and utilized by 15 blood suppliers. Two assays did not detect IgG, but detected either IgM/IgA or IgM. 68·2% of assays targeted the spike protein and 50% the nucleocapsid protein, while 18·2% targeted both viral proteins. The sensitivity and specificity of IgG-specific assays ranged from 71·9% to 100% and from 96·2% to 100%, respectively. As of 18 October 2020, the seroprevalence was below 5% in 10 of 14 countries reporting.
CONCLUSION: Our results highlight the diversity of assays being used. Analyses comparing blood donor seroprevalence across countries should consider assay characteristics with optimization of signal/cut-off ratios and consistent methodology to adjust for waning antibody.

Introduction

With more than 95 million cases and more than 2 million deaths worldwide as of 18 January 2021, the COVID‐19 pandemic is by far the most severe global public health crisis of the last 100 years. As treatment is currently limited to supportive care (with the exception of some novel therapies), and since most vaccines are still awaiting regulatory approvals, social distancing, the use of masks during social gatherings, aggressive testing of suspected cases and contact tracing are crucial for limiting the spread of the responsible virus, SARS‐CoV‐2. Despite strict adherence to social distancing and mask wearing, viral spread can still occur, likely from infected individuals that are asymptomatic or mildly symptomatic [1, 2]. Case data generated from SARS‐CoV‐2 nucleic acid testing may also be skewed because sampling focusses on outbreaks and contact tracing, or because resources are not available to continue laboratory sampling and/or testing [3]. Thus, measuring the degree of exposure of various populations to the virus through seroprevalence studies is of major importance for determining the level of immunity and the proportion of asymptomatic individuals who have encountered the virus. In fact, several seroprevalence studies published in the past few months revealed that the proportion of the population that has been exposed to the virus is approximately four times greater than the cumulative number of cases confirmed by SARS‐CoV‐2 nucleic acid amplification testing of respiratory samples and confirmed by national public health authorities [3].

SARS‐CoV‐2 infection can be detected by either molecular or serological assays. The former detects viral genetic material sampled in the upper and/or lower respiratory tract using real‐time reverse–transcriptase PCR (RT‐PCR), while the latter reveals the presence of antibodies in blood [4]. From a diagnostic standpoint, RT‐PCR has demonstrated superior sensitivity and earlier detection capacity compared to serological assays. In some cases nucleic acid test, results may yield false negatives due to specimen collection timing (e.g. too early or late) or anatomic location of specimen collection [4]. Given that anti‐SARS‐CoV‐2 IgA and IgM antibodies generally appear within the first 7 days after infection while IgG seems to be detectable from 10 days onwards after infection [5], serological assays targeting specific antibodies could be used as a marker of infection. However, since SARS‐CoV‐2 antibody levels often decline within a 100 days post‐infection [6], serological detection of anti‐SARS‐CoV‐2 antibodies (IgA, IgM and IgG) could lose the ability to identify true positives if used as a marker of prior infection. Nonetheless, given the substantial proportion of asymptomatic SARS‐CoV‐2 infections, as revealed by studies which compared cumulative incidence rates detected by RT‐PCR vs. seroprevalence rates [3, 7, 8], the latter could shed light on the ‘true’ infection prevalence at the population level and informs public health authorities on the degree of exposure of a given population to the virus.

It has only been about 13 months since SARS‐CoV‐2 emerged, yet numerous commercial and in‐house serologic assays have been developed during this interval. In fact, at the time of manuscript submission, more than 60 commercial assays have been approved by the U.S. Food & Drug Administration (FDA) under individual emergency use authorizations [9]. These assays can be classified into two broad categories: qualitative lateral flow immunoassays (LFA) [10] and semi‐quantitative enzyme‐linked immunosorbent assays (ELISA) [11] and chemiluminescent immunoassays (CLIA) [12]. These assays detect either total or class‐specific antibodies (IgM, IgG or IgA). In addition, they recognize different antigen as targets: the nucleocapsid protein, the spike protein or the spike protein receptor‐binding domain (RBD). This heterogeneity in assay design leads to variable degrees of sensitivity and specificity. Further to this variability, several systematic reviews have revealed that many assay evaluations are prone to biases as a result of small sample sizes and exclusion of samples from individuals who had experienced asymptomatic or mildly symptomatic COVID‐19 [13, 14, 15, 16].

Data describing test characteristics continue to be published but it is clear that sensitivity, and specificity vary considerably between assays and with different population. We have previously reported that blood centres around the world are conducting seroprevalence studies to inform public health within their countries [17]. The benefit of these data will be maximized by comparison between countries. Numerous factors will influence the measurement of seroprevalence between countries including timing within the pandemic and donor selection, but variability between assays will be a key consideration.

The aim of this study was to asses the diversity of antibody assays used by blood component suppliers, and to report their seroprevalence estimates, through an e‐mail survey.

Materials and methods

Based on a preliminary survey [17], a list of contacts was compiled from the membership of the International Society of Blood Transfusion (ISBT) Transfusion Transmitted Infectious Diseases Working Party and individuals who volunteered after being contacted by a representative of the European Blood Alliance ‐ Emerging Infectious Disease Monitoring Working Group.

Among the 62 countries who were invited to participate in the first survey, a total of 49 blood component providers (listed in Appendix S1) from 39 countries and six continents which were known for conducting donor seroprevalence studies were contacted by e‐mail in September 2020. Prospective survey participants were asked to fill a questionnaire on SARS‐CoV‐2 seroprevalence studies among their blood donor population and the assays and procedures being followed, details on assay sensitivity and specificity, and the results of seroprevalence estimates. The questionnaire was formatted in an Excel spreadsheet and participants were asked about the region they were reporting for, the number of antibody tests used for the SARS‐CoV‐2 seroprevalence study, the reason for using more than one serologic assay, details of antibody tests used for SARS‐CoV‐2 seroprevalence research (name of the assay, supplier, sensitivity, specificity, antigen target and detected antibody class), the use of NAT testing for SARS‐CoV‐2 and, if so, details on the NAT assay. Finally, blood services were questioned about their SARS‐CoV‐2 seroprevalence research results and the actual number of cases confirmed by public health authorities in their country/region.

Survey responses were received and compiled until November 2020. Table 1 lists the assays used by survey participants, with details on the assays’ respective performance characteristics. In cases where there were discrepancies between the sensitivity and specificity characteristics reported by survey participants vs. those of commercial product inserts or information available from the Food and Administration (FDA) website [18], data from products inserts or the FDA website were reported in Table 1. Finally, some missing survey data on the cumulative incidence of COVID‐19 were obtained from the WorldOMeter website [19] for the dates specified by survey participants.

Table 1

List of assays used by survey participants: detected immunoglobulins, target viral protein, sensitivity and specificity

Assay name	Supplier	Country/provider	Detected Ig	Target viral protein	% Sensitivity (95% CI)	% Specificity (95% CI)
SARS‐CoV‐2 IgG assay	Abbott Laboratories (Abbott Park)	IL, USA	IgG	Nucleocapsid	100 (95·8–100)	99·6 (99·0–99·9)
In‐house anti‐SARS‐CoV‐2 ELISA (adapted from Amanat et al. [4] and Stadlbauer et al. [32])	Héma‐Québec	Quebec, Canada	IgA/IgG/IgM	Spike	98·8	98·5
VITROS® Anti‐SARS‐COV‐2 IgG test	Ortho Clinical Diagnostics	NJ, USA	IgG	Spike	90·0 (76·9–96·0)	100 (99·1–100)
VITROS® Anti‐SARS‐COV‐2 Total test	Ortho Clinical Diagnostics	NJ, USA	IgA/IgG/IgM	Spike	100 (92·7–100)	100·0 (99·0–100)
Elecsys® Anti‐SARS‐CoV‐2	F. Hoffmann‐La Roche AG	Switzerland	IgA/IgG/IgM	Nucleocapsid	99·5	99·8
Anti‐SARS‐CoV‐2 ELISA (IgG)	EUROIMMUN Medizinische Labordiagnostika AG	Germany	IgG	Spike	90·0 (74·4–96·5)	100 (95·4–100)
MedTeste Coronavírus (COVID‐19) IgG/IgM (Teste Rápido)	Hangzhou Biotest Biotech Co.	China	IgG/IgM	Nucleocapsid	87 (IgM) 84·5 (IgG)	99·15 (IgM) 100 (IgG)
DPP® COVID‐19 IgM/IgG System	Chembio Diagnostic Systems, Inc.	NY, USA	IgG/IgM	Nucleocapsid	71·9 (IgM) 80·6 (IgG)	NS
Diagnostic Kit for IgM/IgG Antibody to Coronavirus (SARS‐CoV‐2) (Lateral Flow)	Zhuhai Livzon Diagnostics	China	IgG/IgM	NS	82·58 (75·68–88·20)	99·54 (98·66–99·90)
In‐House Virus Neutralization Test (VNT)	Établissement Français du sang (EFS)	France	IgG	NS	NS	NS
COVID‐19 total Antibody	Fortress Diagnostics	United Kingdom	IgG/IgM	Spike/Nucleocapsid	94	100
SARS‐CoV‐2 Total Ab ELISA	Beijing Wantai Biological Pharmacy	China	IgA/IgG/IgM	Spike	98	99·6
SARS‐CoV‐2 IgM ELISA	Beijing Wantai Biological Pharmacy	China	IgM	Spike	86·9	100
Oxford ELISA [5]	University of Oxford	United Kingdom	IgG/IgM	Spike	85 (70–94)	100 (93–100)
COVID‐19 ELISA IgG	Vircell S.L.	Spain	IgG	Spike/Nucleocapsid	85	98
COVID‐19 ELISA IgM + IgA	Vircell S.L.	Spain	IgA/IgM	Spike/Nucleocapsid	88	99
LIAISON® SARS‐CoV‐2 S1/S2 IgG	DiaSorin S.p.A.	Italy	IgG	Spike	97·6 (87·4–99·6)	99·3 (98·6–99·6)
In‐house Spike RBD ELISA developed by Malik Peiris and described in Perera et al., 2020 [33]	Hong Kong Red Cross Blood Transfusion Service	Hong Kong SAR, China	IgG/IgM	Spike	100	(Results pending)
cPass™ SARS‐CoV‐2 Surrogate Virus Neutralization Test (sVNT) Kit	GenScript USA Inc.	NJ, USA	IgA/IgG/IgM	Spike	100 (87·7–100)	100 (95·8–100)
In‐house Plaque Reduction Neutralization Test (PRNT)	Hong Kong Red Cross Blood Transfusion Service	Hong Kong SAR, China	NS	NS	100 (day 28 post‐RT‐PCT‐positive result) 99·11 (days 60‐209 post‐RT‐PCT–positive result)	100
Standard Q Covid‐19 IgM/IgG Combo	SD Biosensor, Inc.	Gyeonggi‐do, South Korea	IgG/IgM	Nucleocapsid	94·51	96·23
ID Screen® SARS‐CoV‐2‐N IgG indirect ELISA	IDVet	France	IgG	Nucleocapsid	93·3 (78·8–98·2)	99·9 (99·6–100)
In‐house ELISA, based on Amanat et al.’s protocol [4]	KEMRI Wellcome Trust Research Programme	Nairobi, Kenya	IgA/IgG/IgM	Spike	NS	NS
2019‐nCoV IgG/IgM Rapid Test Dipstick (WB/S/P)	Hangzhou AllTest Biotech Co., Ltd	China	IgG/IgM	Nucleocapsid	99·9 (IgG) 90·9 (IgM)	98·0 (IgG) 97 (IgM)

NS, Not Specified.

Results

Out of 49 blood component suppliers that were contacted to answer the survey, 38 provided at least partial responses, representing 27 countries (Fig. 1). Cumulative incidence from participant countries until the end of September 2020 was presented in Fig. 2. The Abbott SARS‐CoV‐2 IgG assay was by far the most commonly used kit, utilized by 15 blood suppliers, followed by the EUROIMMUN Anti‐SARS‐CoV‐2 ELISA (IgG) (seven blood suppliers), the Beijing Wantai Biological Pharmacy SARS‐CoV‐2 Total Ab ELISA (six blood suppliers), the F. Hoffmann‐La Roche AG Elecsys® Anti‐SARS‐CoV‐2 (four blood suppliers) and the Ortho Clinical Diagnostics VITROS® Anti‐SARS‐CoV‐2 Total test, the DiaSorin S.p.A. LIAISON® SARS‐CoV‐2 S1/S2 IgG and the Zhuhai Livzon Diagnostics Diagnostic Kit for IgM/IgG Antibody to Coronavirus (SARS‐CoV‐2) (Lateral Flow) (three blood suppliers each). Notably, five blood suppliers used in‐house assays. Only five blood suppliers have indicated that nucleic acid amplification testing of SARS‐CoV‐2 RNA was done; when performed, this test was done to confirm that the donor was not infectious (Fig. 3 and Appendix S1).

Fig. 1

Geographical distribution of surveyed blood component providers.

Fig. 2

COVID‐19 cumulative incidence per million population by participating countries as of 30 September 2020.

Fig. 3

Blood operators and assays characteristics *24 assay used in total and one with missing Ig detection information; # 24 assay used in total and two with missing viral protein target information.

Regarding detected immunoglobulins, target viral protein, sensitivity and specificity of the assays used by blood services, out of 24 assays being used, two assays detected Ig classes other than IgG (e.g. IgA/IgM or IgM), while 68·2% and 50% (15/22 and 11/22 of those who reported data) target the spike or the nucleocapsid viral protein. 18·2% (4/22) of those assays target both the spike and the nucleocapsid proteins. The sensitivity and specificity of IgG‐specific assays ranged from 71·9% to 100%, and from 96·23% to 100%, respectively (Table 1). Of the 38 blood component suppliers who partially replied to the survey, only 14 provided seroprevalence data (Appendix S2). Seroprevalence end of sampling dates ranged from May 2020 to October 2021. Ten out of 14 seroprevalence estimates ranged from 0% to 5·6%. The other four were much higher, ranging from 13·48% to as high as 30·89%. Three of these four providers with high seroprevalence rates were from Brazil; the fourth one was from Iran. For at least three of the four reported seroprevalence rates above 10%, the assay that was used targets the nucleocapsid protein; for the fourth seroprevalence value >10%, the information regarding the target viral protein of the assay (Hangzhou AllTest IgG/IgM Rapid Test Dipstick (WB/S/P)) is unknown. Two of the four Brazilian blood component providers used the Abbott SARS‐CoV‐2 IgG Assay which, as stated earlier, was the most widely used assay among survey participants.

Discussion

Our results reveal that 19 commercial and five in‐house assays have been used by blood component suppliers who replied to the survey. There was substantial variability in the targeted antigen, detected immunoglobulin(s) and overall performances of the assays used by blood centres. These variations could lead to diverging results and interpretations in seroprevalence surveys. Regarding its antigenic determinants, SARS‐CoV‐2 is an enveloped RNA virus composed of Spike, Envelope, Membrane and Nucleocapsid proteins [20]. As studies have indicated that they are the most immunogenic antigens [21, 22], the spike (which contains the receptor‐binding domain (RBD)) and nucleocapsid proteins are prime targets of most serologic assays. In our study, all respondents reported on using assays that target either the nucleocapsid or spike protein (or both). The available evidence suggests that antibodies against these targets are more generally associated with a protective immune response [20]. Antibody responses against the different SARS‐CoV‐2 proteins are of major importance since antibody classes have differential dynamics and neutralizing effects [22, 23]. Moreover, estimates of seroconversion (or seroreversion) rates are highly dependent on the assay target, antibody dynamics and the time point at which testing is conducted in the disease course. In support of the latter point, Post et al. [20] have thoroughly described the kinetics of IgG emergence, persistence and slow decline based on a systematic review of the literature. The overall evidence indicates that the IgG response peaks between three‐ and seven‐week post‐symptom onset. This acute response is followed by a plateau phase, and then, IgG levels slowly decline, yet persist for up to 12 weeks. Whether SARS‐CoV‐2‐specific IgGs remain detectable beyond that time point is unknown, since studies’ follow‐up periods were limited in time.

Among the 14 blood suppliers who provided seroprevalence estimates, 10 reported values of 5·6% or less (in Europe, North America and Asia), whereas the other four reported seroprevalence estimates of 13·5% or more (in Brazil and Iran). These values appear to roughly correlate with COVID‐19 case counts and/or what is known of the intensity of the regional pandemic. Blood donor studies should be very suitable for international comparison because they are all carefully screened individuals who are reasonably representative of the healthy and well adult population thus similarly selected. Nevertheless, there are some important considerations for international comparison such as the regional selection within country, timing of sample collection within the pandemic, local epidemiology (e.g. re‐infection rates and the amplitudes of pandemic waves in particular regions, as well as timings between waves) and as highlighted in our survey, the broad range of assays used for these studies.

While our survey provides an informative overview of blood donor study seroprevalence rates, these data are not appropriate for in depth comparison. There can be broad geographical variability in seroprevalence rates within the same country [24, 25] therefore catchment area of the blood centre must be known. To address differential sampling by demographic variables in donors versus the general population, data need to be sorted by demographics and weighted proportional to the general population. Donor selection and, for example socioeconomic distribution of donors may also differ between countries and complicate generalizability from donors to the general population. Finally, the survey did not include a specific question on the number of samples collected by each blood operator for seroprevalence estimates; the lack of sample size information would make any attempt at comparing seroprevalence rates questionable.

The characteristics of the assay should be considered. While there are sensitivity and specificity data for many commercial assays, it needs to be clear how a positive sample was defined. Sensitivity and specificity thresholds are influenced by the time between infection and sample collection, and threshold adjustment could optimize sensitivity and specificity and ultimately assay performance. There is ongoing debate as to what the signal to cut‐off (S/Co) ratio should be. The Abbott IgG Assay lists S/Co as 1·4 to define positivity but a grey zone of possible positivity has been proposed for samples as low as S/Co0·49. A head‐to‐head benchmark evaluation of the sensitivity and specificity of five immunoassays for SARS‐CoV‐2 (SARS‐CoV‐2 IgG assay (Abbott, Chicago, IL, USA), LIAISON SARS‐CoV‐2 S1/S2 IgG assay (DiaSorin, Saluggia, Italy), Elecsys Anti‐SARS‐CoV‐2 assay (Roche, Basel, Switzerland), SARS‐CoV‐2 Total assay (Siemens, Munich, Germany) and a novel 384‐well ELISA (the Oxford immunoassay)) revealed that all assays achieved sensitivity of ≥98% with thresholds optimized to achieve specificity ≥98% on samples collected ≥30 days post‐symptom onset [26].

Given the documented waning of antibody levels [6], antibody detection in the days that follow symptom onset could be critical for properly assessing seroprevalence. In support of this notion, a longitudinal analysis of convalescent plasma donors found that 40% of them became seronegative within four months after initial antibody detection [27]. Asymptomatic blood donors experience faster antibody decay compared to individuals with symptomatic disease [28, 29]. Moreover, inconsistent detection between assays was observed in symptomatic COVID‐19 patients over around 100 days from acute infection [30]. The Abbott CIMA test seems to show a greater decline in signal after symptom onset compared to the Roche serological assay [30]. Waning antibody should be considered in the estimate of seroprevalence. Mathematical modelling such as stochastic Monte‐Carlo approaches would be appropriate but should be applied consistently across the studies being compared.

It is also noteworthy that despite the high sensitivity and specificity of the assays used by blood centres, the infection prevalence has a direct impact on the predictive‐positive value (PPV). PPV is the proportion of true‐positive results, which is equal to (sensitivity × prevalence)/[(sensitivity × prevalence) + ((1–specificity) × (1–prevalence))] [31]. Given the relatively low prevalence of SARS‐CoV‐2 infection in many of the studies, even using a highly sensitive test, the PPV will necessarily be imperfect, and a some false‐positive results will occur.

In conclusion, aside from a few unusually high numbers, the majority of seroprevalence estimates are consistent with values that have been published in the past few months. Compared to residual samples from patients, often sick people, from medical or commercial laboratories, blood banks are preferred organizations for seroepidemiological studies due to superior sample quality, high sample accessibility and sample representativeness of a generally healthy adult population. Continuous monitoring of seroprevalence among blood donors provides a valuable indication of the level of exposure to SARS‐CoV‐2, which further informs public health authorities of the extent of the overall immunity against this virus in the general population and assists to evaluate public health interventions. Depending the testing assays used, these studies may become invaluable in monitoring vaccine uptake in the months to come. Blood bank seroepidemiological studies can provide a valuable estimate of infection across many geographic regions. However, seroprevalence estimation requires careful attention to details including the geographic region within each country, stratification by age with weighting proportionate to the general population and consistency of interpretation of the test result. Our survey shows that there is considerable heterogeneity in assays used for blood donor seroprevalence studies around the world. We believe that comparative studies are best carried out collaboratively with investigators from each seroprevalence study to ensure that data are correctly represented.

Conflicts of interest

The authors have no conflicts of interest to declare.

Appendix S1 Participating blood component suppliers and serologic assays used.

Click here for additional data file.

Appendix S2 Seroprevalence results.

Click here for additional data file.