Performance Comparison of Five SARS-CoV-2 Antibody Assays for Seroprevalence Studies.

BACKGROUND: Seroprevalence studies of coronavirus disease 2019 (COVID-19) cases, including asymptomatic and past infections, are important to estimate the scale of the disease outbreak and to establish quarantine measures. We evaluated the clinical performance of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) antibody assays available in Korea for use in seroprevalence studies.
METHODS: The sensitivity, specificity, cross-reactivity, and interference of five SARS-CoV-2 antibody assays were evaluated using the following: 398 serum samples from confirmed COVID-19 patients, 510 negative control samples from before 2018 (pre-pandemic), 163 serum samples from patients with SARS, Middle East respiratory syndrome (MERS), and other viral infections, and five samples for the interference study.
RESULTS: The sensitivities of the five assays ranged from 92.2% to 98%, and their specificities, including cross-reactivity and interference, ranged from 97.5% to 100%. The agreement rates were excellent (kappa >0.9). Adjustment of the cutoff values could be considered through ROC curve analysis. The positive predictive values of the individual assays varied from 3.5% to 100% at a 0.1% prevalence but were as high as ≥95% when two assays were combined.
CONCLUSIONS: The prevalence of COVID-19 in Korea is considered to be exceptionally low at present; thus, we recommend using a combination of two or more SARS-CoV-2 antibody assays rather than a single assay. These results could help select SARS-CoV-2 antibody assays for COVID-19 seroprevalence studies in Korea.

INTRODUCTION

Coronavirus disease 2019 (COVID-19), which originated in Wuhan, China in December 2019, is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) [1]. More than 100 million people have been infected with SARS-CoV-2 and more than two million deaths due to COVID-19 have been reported worldwide in approximately one year [2]. The number of patients with confirmed disease includes only those who have been tested positive for SARS-CoV-2 following a hospital visit [3]. Therefore, the actual number of COVID-19 positive cases has been underestimated. To determine the size of the infected population and to establish quarantine measures, accurate serological testing is required.

Seroprevalence studies have been conducted in many countries, including the United States, the United Kingdom, Spain, and Korea [4-8]. In less than a year, several types of antibody assays have been developed worldwide. However, comparative studies on the performance of assays available in Korea to determine seroprevalence have not yet been conducted. The available antibody assays mainly use recombinant spike (S) proteins, nucleocapsid (N) proteins, receptor-binding domains, S1 antigens, and combinations of these antigens to detect IgG, IgM, and total antibody levels [9-16].

We evaluated the clinical performance of COVID-19 antibody assays available in Korea for seroprevalence studies. We further estimated the positive predictive values (PPVs) of individual and two combined assays using the sensitivities and specificities obtained from this study and the expected prevalence in Korea. We also investigated cross-reactivity using serum samples from patients with antibodies to various viruses and bacteria, autoimmune disease, or monoclonal gammopathy.

MATERIALS AND METHODS

Clinical samples

Serum samples, leftover from laboratory tests and designated to be discarded, from 398 patients diagnosed as having COVID-19 at two hospitals (Seoul Medical Center, Seoul, Korea and Hallym University Dongtan Sacred Heart Hospital, Hwaseong, Korea) and the Korea Disease Control and Prevention Agency (KDCA) were collected between March and September 2020 and stored at –70°C until analysis. The dates of symptom onset and hospital admission were obtained retrospectively from the medical records at the two hospitals. Serum samples of 510 negative controls, collected before 2018 (pre-pandemic period), were obtained from the National Biobank of Korea, the KDCA, and the High-Risk Human Serum Bank of Chung-Ang University (Seoul, Korea). A total of 168 samples were tested for cross-reactivity, including 136 residual serum samples of patients with antibodies to other viruses (human (h)CoV-229E, -NL63, -OC43, and -HKU1; adenovirus; influenza A virus; influenza B virus; human metapneumovirus; parainfluenza virus type 1/2/3/4; respiratory syncytial virus; rhinovirus; Mycoplasma pneumoniae; Chlamydia pneumoniae; hepatitis A/B/C virus; Epstein-Barr virus; cytomegalovirus; herpes simplex virus; mumps virus; and varicella zoster virus) collected from two hospitals (Severance Hospital, Seoul, Korea and Hallym University Dongtan Sacred Heart Hospital). Serum samples positive for fluorescence anti-nuclear antibody, rheumatoid factor, and monoclonal gammopathy were collected from the same two hospitals.

Twenty-three serum samples positive for antibodies to SARS-CoV-1, Middle East respiratory syndrome (MERS)-CoV, hCoV-229E, -NL63, -OC43, and -HKU1, influenza B virus, respiratory syncytial virus, and adenovirus were purchased from Trina Bioreactives (Nänikon, Switzerland). Five serum samples from patients with monoclonal gammopathy were collected from the Korea Institute of Radiological and Medical Sciences Radiation Biobank (Seoul, Korea), and four MERS-CoV convalescent serum samples were provided by the International Vaccine Institute (Seoul, Korea) (Fig. 1). The Public Institutional Review Board designated by the Ministry of Health and Welfare (http://public.irb.or.kr/) approved the study protocol and waived the need for informed consent (P01-202008-31-002).

Fig. 1

Samples tested in this study.

Abbreviations: CAU, Chung-Ang University; C. pneumoniae, Chlamydia pneumoniae; CMV, cytomegalovirus; EBV, Epstein-Barr virus; FANA, fluorescence anti-nuclear antibody; HAV, hepatitis A virus; HCV, hepatitis C virus; hCoV, human coronavirus; HSV, herpes simplex virus; HUDSHH, Hallym University Dongtan Sacred Heart Hospital; IVI, International Vaccine Institute; KDCA, Korea Disease Control and Prevention Agency; KIRMS, Korea Institute of Radiological and Medical Sciences; MERS-CoV, Middle East respiratory syndrome coronavirus; M. pneumoniae, Mycoplasma pneumoniae; PIV, parainfluenza virus; RSV, respiratory syncytial virus; SARS-CoV-1, severe respiratory syndrome coronavirus 1; SH, Severance Hospital; SMC, Seoul Medical Center; VZV, varicella zoster virus.

SARS-CoV-2 antibody assays

In August 2020, the KDCA received applications from 18 manufacturers to evaluate the performance of 21 SARS-CoV-2 antibody assays. It was not feasible to evaluate all 21 assays given the limited amount of serum samples; hence, a screening evaluation was required. After the Evaluation Committee did a preliminary assessment of the data submitted by the manufacturers, five assays were selected for performance evaluation: Elecsys Anti-SARS-CoV-2 assay on the Cobas e801 platform (Roche Diagnostics, Mannheim, Germany), Abbott Architect SARS-CoV-2 IgG assay on the Architect i2000SR platform (Abbott Laboratories, Abbott Park, IL, USA), Atellica IM SARS-CoV-2 Total assay on the Atellica platform (Siemens, Munich, Germany), SD Biosensor Standard E COVID-19 Total Ab assay (SD Biosensor, Suwon, Korea), and LG Chem AdvanSure SARS-CoV-2 IgG assay (LG Chem, Seoul, Korea). SD Biosensor and LG Chem ELISAs were performed using the Epoch Microplate Spectrophotometer and ELx50 Filter Microplate Washer (both from BioTek Instruments, Winooski, VT, USA). The principle, instrument, antibody detected, reagents used, cutoff value, sample volumes, and time to first results of all assays are listed in Table 1. All assays were carried out according to the manufacturers’ instructions. Most assays were performed at Hallym University Dongtan Sacred Heart Hospital by one laboratory technician and one scientific researcher, and the Atellica IM SARS-CoV-2 total assay was performed at Severance Hospital by one laboratory technician.

Table 1

Characteristics and sensitivity and specificity of the five SARS-CoV-2 antibody assays evaluated in this study

	Roche (Roche Diagnostics, Mannheim, Germany)	Abbott (Abbot Laboratories, Abbott Park, IL, USA)	Siemens (Siemens, Munich, Germany)	SD Biosensor (SD Biosensor, Suwon, Korea)	LG Chem (LG Chem, Seoul, Korea)
Characteristics of the assays
Product name	Elecsys anti-SARS-CoV-2	SARS-CoV-2 IgG	SARS-CoV-2 total (COV2T)	STANDARD E COVID-19 Total Ab	AdvanSure SARS-CoV-2 IgG (S1)
Analyzer	Elecsys Cobas e801	Architect i2000SR	Atellica IM	ELISA	ELISA
Principle	ECLIA	CMIA	CLIA	ELISA	ELISA
Target antibody	Total (IgG+IgM)	IgG	Total (IgG+IgM)	Total (IgG+IgM)	IgG
Used reagent antigen	Nucleoprotein	Nucleoprotein	RBD	Spike+Nucleoprotein	S1
Sample type	Serum, plasma	Serum, plasma	Serum, plasma	Serum, plasma	Serum, plasma
Sample volume	20 µL	25 µL	50 µL	50 µL	10 µL
Cut-off value (unit)	1.0 (COI)	1.4 (index)	1.0 (index)	NC+0.3 (OD)	1.0 (S/CO)
Time to first result (min)	18	29	15	150	150
Sensitivity and specificity of the assays according to days after symptom onset (or admission)
1–7 (N = 13)	15.4% (2/13)	15.4% (2/13)	15.4% (2/13)	53.8% (7/13)	46.2% (6/13)
8–14 (N = 65)	86.2% (56/65)	86.2% (56/65)	95.4% (62/65)	98.5% (64/65)	92.3% (61/65)
15–21 (N = 84)	98.8% (83/84)	98.8% (83/84)	98.8% (83/84)	100% (84/84)	100% (84/84)
22–28 (N = 43)	100% (43/43)	100% (43/43)	100% (43/43)	100% (43/43)	100% (43/43)
> 28 (N = 97)	96.9% (93/96)	94.8% (92/97)	98.9% (95/96)	98.9% (95/96)	98.9% (95/96)
Unknown (N = 96)	97.9% (94/96)	93.8% (90/96)	97.9% (94/96)	100% (96/96)	100% (96/96)
Total sensitivity (N = 398)	93.5% (371/397, 95% CI 90.6–95.7)	92.2% (367/398, 95% CI 90.0–95.3)	95.7% (380/397, 95% CI 93.2–97.5)	98.0% (389/397, 95% CI 96.1–99.1)	97.0% (385/397, 95% CI 94.5–98.2)
Total specificity
Pre-pandemic controls (N = 510)	99.6% (508/510)	99.2% (506/510)	100% (510/510)	99.6% (508/510)	97.2% (496/510)
Including cross-reactivity and interference (N = 683)	99.7% (681/683, 95% CI 98.9–100)	99.4% (679/683, 95% CI 98.5–99.8)	100% (683/683, 95% CI 99.5–100)	99.3% (678/683, 95% CI 98.3–99.8)	97.5% (666/683, 95% CI 95.9–98.4)

Abbreviations: CMIA, chemiluminescence microparticle immunoassay; CLIA, chemiluminescence immunoassay; CI, confidence interval; COI, cutoff index; ECLIA, electrochemiluminescence immunoassay; ELISA, enzyme-linked immunosorbent assay; NC, negative control; OD, optical density; RBD, receptor-binding domain; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

Performance evaluation of the five SARS-CoV-2 antibody assays

Sensitivity and specificity

The diagnostic sensitivity and specificity (and the 95% confidence intervals) of the five assays were calculated according to days after symptom onset using MedCalc software version 19.8 (MedCalc Software, Ostend, Belgium). The date of symptom onset was not available for some patients.

Cross-reactivity and interference

The potential cross-reactivity with other coronaviruses, such as SARS-CoV-1, MERS, respiratory viruses, and other viruses, bacteria, autoimmune diseases, and monoclonal gammopathy was evaluated using patient serum samples that were positive for antibodies against the relevant pathogen. For interference testing, Hb (10 mg/mL), bilirubin (0.4 ng/mL), a triglyceride mix (20 mg/mL), or zanamivir (10 mg/mL; all from Sigma-Aldrich) were spiked into serum samples that were negative for these antibodies. The results of spiked samples were compared with those of non-spiked samples and considered acceptable if the difference was <15%.

Agreement and correlation

The agreement and correlations between the results of the five SARS-CoV-2 antibody assays were calculated using MedCalc software and were reported as correlation graphs. For the Siemens Atellica IM SARS-CoV-2 Total assay, antibody titers <0.05 were considered 0, and those >10 were considered 10. The upper limits of detection values were determined to be 4.0 for the SD Biosensor assay and 4.5 for the LG Chem assay. Positive, negative, and total agreement between assays were evaluated using Cohen’s kappa statistics and categorized as poor (<0.00), slight (0.00–0.20), fair (0.21–0.40), moderate (0.41–0.60), substantial (0.61–0.80), or nearly perfect (0.81–1.00).

ROC curve analysis

ROC curve analyses and areas under the ROC curve (AUC) calculations for the five assays were performed using the MedCalc software. P≤0.05 was considered statistically significant. For each ROC curve, we recalculated the cutoff values so that the Youden index (=sensitivity+specificity–1) was maximized, and the sensitivity and specificity were determined from the calculated cutoff values.

PPVs and negative predictive values (NPVs) for individual and two combined assays using varying seroprevalence settings

PPVs and NPVs of individual and two combined assays using the calculated sensitivity and specificity values and the predictive seroprevalence of 10%, 5%, 2%, 1%, and 0.1% were calculated using a US Food and Drug administration online calculator [17].

RESULTS

Sensitivity and specificity

The diagnostic sensitivities were 93.5% for the Roche assay, 92.2% for the Abbott assay, 95.7% for the Siemens assay, 98.0% for the SD Biosensor assay, and 97.0% for the LG Chem assay. The diagnostic specificities were 99.7% for the Roche assay, 99.4% for the Abbott assay, 100% for the Siemens assay, 99.3% for the SD Biosensor assay, and 97.5% for the LG Chem assay (Table 1). The positivity rate was low in the first week after symptom onset but was nearly 100% three to four weeks after symptom onset. The SD Biosensor assay showed the highest sensitivity and the Siemens assay showed the highest specificity.

Cross-reactivity and interference

The Abbott, Roche, and Siemens assays showed negative cross-reactivity for 33 species of viruses and bacteria in 168 positive serum samples (Supplemental Data Table S1). The SD Biosensor and LG Chem assays each showed three false-positive results in the cross-reactivity study. No samples showed interference for Hb, a triglyceride mix, bilirubin, albumin, or zanamivir.

Agreement and correlation

The agreement among all assays was nearly perfect (kappa>0.90) (Table 2); however, correlations among the assays were non-linear (Supplemental Data Fig. S1), and several samples exceeded the upper limit of detection in the Siemens, SD Biosensor, and LG Chem assays. The Siemens and SD Biosensor assays showed the highest agreement rate at 98.7% (95% CI, 97.8–99.3; kappa, 0.966). Discordant results among the assays are shown in Supplemental Data Table S2.

Table 2

Agreement rates between the five SARS-CoV-2 antibody assays

A/B	Roche/ Abbott	Roche/ Siemens	Roche/SD Biosensor	Roche/ LG Chem	Abbott/ Siemens	Abbott/SD Biosensor	Abbott/LG	Siemens/ SD Biosensor	Siemens/LG Chem	SD Biosensor/ LG Chem
Positive/Positive (total number)	364	368	371	371	363	366	366	380	378	385
Positive/Negative (total number)	9	5	2	2	7	4	4	0	2	17
Negative/Positive (total number)	6	12	23	31	17	28	36	14	24	9
Negative/Negative (total number)	700	694	683	675	692	682	674	685	675	669
Positive agreement of A to B (%) (95% CI)	98.4 (96.5–99.4)	96.8 (94.5–98.4)	94.2 (91.4–96.3)	92. 3 (78.2–94.7)	95.5 (92.9–97.4)	92.9 (89.9–95.2)	91.0 (87.8–93.6)	96.5 (94.1–98.0)	94.0 (91.2–96.1)	97.8 (95.7-99.0)
Negative agreement of A to B (%) (95% CI)	98.7 (97.6–99.4)	99.3 (98.3–99.8)	99.7 (98.9–100)	99.7 (98.9–100)	99.0 (97.9–99.6)	99.4 (98.5–99.8)	99.4 (98.5–99.8)	100 (99.5–100)	99.7 (98.9–100)	97.5 (96.1-98.6)
Positive agreement of B to A (%) (95% CI)	97.6 (95.5–98.9)	98.7 (96.9–99.6)	99.5 (98.1–99.9)	99.5 (98.1–99.9)	98.1 (96.1–99.2)	98.9 (97.3–99.7)	98.9 (97.3–99.7)	100 (99.0-100)	99.5 (98.1–99.9)	95.8 (93.3-97.5)
Negative agreement of B to A (%) (95% CI)	99.2 (98.2–99.7)	98.3 (97.1–99.1)	96.7 (95.2–97.9)	95.6 (93.8–97.0)	97.6 (96.2–98.6)	96.1 (94.4–97.4)	94.9 (93.0–96.4)	98.0 (96.7–98.9)	96.6 (94.9–97.8)	98.7 (97.5-99.4)
Total agreement (%) (95% CI)	98.6 (97.7–99.2)	98.4 (97.5–99.1)	97.7 (96.6–98.5)	96.9 (95.7–97.9)	97.8 (96.7–98.6)	97.0 (95.8–98.0)	96.3 (95.0–97.3)	98.7 (97.8–99.3)	97.6 (96.5–98.4)	97.6 (96.5-98.4)
Kappa	0.961	0.962	0.946	0.930	0.943	0.934	0.918	0.966	0.942	0.949

Abbreviations: CI, confidence interval; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

ROC curve and cutoff analysis

The AUCs calculated from ROC curve analysis were 0.976, 0.987, 0.984, 0.994, and 0.987 for the Roche, Abbott, Siemens, SD Biosensor, and LG Chem assays, respectively (Table 3). The optimal cutoff values, calculated using the Youden index method, were 0.188 cutoff index (COI) for the Roche assay, 0.44 index for the Abbott assay, 0.57 index for the Siemens assay, 0.404 OD for the SD Biosensor assay, and 1.163 S/CO (signal/cutoff) for the LG Chem assay, which differed from the original manufacturers’ cutoff values. Using the calculated cutoff values, the sensitivities of the Roche, Abbot, and Siemens assays increased (Roche, from 93.5% to 96.5%; Abbott, from 92.2% to 96.2%; Siemens, from 95.7% to 96.7%), whereas the specificities decreased (Roche, from 99.7% to 98.1%; Abbott, from 99.4% to 99.0%; Siemens, from 100% to 99.6%). By contrast, the sensitivities and specificities of the SD Biosensor and LG Chem assays did not significantly differ when using the manufacturers’ or calculated optimal cutoff values (Table 3).

Table 3

ROC curve analysis and calculated cutoff values for the five SARS-CoV-2 antibody assays

	Roche	Abbott	Siemens	SD Biosensor	LG Chem
AUC (P)	0.976 (P < 0.001)	0.987 (P < 0.001)	0.984 (P < 0.001)	0.994 (P < 0.001)	0.987 (P < 0.001)
Manufacturer’s cutoff	1.0 COI	1.4 index	1.0 index	(NC+0.3) OD	1.0 S/CO
Sensitivity % (95% CI) according to the manufacturer’s cutoff	93.5 (90.6–95.7)	92.2 (90.0–95.3)	95.7 (93.2–97.5)	98.0 (96.1–99.1)	97.0 (94.5–98.2)
Specificity % (95% CI) according to the manufacturer’s cutoff	99.7 (98.9–100)	99.4 (98.5–99.8)	100 (99.5–100)	99.3 (98.3–99.8)	97.5 (95.9–98.4)
Cutoff calculated based on the Youden index	0.19 COI	0.44 index	0.57 index	0.40 OD	1.16 S/CO
Sensitivity % (95% CI) according to the calculated cutoff	96.5 (94.2–98.1)	96.2 (93.9–97.9)	96.7 (94.5–98.2)	97.7 (95.7–99.0)	96.7 (94.5–98.2)
Specificity % (95% CI) according to the calculated cutoff	98.1 (96.8–99.0)	99.0 (97.9–99.6)	99.6 (98.7–99.9)	99.4 (98.5–99.8)	98.0 (96.6–98.9)

Abbreviations: AUC, area under the curve; COI, cutoff index; NC, negative control; OD, optical density; S/CO, signal/cutoff; CI, confidence interval; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

PPVs and NPVs for individual and two combined assays using determined sensitivity, specificity, and seroprevalence

The lower the prevalence rate (from 10% to 0.1%), the lower is the PPV. The Siemens assay showed the highest specificity of 100% (95.2%; PPVs calculated using the lowest value of the 95% CI of the calculated specificity are shown in parentheses because the specificity was calculated as 100%, even at the lowest prevalence rate) among the five assays at a 10% prevalence and the highest specificity of 100% (15.1%) at a 0.1% prevalence (Table 4). When the predicted prevalence rate of 0.1% in Korea was considered, the PPV was as low as 24.2% for the Roche assay and 13.7% for the Abbott assay. However, the PPVs increased when two assays were both positive or when the orthogonal test algorithm (i.e., employing two assays in sequence when the first assay yields a positive result) [17] was used (Table 4).

Table 4

PPVs when one assay or two combined assays were positive for five exemplary populations with 10%, 5%, 2%, 1%, and 0.1% SARS-CoV-2 prevalence

	PPV (%) when one assay or two combined assays are both positive*
	SARS-CoV-2 10% prevalence	SARS-CoV-2 5% prevalence	SARS-CoV-2 2% prevalence	SARS-CoV-2 1% prevalence	SARS-CoV-2 0.1% prevalence
Roche	97.3 ([†]90.8)	94.4 ([†]82.4)	86.7 ([†]64.4)	76.3 ([†]47.2)	24.2 ([†]8.2)
Abbott	94.6 ([†]87.4)	89.3 ([†]76.6)	76.0 ([†]56.0)	61.6 ([†]38.6)	13.7 ([†]5.9)
Siemens	100 ([†]95.2)	100 ([†]90.3)	100 ([†]78.4)	100 ([†]64.2)	100 ([†]15.1)
SD Biosensor	93.7 ([†]86.5)	87.6 ([†]75.2)	73.2 ([†]54.1)	57.5 ([†]36.8)	10.8 ([†]5.5)
LG Chem	80.3 ([†]72.2)	65.9 ([†]55.2)	42.8 ([†]32.3)	27.0 ([†]19.1)	3.5 ([†]2.3)
Roche+Abbott	100 ([†]99.8)	100 ([†]99.7)	99.9 ([†]99.1)	99.8 ([†]98.2)	98.1 ([†]84.7)
Roche+Siemens	100 ([†]99.9)	100 ([†]99.9)	100 ([†]99.7)	100 ([†]99.4)	100 ([†]94.0)
Roche+SD Biosensor	100 ([†]99.8)	100 ([†]99.6)	99.9 ([†]99.1)	99.8 ([†]98.1)	97.7 ([†]83.6)
Abbott+Siemens	100 ([†]99.9)	100 ([†]99.8)	100 ([†]99.6)	100 ([†]99.1)	100 ([†]91.7)
Abbott+SD Biosensor	100 ([†]99.7)	99.9 ([†]99.5)	99.8 ([†]98.7)	99.5 ([†]97.3)	95.5 ([†]78.2)
Siemens+SD Biosensor	100 ([†]99.9)	100 ([†]99.8)	100 ([†]99.5)	100 ([†]99.0)	100 ([†]91.1)

*Calculated using online calculators on the US Food and Drug administration [13]; †PPVs calculated using the lowest value of the 95% CI of the calculated specificity are shown in parentheses.

Abbreviations: CI, confidence interval; PPV, positive predictive particle; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

DISCUSSION

The sensitivities of the five assays were all >93%, and the sensitivities were low in the first week after symptom onset but were nearly 100% three to four weeks after symptom onset. These results are similar to or better than those in previous studies [10, 12-20]. In a previous study, some SARS-CoV-2 antibody assays showed a sensitivity of 100%; however, that study used only samples collected two weeks after symptom onset [17]. The assay sensitivity may vary across studies depending on how many of the samples that are collected in the first week after symptom onset (especially on days 0–3) are included. In this study, the SD Biosensor and LG Chem assays had higher sensitivity but lower specificity than the Roche, Abbott, and Siemens assays. The Roche, Abbott, and Siemens assays use high cutoff values to increase specificity, according to the CDC guidelines [21]. The sensitivity and specificity and the cutoff may vary depending on the purpose of the assay. If the assay is used for diagnostic screening, a high sensitivity is preferred. However, the specificity should be high when the assay is used to investigate seroprevalence or the effect of vaccination. Laboratories can adjust assay cutoff values according to the intended purpose.

We extensively investigated cross-reactivity using serum samples of patients with antibodies to various viruses and bacteria, autoimmune disease, or monoclonal gammopathy. The Roche, Abbott, and Siemens assays showed negative results for 163 samples tested for cross-reactivity, whereas the SD Biosensor and LG Chem assays showed false-positive results in three out of the 163 samples. These false-positive results seemed to be due to nonspecific reactions as only one sample yielded a positive result in each species. We found no cross-reactivity in serum with anti-SARS-CoV-1 antibody, which is considered the most similar to the anti-SARS-CoV-2 antibody, or in hypergammaglobulinemia serum.

This study was the first to estimate the PPVs of individual and two combined assays based on the orthogonal test algorithm, using the sensitivities and specificities calculated in this study and the expected prevalence in Korea. As the prevalence in Korea is expected to be <0.1% at present, if only one assay was used to determine the seroprevalence, the PPV was as low as 24.2% (8.2%) for the Roche assay, 13.7% (5.9%) for the Abbott assay, 100% (15.1%) for the Siemens assay, 57.5% (10.8%) for the SD Biosensor assay, and 3.5% (2.3%) for the LG Chem assay (Table 4). When two assays were combined, the PPV increased to >95% for all combinations.

We evaluated the performances of SARS-CoV-2 antibody assays available in Korea during September 2020, and the assay reagents were still in development during this period; hence, the sensitivity and specificity of all assays could be improved, and the cutoff values could be adjusted. Presently, other SARS-CoV-2 antibody assays have been developed and are available in Korea.

One limitation of this study is that neutralizing antibody assays that may represent personal immunity were not included [16]. Because most vaccines use the S protein as an immunogen, SARS-CoV-2 antibody assays targeting the N protein, such as the Abbott and Roche assays evaluated in this study, would not be useful to evaluate the vaccine response. However, the use of combinations of assays targeting the N and S proteins may allow discriminating natural infections from vaccinations.

In summary, this study was performed to select appropriate SARS-CoV-2 antibody assays for implementation in a large-scale seroprevalence study in Korea. We estimated PPVs of individual and two combined assays based on the orthogonal test algorithm, using calculated sensitivity and specificity and expected prevalence of COVID-19 in Korea. Because the prevalence of COVID-19 in Korea is considered exceptionally low at present, we recommend using a combination of two or more assays rather than a single assay.