Assessment of commercial SARS-CoV-2 antibody assays, Jamaica.

BACKGROUND: The performance of the Roche Elecsys® Anti-SARS-CoV-2, Abbott Architect SARS-CoV-2 IgM, Abbott Architect SARS-CoV-2 IgG, Euroimmun SARS-CoV-2 IgA, Euroimmun SARS-CoV-2 IgG ELISA, and Trillium IgG/IgM rapid assays was evaluated in Jamaica.
METHODS: Diagnostic sensitivities of the assays were assessed by testing serum samples from SARS-CoV-2 PCR-confirmed persons and diagnostic specificity was assessed by testing serum samples collected during 2018-2019 from healthy persons and from persons with antibodies to a wide range of viral infections.
RESULTS: Serum samples collected ≥14 days after onset of symptoms, or an initial SARS-CoV-2 RT-PCR positive test for asymptomatics, showed diagnostic sensitivities ranging from 67.9 to 75.0% when including all possible disease severities and increased to 90.0-95.0% when examining those with moderate to critical disease. Grouping moderate to critical disease showed a significant association with a SARS-CoV-2 antibody positive result for all assays. Diagnostic specificity ranged from 96.7 to 100.0%. For all assays examined, SARS-CoV-2 real-time PCR cycle threshold (Ct) values of the initial nasopharyngeal swab sample testing positive were significantly different for samples testing antibody positive versus negative.
CONCLUSIONS: These data from a predominantly African descent Caribbean population show comparable diagnostic sensitivities and specificities for all testing platforms assessed and limited utility of these tests for persons with asymptomatic and mild infections.

Introduction

The SARS-CoV-2 pandemic has resulted in an unprecedented need for reliable commercial laboratory diagnostics. While SARS-CoV-2 antibody assays have recently become commercially available, performance data have mainly assessed high-income country populations (Van Walle et al., 2020), with data from populations of mostly African descent lacking. To our knowledge, there has been no published performance assessment of SARS-CoV-2 antibody assays with a predominantly black population. In this study in Jamaica, serum samples were used to assess the diagnostic sensitivity and specificity of the Roche Elecsys® Anti-SARS-CoV-2, Abbott Architect SARS-CoV-2 IgM and IgG, Euroimmun SARS-CoV-2 IgA and IgG, and Trillium IgG/IgM assays.

Methods

For diagnostic sensitivity analysis, 42 blood samples collected in tubes without coagulant were obtained from 37 consenting persons (5 persons were sampled at two different time points) testing SARS-CoV-2 real-time PCR positive at the Jamaica National Influenza Centre using the Corman et al. method (Corman et al., 2020). Disease severity was classified according to WHO criteria. Samples were collected 6–103 days after disease onset for symptomatic persons and 20–69 days after a positive real-time PCR test for asymptomatic persons. An additional 17 consenting SARS-CoV-2 real-time PCR positive persons were recruited to compare whole blood and serum for Trillium IgG/IgM rapid test kits. Paired whole blood and serum samples were collected 11–43 days after disease onset for symptomatic persons and 11–19 days after a positive real-time PCR test for asymptomatic persons. Residual 2018–2019 serum samples from 122 different patients and testing positive for antibodies to a wide range of viral infections or from healthy donors were identified from the University of the West Indies Virology Laboratory to assess diagnostic specificity (Supplementary Table). Not all residual sera used for specificity analysis was tested across all platforms as sufficient volume was not present for some tests and a limited number of Trillium IgG/IgM test kits were provided for validation.

Detection of antibodies was conducted with an Architect i2000SR for the Architect SARS-CoV-2 IgM and IgG assays, a cobas® 6000 analyzer for the Elecsys® Anti-SARS-CoV-2 assay, a Thermo Scientific Multiskan FC Microplate Photometer for the Euroimmun SARS-CoV-2 IgA and IgG ELISAs and lateral flow assay rapid test for the Trillium IgG/IgM assay. Each manufacturer’s instructions were used for cutoff index values and interpretation of rapid test results. For Euroimmun assays, borderline index values were considered negative to allow for equivocal results to be included for sensitivity and specificity analysis, and to be consistent with other assays that do not include a borderline (or equivocal) interpretation. Previous studies (Buss et al., 2020, Eyre et al., 2020, Péré et al., 2020) of the Architect SARS-CoV-2 IgG assay indicate that samples with a high negative index value result are likely true positive but in this study were considered negative in keeping with the manufacturer’s instructions and consistent with Euroimmun borderline results being considered as negative.

This study was approved by the UWI Mona Campus Research Ethics Committee (ECP 244 19/20).

Results

Diagnostic specificity was 100.0% (104/104) for Elecsys® Anti-SARS-CoV-2, 100.0% (105/105) for Architect SARS-CoV-2 IgM, 98.2% (109/111) for Architect SARS-CoV-2 IgG, 97.5% (119/122) for Euroimmun SARS-CoV-2 IgA, 100.0% (122/122) for Euroimmun SARS-CoV-2 IgG, 96.7% (87/90) for Trillium IgM, and 98.9% (89/90) for Trillium IgG (Supplementary Table). Diagnostic sensitivities for samples collected 6–9 days, 10–13 days and ≥14 days after symptom onset were 42.9–71.4%, 85.7–100.0% and 90.0–95.0%, respectively (Figure 1 ). Samples from asymptomatic and mildly affected persons were only available ≥14 days after onset of symptoms or after an initial SARS-CoV-2 PCR test. Sensitivities ranged from 67.9 to 75.0% when all disease severities were included (Figure 1). Grouping moderate, severe and critical disease showed a significant association with testing antibody positive for all assays: Elecsys® Anti-SARS-CoV-2 (χ2 = 13.14, p < 0.001), Architect SARS-CoV-2 IgM (χ2 = 13.81, p = 0.003), Architect SARS-CoV-2 IgG (χ2 = 11.00, p = 0.001), Euroimmun SARS-CoV-2 IgA (χ2 = 16.92, p < 0.001) and IgG (χ2 = 14.30, p = 0.001), and Trillium IgM (χ2 = 6.61, p = 0.010) and IgG (χ2 = 11.70, p = 0.001). Detection of antibodies was highly congruent between assays (Figure 2 ). Additional participants were recruited to compare whole blood (point of care) and serum (laboratory) for the Trillium IgG/IgM rapid lateral flow assay. Results for Trillium IgG were identical for whole blood and serum; however, Trillium IgM results showed discrepancies when comparing whole blood and serum (Supplemental Figure).

Figure 1

SARS-CoV-2 antibody assay results by days after symptom onset for SARS-CoV-2 PCR positive persons. Means with standard deviations are displayed for (A) Architect SARS-CoV-2 IgM, (B) Architect SARS-CoV-2 IgG, (C) Elecsys® Anti-SARS-CoV-2, (D) Euroimmun SARS-CoV-2 IgA, and (E) Euroimmun SARS-CoV-2 IgG assays. Horizontal dotted lines indicate cutoff values. For (F) Trillium SARS-CoV-2 IgM and (G) Trillium SARS-CoV-2 IgG, white bars indicate the number of positive samples and colored bars indicate samples testing negative. Disease severity is color coded as follows: green = asymptomatic, blue = mild, orange = moderate, yellow = severe, and red = critical.

Figure 2

Agreement between SARS-CoV-2 antibody assays. Results for all SARS-CoV-2 RT-PCR positive samples tested for each antibody testing platform are shown. Positive results are shown in white, borderline results in light grey, negative results in dark grey. Boxes with an X indicate no result for sample due to insufficient sample volume.

SARS-CoV-2 real-time PCR cycle threshold (Ct) values were compared with the presence of antibodies from persons with samples collected ≥14 days after onset of symptoms or an initial SARS-CoV-2 PCR positive test. For the Elecsys® Anti-SARS-CoV-2, Architect SARS-CoV-2 IgG, Euroimmun SARS-CoV-2 IgG, and Trillium IgG assays, the Ct value was 23.5 ± 5.7 (mean ± SD) and 34.6 ± 1.0 for samples testing antibody positive and negative, respectively (p < 0.0001). Ct values for Architect SARS-CoV-2 IgM were 23.0 ± 6.2 for samples testing antibody positive and 33.5 ± 2.8 for samples testing antibody negative (p = 0.0008), for Euroimmun IgA were 24.0 ± 5.7 for samples testing antibody positive and 31.8 ± 6.8 for samples testing antibody negative (p < 0.0001), and for Trillium IgM, 23.0 ± 5.8 for samples testing antibody positive and 33.5 ± 2.8 for samples testing antibody negative (p = 0.0003).

Discussion

Our data examining three chemiluminescent assays, two ELISA assays and one rapid test show that the diagnostic sensitivity of these assays for SARS-CoV-2 antibodies is comparable. The similar diagnostic sensitivity and specificity of the Trillium IgM/IgG rapid diagnostic test with chemiluminescent and ELISA assays makes this test suitable for resource-limited laboratories lacking high cost instruments. However, discrepant IgM results between whole blood and serum for the Trillium IgM/IgG rapid diagnostic test warrant additional investigation should results of the IgM component of the test be considered.

An accumulating body of evidence indicates that after a SAR-CoV-2 infection antibodies become detectable approximately one week after disease onset (Deeks et al., 2020). In agreement with these studies, approximately half of the SARS-CoV-2 infected persons in our study had detectable antibodies 6–9 days after onset of symptoms, with most having antibodies ≥10 days after symptom onset. When asymptomatic and mild groups were included in our analysis, sensitivities decreased for all assays, consistent with previous studies [3,4].

Comparing SARS-CoV-2 antibody results revealed a striking difference in Ct values between persons testing antibody negative or positive. These data are consistent with a recently published study examining SARS-CoV-2-infected asymptomatic contacts and outpatients showing that SARS-CoV-2 PCR Ct values are inversely related to SARS-CoV-2 IgG index values (Wellinghausen et al., 2020). High SARS-CoV-2 viral loads may cause a more robust production of SARS-CoV-2 antibodies (Zhang et al., 2020).

Our data assessing a Caribbean population of predominantly African descent highlights the limited diagnostic sensitivity of the assays examined for persons with asymptomatic and mild SARS-CoV-2 infections and has important implications for future seroprevalence studies in which a sizable proportion of the SARS-CoV-2-infected population may have experienced no symptoms or mild disease.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflict of interest

All authors declared no conflict of interest.

CRediT authorship contribution statement

Tiffany R. Butterfield: Conceptualization, Investigation, Writing - original draft, Visualization. Alrica Bruce-Mowatt: Conceptualization, Investigation, Writing - original draft. Yakima Z.R. Phillips: Investigation. Nicole Brown: Investigation. Keisha Francis: Investigation. Jabari Brown: Investigation. Jerome P. Walker: Conceptualization, Investigation. Niel A.L. McKnight: Conceptualization, Investigation. Kelvin Ehikhametalor: Conceptualization, Investigation. Devon K. Taylor: Conceptualization, Writing - review & editing. Carl A. Bruce: Conceptualization, Writing - review & editing. Donovan McGrowder: Conceptualization, Writing - review & editing. Gilian Wharfe: Conceptualization, Writing - review & editing. Simone L. Sandiford: Conceptualization, Writing - review & editing. Tamara K. Thompson: Conceptualization, Writing - review & editing. Joshua J. Anzinger: Conceptualization, Investigation, Writing - original draft, Visualization, Supervision.