Evaluation of the genalyte maverick SARS-CoV-2 multi-antigen serology panel.

Serologic testing for SARS-CoV-2 can be used for evaluation of past infection in individual patients and for community seroprevalence studies. We evaluated the analytical and clinical performance of the Genalyte Maverick SARS-CoV-2 Multi-Antigen Serology Panel compared to the Roche Elecsys Anti-SARS-CoV-2 nucleocapsid (NC) qualitative immunoassay, using well characterized clinical serum samples. A total of 143 pre-pandemic sera and 48 sera collected from patients with a negative molecular SARS-CoV-2 result were used for specificity studies. For sensitivity analyses, 179 sera were used, obtained 3-7 days, 8-14 days, or ≥ 15 days after symptom onset from patients with confirmed SARS-CoV-2 infection. Specificity was determined to be 95.3% (182/191) for the Genalyte Maverick. Overall sensitivity of the Genalyte Maverick was similar to that observed for the Roche Elecsys NC test, 79.3% (142/179) vs. 76.5% (137/179), respectively. Genalyte Maverick trended, without statistical significance, towards higher sensitivity as compared to the Roche Elecsys NC test in the 3-7 days (11/25 vs. 9/25, respectively) and 8-14 days (21/28 vs. 19/28, respectively) post-symptom onset sample sets, but was identical in the ≥ 15 days post-symptom onset group (106/116 vs. 106/116, respectively). Therefore, the Genalyte Maverick serologic test had similar overall sensitivity to the Roche Elecsys NC assay, but may have slightly improved sensitivity for early seroconversion. The lower Genalyte Maverick specificity as compared to the Roche Elecsys NC assay as reported by other studies (>99%), may necessitate confirmatory testing of positive Genalyte Maverick results if implemented for clinical use.

Introduction

The global pandemic from severe acute respiratory syndrome-coronoavirus-2 (SARS-CoV-2) necessitated the development of multiple laboratory methods to assess both active and prior coronavirus disease-2019 (COVID-19). While molecular and antigen detection assays are used to identify active viral replication, serologic assays to detect the body's humoral immune response to SARS-CoV-2 are generally used to document previous infection. The clinical utility of SARS-CoV-2 serologic tests are limited, but include investigating local and community seroprevalence, assessing whether individual patients were previously infected, and identification of COVID-19 convalescent plasma donors [1]. While serologic testing can be utilized for determining immunity against other vaccine-preventable diseases [2, 3], a minimum antibody ‘immunity threshold’ has not yet been established for SARS-CoV-2 and post-vaccination serologic testing is not currently recommended.

Depending on their design, serologic tests will detect IgM and/or IgG antibodies, with or without immunoglobulin differentiation, to SARS-CoV-2 in human blood. Seroconversion rates peak at 4-5 weeks post-symptom onset [4]. The SARS-CoV-2 target proteins used in serologic tests most commonly include recombinant, full or partial S (spike) or subunit 1/2 (S1, S2) proteins, S1 receptor binding domain (RBD), or NC (nucleocapsid) protein. Several methodologies for serologic testing have been developed including chemiluminescence immunoassays, enzyme-linked immunosorbent assays (ELISA), and lateral flow immunoassays for use in the central laboratory or at the point of care [5], [6], [7]. Currently, over 70 serology tests have received emergency use authorization (EUA) by the Food and Drug administration (FDA); 19 for high complexity testing only, 51 for high/moderate complexity testing, and 5 for high/moderate/waived complexity testing (https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/vitro-diagnostics-euas, accessed 4/13/2021).

The Genalyte Maverick SARS-CoV-2 Multi-Antigen Serology Panel v2 received FDA EUA for use in high or moderate complexity laboratories on October 8, 2020 and is a qualitative detection system for antibodies to SARS-CoV-2. The panel includes detection of IgG and IgM against five SARS-CoV-2 antigens (full length NC, full length S, S1, S2 and RBD), the SARS-CoV NC antigen, the Middle East respiratory syndrome (MERS) S1 antigen, and two influenza A hemagglutinin (H) antigens (H1 and H3). The test is unique among the available SARS-CoV-2 serology assays because it semi-quantitatively detects both IgM and IgG antibodies to multiple SARS-CoV-2 antigens and three other respiratory viruses. The system is based on photonic ring resonance technology on a silicon chip, and using a machine learning algorithm, simultaneously measures and interprets all reactions, releasing results within approximately 20 minutes. Here, we evaluated the test performance of the Genalyte SARS-CoV-2 serology panel against an automated reference method for detection of total antibodies against the SARS-CoV-2 NC antigen.

Materials and methods

Specimens

The SARS-CoV-2 antibody negative sample set consisted of 143 pre-pandemic, residual sera collected in Rochester, MN between May 8, 2015 and November 30, 2019, and stored at -80°C until thawed for the study. An additional 48 presumed SARS-CoV-2 antibody negative sera were residual specimens obtained primarily on May 1-3, 2020 from asymptomatic patients undergoing pre-procedural SARS-CoV-2 NAAT screening and antibody testing. Presumed negative sera were negative by the EuroImmun Anti-SARS-CoV-2 IgG ELISA (Lubeck, Germany), and collected from patients with a negative nucleic acid amplification test (NAAT) for SARS-CoV-2, with NAAT performed either on the day of blood collection or within the preceding 3 days. NAAT testing was performed in a CLIA-certified laboratory by one of several methods with FDA EUA. Presumed negative sera were stored at -80°C until being thawed for the study.

The SARS-CoV-2 antibody positive samples consisted of 179 residual sera from patients who had tested positive for SARS-CoV-2 by one of several FDA EUA NAAT tests. Serum samples were characterized as being collected 3-7 days, 8-14 days or, ≥ 15 days after symptom onset. For serum collected within 15 days of NAAT testing, electronic medical record review was performed to determine the number of days between symptom onset and blood collection. For serum collected more than 15 days after NAAT testing, no electronic medical record review was performed, and these samples were assumed to fall into the ≥15 days from symptom onset category. The range of days between NAAT and blood collection for these samples was 15 to 159 days, with >85% collected within 60 days of NAAT testing. Serum samples were initially stored up to 3 days on gel at 4-6°C before being aliquoted and stored at 4-6°C for an additional 4 days. The aliquots were then frozen and stored at -80°C until thawed for the study. For these samples, the care environment (inpatient, outpatient, emergency department) at the time of blood collection was recorded. Among the 179 samples, 89 were collected from inpatients, 86 were collected from outpatients, and 4 were collected in the emergency department. Among the 116 samples collected from patients ≥15 days after a positive NAAT or onset of symptoms, 31 were collected from inpatients, 81 were collected from outpatients and 4 were collected in the emergency department.

A second set of 40 SARS-CoV-2 antibody positive samples were characterized as positive by both the Roche Diagnostics Elecsys Anti-SARS-CoV-2 NC Total Antibody and the Ortho-Clinical Diagnostics VITROS Anti-SARS-CoV-2 IgG serologic tests. No NAAT test results or other patient clinical information were available for these samples. The samples were obtained from residual positive Ortho VITROS results and stored at -80°C and thawed twice; once for testing on Roche total antibody assay and once for testing by the Genalyte assay.

SARS-CoV-2 serologic assays

All serum samples were tested on a Genalyte Maverick instrument using the Maverick SARS-CoV-2 Multi-Antigen Serology panel v.2 assay (Genalyte Inc, San Diego CA). The assay uses photonic ring resonance [8] for the qualitative detection of total antibodies (IgG and IgM) to SARS-CoV-2 using EDTA whole blood, EDTA plasma, or serum samples. The assay uses a silicon chip to which 13 antigens are adhered, and the instrument detects changes in resonance wavelength as antibodies (IgG and/or IgM) are bound. The viral antigens used include 5 SARS-CoV-2 antigens (NC, S, S1, S2 and RBD), 4 antigens derived from commonly circulating coronaviruses (SARS-CoV-229E S, SARS-CoV-NL63 NC, SARS-CoV-OC43 S and SARS-CoV-HKU1 S), 2 influenza A hemagglutinin antigens (H1 and H3), and the NC and S1 antigens derived from SARS-CoV and MERS, respectively. External positive and negative quality controls were run daily, and serum samples were analyzed according to manufacturer's instructions for use, including assay calibration. Each chip has two channels allowing for two samples to be analyzed, and the instrument has two bays such that 4 samples can be analyzed simultaneously, with testing time of approximately 20 minutes. Test results are transmitted to the CloudLab software, where signals from the 26 antigens (separate IgG and IgM signals for the 13 antigens), along with additional control signals, are analyzed by the proprietary algorithm to determine whether each run (channel) was valid. If the run is valid, the algorithm returns results for total antibody of negative, indeterminate, or positive (EUA version of software). Invalid results that remained invalid upon retesting were excluded from our analysis. In the development version of the software used for this study, results returned from the CloudLab software included overall probability score (0-1), with a probability score of 0-0.45 interpreted as negative, 0.451-0.549 interpreted as indeterminate, and 0.55-1 interpreted as positive.

The Roche Diagnostics Elecsys Anti-SARS-CoV-2 assay (Roche Diagnostics, Indianapolis IN) used in our automated central laboratory was used as the primary reference method against which the Genalyte assay was compared. The Roche Elecsys assay is immunoglobulin agnostic, uses a recombinant SARS-CoV-2 NC antigen, and is designed as a dual-antigen binding sandwich immunoassay. Testing was performed on a Roche Cobas 8000 e801immunoassay analyzer. Depending on timing of sample collection relative to symptom onset, the sensitivity of the Roche Elecsys assay ranges from approximately 80% to 95.8% in sera collected 2-3 or ≥4 weeks post-symptom onset, respectively, with specificity approaching 100% [9], [10], [11]. The Ortho-Clinical Diagnostics Anti-SARS-CoV-2 IgG assay (Ortho Clinical Diagnostics, Rochester, NY) uses an immunometric two stage reaction to detect IgG antibodies against the S protein, with testing performed on the VITROS 3600 analyzer. The EuroImmun Anti-SARS-CoV-2 IgG ELISA (EuroImmun US Inc, Lubeck, Germany) uses microplate strips, each with 8 wells coated with recombinant SARS-CoV-2 S1 and was performed on the Dynex Agility instrument (Dynex Technologies, Chantilly VA). Assay performance characteristics for the Ortho-Clinical and EuroImmun assays have been described previously [12]. All assays have received FDA EUA and were performed according to manufacturer instructions, without deviation, in a CLIA-certified laboratory by trained laboratory technologists.

Statistical methods

Specificity was calculated as percent of samples with a negative Genalyte Maverick SARS-CoV-2 Multi-Antigen panel result among the 191 negative samples; separate analysis of the 143 pre-pandemic samples and 48 presumed negative samples was also performed. Sensitivity for both the Genalyte Maverick SARS-CoV-2 Multi-Antigen panel and Roche Elecsys Anti-SARS-CoV-2 Total Antibody tests were calculated as percent of positive samples (serum samples obtained from patients with previous positive NAAT test) testing total antibody positive by each method. This was done for the entire 179 positive sample set, as well as within subsets of samples obtained from patients within 3-7 days, 8-14 days, and with ≥15 days from symptom onset or NAAT testing. A separate comparison was done measuring sensitivity (percent Maverick SARS-CoV-2 Multi-Antigen panel positive) among 40 samples testing positive by both the Roche Elecsys and Ortho VITROS IgG assays.

Results

Invalid results obtained on Genalyte Maverick

A total of 22 samples from 19 unique patients were invalid on the first run on the Genalyte Maverick. Four patients in which a failed run occurred had additional samples included from different collections. In 3 out of 4 patients with multiple samples collected, others collected samples gave valid results on the first attempt suggesting that test failures were not a result of a patient-specific interference. After repeat testing, 10 samples (45%) gave a result while the other 12 samples (55%) gave a second invalid result. The 12 samples with double invalid result were excluded from all subsequent analyses and sample counts. The double invalid samples were both known negatives (n = 8) and known positive samples (n = 4). For the 10 samples that gave a valid result on the second test, the second valid result was used in the sample counts and performance metrics. The single invalid samples were both known negatives (n = 5) and known positive samples (n = 5).

Specificity of the Genalyte Maverick

Overall, 182 of 191 (95.3%) negative serum samples tested negative (probability ≤0.45) by the Genalyte Maverick. Three samples were positive (probability >0.55) and six samples were indeterminate (probability 0.46-0.55). Including indeterminate results as negative (considering only positive results to be false positives), the specificity improved to 98.4% (188 of 191). Among the 143 pre-pandemic serum samples, 136 (95.1%) were negative by the Genalyte Maverick; while 46 of 48 (95.8%) presumed negative samples were negative by Genalyte Maverick.

Sensitivity

The Genalyte Maverick assay detected total antibody more frequently than the Roche Elecsys in the 179 sera from SARS-CoV-2 NAAT positive patients (Table 1 ).

Table 1

Genalyte Maverick and Roche Elecsys NC positive results in serum samples obtained from patients previously testing positive for SARS-CoV-2 by NAAT testing

	Overall	3-7 days	8-14 days	≥ 15 days
Genalyte	142/179 (79.3%)	11/25 (44.0%)	21/28 (75.0%)	106/116 (91.4%)
Roche Elecsys	137/179 (76.5%)	9/25 (36.0%)	19/28 (67.9%)	106/116 (91.4%)

Genalyte Maverick also detected antibody in more samples collected either 3-7 days or 8-14 days after symptom onset as compared to the Roche Elecsys. The higher sensitivity of the Genalyte assay as compared to the Roche test for early detection was seen in samples collected 5-10 days after symptom onset (Fig. 1 ).

Fig. 1

Percent of serum samples (collected from patients with a positive NAAT test for SARS-CoV-2) with detectable total antibody to SARS-CoV-2 by Genalyte Maverick and Roche Elecsys assays. The bar graph shows percent antibody positive serum samples collected 3-4 days, 5-6 days, 7-8 days, 9-10 days, 11-12 days, or 13-14 days after onset of symptoms. Samples collected ≥15 days from either symptom onset or the date of NAAT testing are also shown. The number of samples included in each symptom onset category is included above the bars in the graph.

Among 116 samples collected from patients with symptom onset ≥15 days or with NAAT testing ≥15 days before serum collection, the Genalyte Maverick and Roche Elecsys tests were both positive in 91.4% of samples (Table 1).

Among the 40 samples previously positive by both the Roche Elecsys and Ortho VITROS IgG assays, 39 of 40 (97.5%) were also positive by the Genalyte Maverick assay.

Discussion

Test performance is critically important when using SARS-CoV-2 serologic tests to detect disease seroprevalence in a community or prior individual infection. Given the variable disease prevalence, high specificity of serologic assays is essential in order to ensure accurate results. Currently, the recommended specificity for serologic assays, as defined by the Centers for Disease Control and Prevention (CDC) is ≥ 99.5% (https://www.cdc.gov/coronavirus/2019-ncov/lab/resources/antibody-tests-guidelines.html, accessed March 18, 2021). Serologic test sensitivity, while equally important, is expectedly more variable given the multiple factors that impact timing of seroconversion (e.g., disease severity, immune status, specimen collection timing, etc.). Here, we show the test performance of the Genalyte Maverick serologic test compared to the reference Roche Elecsys NC assay, using clinical samples that were well characterized for COVID-19 status and date of symptom onset.

The Genalyte Maverick serologic test had similar, if not slightly improved, sensitivity for detection of antibodies to SARS-CoV-2 in confirmed samples as compared to the Roche Elecsys assay. For samples collected 15 or more days after symptom onset or NAAT testing, the sensitivity of the Genalyte and reference method were identical at 91.4%. However, the Genalyte assay showed slightly better sensitivity in samples collected both 3-7 days (44.0% vs. 36.0%) and 8-14 days (75.0% vs. 67.9%) after symptom onset. Although larger data sets are necessary to demonstrate whether these differences are statistically significant, it seems likely that use of multiple antigens in the Genalyte Maverick panel may allow for earlier detection of antibodies to SARS-CoV-2.

The Genalyte Maverick assay uses five different target peptides for SARS-CoV-2 and the signals from all antigens are used to determine test positivity. In contrast, the Roche Elecsys assay relies on antibody detection against only the NC antigen. Sensitivity of the Genalyte assay from COVID-19 confirmed patients followed a similar trend relative to other assays, showing lower positivity (~40%) within the first week and increasing to near 100% in samples ≥15 days post symptom onset [5,13,14]. The 91% positivity rate of samples collected ≥15 days post symptom onset in our sample set is lower than reported in other studies. This lower positivity rate is unlikely an analytical performance issue given that the Genalyte and Roche tests reported identical positive rates in this cohort and nearly identical rates in another study [15]. It is more likely that our cohort included samples from patients with a lower rate of seroconversion because many (48% of samples overall and 70% of samples collected ≥15 days from symptom onset or NAAT testing) of these specimens were obtained from patients treated in the outpatient practice and likely experienced more mild disease. As has been reported, asymptomatic and mildly infected patients typically have lower anti-SARS-CoV-2 antibody levels and may be less likely to seroconvert compared to those with more severe symptoms [16], [17], [18].

For the purposes of prevalence studies, the false positive rate of any assay is critical when considering testing individuals in low-prevalence settings. Therefore, the specificity of a serologic test would ideally be as close to 100% as possible. In our study, the specificity of the Genalyte test was found to be approximately 95%. Similar to methods used in other studies, analytic specificity was determined in part using pre-pandemic samples collected well before the first COVID-19 case was identified in the United States, and in part, using samples collected from patients with negative SARS-CoV-2 NAAT tests. The Genalyte assay specificity was lower than that reported for the Roche Elecsys assay(~100%) [19]. The difference in specificity may be related to the antigens used in the two assays and the different analytical methods. The NC protein solely used by the Roche Elecsys assay is relatively small with a highly conserved sequence compared to the S protein which is larger and less well conserved. Thus, serologic tests that use both the NC and S proteins, such as the Genalyte test, may be more susceptible to cross-reacting antibodies from other, closely related CoVs. The Genalyte panel compensates for this by measuring antibodies to both SARS-CoV-2 and commonly circulating CoV antigens and by using a machine learning algorithm to determine positivity for SARS-CoV-2 total antibody. While this approach appears to allow greater sensitivity for early detection of antibody to SARS-CoV-2, it may result in somewhat decreased specificity compared to the dual-antigen binding, automated immunoassay using nucleocapsid antigens. The specificity shown by Genalyte might be acceptable for some clinical situations, especially where the pre-test probability is high.

Several limitations to our study should be noted. We included only residual samples and not fresh collections so analytical problems or assay inaccuracies from sample storage might have affected our results. Additionally, assay interferences due to fibrin strands or processing of fresh blood samples that might occur with use of fresh samples may have been missed. Although the number and characterization of samples used in the study was similar to that included in previous publications comparing SARS-CoV-2 assays [5,9,19], the study size was not large enough to assess the statistical significance of differences in sensitivity and specificity between tests. Additional studies are necessary to determine whether the Genalyte method can consistently detect SARS-CoV-2 antibodies earlier than other, high-throughput, automated serologic assays. Lastly, it is unclear how the Genalyte test would perform in patients further removed from their initial COVID-19 infection, as few samples (<15%) were collected more than 60 days after symptom onset or date of NAAT testing.

Overall, the Genalyte Maverick SARS-CoV-2 Multi-Antigen Serology Panel v2 showed good sensitivity for detection of antibodies to SARS-CoV-2, perhaps even better sensitivity than the reference method, in samples collected within the first two weeks of symptom onset. However, the modest decrease in specificity compared to the reference method may necessitate confirmatory testing if the test were to be utilized for population screening or for assessing previous infection in low prevalence populations.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.