Clinical Evaluation of Siemens SARS-CoV-2 Total Antibody assay and IgG assay using the Dimension EXL 200 in the Tokyo Metropolitan area.

BACKGROUND: We evaluated the efficacy of the Siemens SARS-CoV-2 Total Antibody assay (CV2T) and IgG assay (CV2G) that can detect antibodies against the receptor binding domain of S antigen in patients with COVID-19 in a Tokyo metropolitan area.
METHODS: Sensitivity and antibody levels were examined by CV2T and CV2G on Dimension EXL 200 using 236 serum samples obtained from 79 RT-PCR confirmed COVID-19 patients at multiple time points and were compared with disease severity by the World Health Organization criteria. The assay specificity was evaluated using samples collected before the COVID-19 pandemic.
RESULTS: The sensitivity of CV2T and CV2G were low (16.7-21.4%) in days 0-6 and increased to 43.8-52.5% in days 7-13 and to 80.8-90.0% in days 14-20. The seroprevalences persisted after day 21 to days past 42 regardless of disease severity. In every day grouping, mean antibody levels were higher in severe cases than in mild cases with a significant difference in days 14-20 and days 20-27. The specificity was 97.9 % (95% CI; 92.8-99.8) for CV2T and 99.0 % (95% CI; 94.6-100) for CV2G.
CONCLUSIONS: Our results indicate a high specificity and high sensitivity at 14 days of CV2T and CV2G as antibody detection assays.

Introduction

A serious respiratory illness caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), referred to as Coronavirus disease 2019 (COVID-19), spread globally, and the World Health Organization (WHO) declared a pandemic in March 2020. The real-time reverse transcriptase polymerase chain reaction (RT-PCR) test, which detects SARS-CoV-2 viral RNA, was developed to confirm the diagnosis of COVID-19 [1]. However, sensitivity of RT-PCR tests can be affected by viral loads in specimens, timing of sample collection during the disease course, and the oligo target of RT-PCR. In fact, several studies demonstrated false negative results in RT-PCR tests [2, 3, 4].

To date, various kinds of serological tests, including lateral flow assays and fully automated antibody detection devices using Chemiluminescent microparticle immunoassay or Electro-Chemiluminescent immunoassay, have been developed to evaluate the status of past infection or to support diagnosis by RT-PCR assays [5]. Many of these serological tests received Emergency Use Authorization by the U.S. Food and Drug Administration (FDA) [6]. Although the performance of these assays have claimed to be rigorously tested, including high throughput capability and high sensitivity and specificity, the clinical significance of seroprevalence remains undetermined except for epidemiological studies described in the COVID-19 Serology Surveillance Strategy by the Centers for Disease Control and Prevention (CDC) [7].

Conversely, as vaccinations using messenger RNA (mRNA) composed of full length S1-receptor binding protein (S1-RBD) and S-antigen have been rapidly developed and used in many countries [8], the measurement of IgG antibody titers against the S1-RBD gained much more attention since the titers showed a good correlation with neutralizing antibody activity [9, 10, 11, 12]. However, these studies used in-house S1-RBD ELISA to determine IgG titer which is time consuming and laborious. Thus, the development of a fully automated reliable assay targeting the S1-RBD is indispensable since it has potential for using the IgG titer as a surrogate for the neutralizing antibody assay result.

Recently, Siemens Healthcare Diagnostics developed a fully automated assay, SARS-CoV-2 Total Antibody assay and IgG assay, that can specifically detect total and IgG antibodies against S1-RBD and was recently approved for emergency use by the FDA. In this study, we evaluated the clinical performance of this antibody assay using samples collected from patients with COVID-19 with varying disease severity at Juntendo University Hospital in Tokyo, Japan.

Materials and methods

Patient information and sample collection

Two hundred and thirty-six (236) serum samples were collected from a total of 79 symptomatic COVID-19 patients between March and August 2020. Table 1 shows the clinical information of the patients. All patients were diagnosed with COVID-19 by RT-PCR testing. RT-PCR was performed with the Light Mix Modular SARS-CoV-2 (COVID-19) N-gene and E-gene assay (Roche Diagnosis, Tokyo, Japan) or the 2019 Novel Coronavirus Detection Kit (Shimadzu, Kyoto, Japan). All clinical information was acquired by reviewing the patient charts. Based on patients’ disease severity, they were classified into two groups: Group M, equivalent to mild (without pneumonia or hypoxia) and moderate (with signs of pneumonia and SpO2 ≥ 90% in room air) cases, and Group S, equivalent to severe (with signs of pneumonia and respiratory rate of ≥30/min or SpO2 < 90% in room air) and critical (under acute respiratory distress syndrome) cases, as defined by WHO criteria. To assess specificity of the assay, samples collected in 2017 and 2018 (i.e., pre-COVID-19) were used as negative controls. The samples were stored at -80 °C until use.

Table 1

Clinical characteristics of patients with COVID-19.

Disease severity	Group M		Group S
	Mild	Moderate	Severe	Critical
Patient
Number (n = 79)	51.9% (41/79)	26.6% (21/79)	15.2% (12/79)	6.3% (5/79)
Male (%)	68.3% (28/41)	61.9% (13/21)	83.3% (10/12)	80.0% (4/5)
Age, y (mean)	23-87 (46.4)	18-75 (47.0)	46-84 (62.9)	66-86 (77.0)
Sample number (n = 236)
Days from symptom onset
0-6	22	6	9	3
7-13	23	17	14	2
14-20	16	10	7	3
21-27	14	17	5	6
28-34	8	6	5	2
35-41	5	3	2	2
42+	6	2	4	17

The research related to human use has complied with all the relevant national regulations and institutional policies, and was conducted in accordance with the tenets of the Helsinki Declaration. It has been approved by the Institutional Review Board (IRB) at Juntendo University Hospital (IRB #20-036). Informed consents were obtained from all individuals included in this study.

Anti-SARS-CoV-2 antibody testing

The Siemens SARS-CoV-2 Total Antibody assay (CV2T) and SARS-CoV-2 IgG (CV2G) were performed using serum samples on the Dimension EXL 200 Integrated Chemistry System (Siemens Healthcare Diagnostics, Delaware, USA) according to manufacturer instructions. These assays are homogeneous sandwich chemiluminescent immunoassays based on the Luminescent Oxygen Channeling Immunoassay (LOCI) that uses the S1-RBD as a target for antibody detection. Fluorescence signal intensities measured at a wavelength of 612 nm were directly correlated with antibody concentrations, which were indicated as antibody levels (QUAL). For semiquantitative CV2G assay, Ind is used instead of QUAL. One QUAL equal to one Ind. The limit of detection (LoD) for CV2G is 600 Ind, and assay linear range is between 600 and 140,000 Ind. However, CV2T is a qualitative assay, and the linear range and LoD have not been determined. Antibody levels greater than or equal to 1000 QAUL and 1000 Ind units were interpreted as positive [13]. According to the manufacture, 1000 QUAL equals to 12 BAU/ml for CV2T and 1000 Ind equals to 17 BAU/ml for CV2G.

Statistical analysis

RT-PCR was considered the gold standard for the detection of SARS-CoV-2, and the sensitivity for the assays was assessed using days from the onset of symptoms. Confidence intervals (CI) were calculated by the Clopper-Pearson method for sensitivity and specificity. Antibody levels between Group M and Group S were compared with Mann-Whitney U testing. Antibody levels between groups of days from onset of symptoms were compared by Kruskal-Wallis analysis. A two-tailed p value of <0.05 was considered statistically significant. Statistical analysis was performed by EZR [14].

Results

The precision of the antibody assay was evaluated according to the Clinical and Laboratory Standards Institute guideline EP15-A3 using QC materials consisting of positive and negative serum controls for the antibodies against S1-RBD provided by the manufacturer (Siemens Healthcare Diagnostics, Delaware, USA) and one pooled sample obtained from patients with COVID-19 at Juntendo University Hospital. Reproducibility (n = 20) was assessed by replicates in one day on one instrument. Confidence of variation (CV) was 2.2–7.0 % for CV2T and 1.3–3.8 % for CV2G. Within-laboratory reproducibility (n = 20) was assessed by duplicate assays two times a day for five days. The CV was 4.0–5.7 % for CV2T and 2.9–7.5 % for CV2G.

Specificity of the antibody assay was assessed using the pre-COVID-19 samples. Two out of 98 samples were detected as positive by CV2T, and 1 out of 100 samples was detected as positive by CV2G, yielding the specificity of 97.9 % (95% CI: 92.8–99.8) for CV2T and 99.0 % (95% CI: 94.6–100) for CV2G. Figure 1 shows the actual antibody levels of all negative control samples simultaneously measured by CV2T (panel A) and CV2G (panel B). One false positive sample with an antibody level of 4004.4 QUAL by CV2T showed 2287.7 Ind by CV2G. In contrast, the other false positive sample with 1014.3 QUAL by CV2T showed 908.6 Ind by CV2G, which was interpreted as negative since the cut off value was set to 1000 Ind. Nonetheless, all other negative controls showed negligible antibody levels, which validated high specificity of this assay.

Figure 1

Distribution of antibody levels using pre-COVID-19 samples. CV2T (A) and CV2G (B) results are shown in rank order and more than 100 QUAL and 100 Ind are labeled. Cut off (1000 QUAL and 1000 Ind) is shown in dotted line.

Next, sensitivity of the antibody assay was examined using a total of 236 samples obtained from 79 patients whose RT-PCR tests were positive for SARS-CoV-2. Table 2 summarizes the seroprevalences measured in various timing from onset of symptoms. The sensitivity increased relative to the day from onset of symptoms. The seroprevalences measured by CV2T were relatively low (16.7–52.5%) in early phase (day 0–13) in both Group M and Group S. However, the seroprevalences by CV2T were relatively high (80.8–93.5%) in day 14–20 and day 21–27 in both groups. Between day 28 and 42 and higher, Group M showed 100% seroprevalence, and Group S showed 75–100% seroprevalence. The seroprevalences measured by CV2G showed similar results as measured by CV2T. The seroprevalences were relatively low in early phase and were relatively high in day 14–27. Then, Group M showed 100% seroprevalence and Group S showed 75–100% seroprevalence in day 28–42 plus.

Table 2

Assay sensitivity of CV2T and CV2G.

Days from symptom onset (Group M, Group S)	Sensitivity (95% CI)
	CV2T		CV2G
	Group M	Group S	Group M	Group S
0-6 (n = 28,12)	21.4% (8.3–41.0)	16.7% (2.1–48.4)	17.9% (6.1–36.9)	16.7% (2.1–48.4)
7-13 (n = 40, 16)	52.5% (36.1–68.5)	50.0% (24.7–75.3)	52.5% (36.1–68.5)	43.8% (19.8–70.1)
14-20 (n = 26, 10)	80.8% (60.6–93.4)	90.0% (55.5–99.7)	80.8% (60.6–93.4)	90.0% (55.5–99.7)
21-27 (n = 31, 11)	93.5% (78.6–99.2)	90.9% (58.7–99.8)	93.5% (78.6–99.2)	90.9% (58.7–99.8)
28-34 (n = 14, 7)	100% (76.8–100)	85.7% (42.1–99.6)	100% (76.8–100)	85.7% (42.1–99.6)
35-41 (n = 8, 4)	100% (63.1–100)	75.0% (19.4–99.4)	100% (63.1–100)	75.0% (19.4–99.4)
42+ (n = 8, 21)	100% (63.1–100)	100% (83.9–100)	100% (63.1–100)	100% (83.9–100)

Interestingly, one patient in Group S retained negative seroprevalence for all collected samples. For this patient, more than one sample was collected every week and the last sample collection was in day 37. The patient suffered from Pneumocystis pneumonia, Cytomegalovirus (CMV) infection, and deep-seated mycosis, indicating the patient was immunocompromised. When this patient was excluded from the analysis, all samples collected in Group S after day 14 had positive seroprevalence and the sensitivity reached 100%.

To determine dynamic changes of antibodies in the same patients, antibody levels of the seroconverted patients who had more than three consecutive samples were plotted as a function of days from onset of symptoms. Figure 2 shows that antibody levels measured by either CV2T or CV2G increased in the early phase of the infection and gradually decreased but remained positive in our observation period in both groups (Figure 2A-D). To assess the difference of mean antibody levels among the days from onset of symptoms, Kruskal-Wallis analysis was performed, and a significant difference was observed among the days from the onset of symptoms (p < 0.001). Post hoc analysis using Steel-Dwass analysis demonstrated a significant difference in mean antibody levels in days 0–6 when compared to later days. There was also a significant difference in mean antibody levels in groups of day 7–13 when compared to other days. However, no significant difference was observed in mean antibody levels in groups later than days 14–20.

Figure 2

Chronological changes of antibody levels. Antibody levels of individuals who had more than three results are plotted by disease severity and by assays; (A) Group M, CV2T (B) Group S, CV2T (C) Group M, CV2G (D) Group S, CV2G. Red line indicates the cut off (1000 QUAL and 1000 Ind).

Figure 3 shows comparisons of mean antibody levels between Group M and Group S in each day groups to assess the correlation between disease severity and antibody levels. The mean antibody levels were significantly higher in Group S than Group M in group day 14–20 and 21–27 by both CV2T (Figure 3A) and CV2G (Figure 3B).

Figure 3

Antibody responses to SARS-CoV-2. CV2T (A) and CV2G (B) antibody levels against SARS-CoV-2 in patients compared by the disease severity at different times from symptom onset are presented in box plot. Asterisks indicate p < 0.05.

Discussion

This study was conducted to evaluate the efficacy of the Siemens SARS-CoV-2 Total Antibody assay and IgG assay using 236 serum samples from 79 patients in a university hospital in Tokyo, Japan. Although several Siemens assays have been developed, the Dimension using the LOCI method measuring IgG has not been reported [15, 16, 17, 18].

Due to the pressing need for assays detecting antibody against SARS-CoV-2, many kits have emerged and applied for the Emergency Use Authorization (EUA) by the FDA. However, little clinical information of the assays authorized for emergency use is known. Thus, it is crucial for clinical laboratories to evaluate the assay performance declared by the vendor before using EUA products. Thus, we first validated the assay precision. According to the package insert, the CVs were set to be under 10% and under 12% for repeatability and within-laboratory reproducibility, respectively. Our results were lower than this and confirmed the manufacturer's claims.

In the determination of seroprevalences, most patients were seropositive at days 14–20 and seropositivity persisted till days 42 plus. Similar to the previous studies, one immunocompromised patient failed to seroconvert [19, 20]. This may indicate that detectable amounts of antibodies cannot be produced in immunocompromised patients.

The sensitivity after 14 days from onset of symptoms in our study nearly matched the package insert (CV2T: 100% and CV2G: 100%) [13]. However, the sensitivity reported by the manufacturer was higher in days 0–6 (CV2T: 66.67%; CV2G: 61.9%) and days 7–13 (CV2T: 91.30%; CV2G: 92.3%) compared to our results. A study using the Dimension Vista that shares the same technologies with Dimension EXL 200 in the U.S. reported a sensitivity of 78% by CV2T after 14 days from onset of symptoms, which that is lower than our study [18]. This inconsistency might occur due to the locality or the character diversity of the sample such as ethnicity, age, gender, and underlying diseases.

In the detection of SARS-CoV-2 antibody, cross reaction with other coronaviruses, such as severe acute respiratory syndrome coronavirus (SARS) and Middle East respiratory syndrome coronavirus (MERS), has been reported [21]. Moreover, cross reaction with CMV has also been reported [22]. However, upon reviewing the patients’ charts, we did not find evidence of previous exposure to SARS or MERS, which are quite rare disease in Japan. Using the pre-COVID-19 samples in the specificity assay, our study showed high specificity in both CV2T (97.9%) and CV2G (99.0%), which is essential for detecting antibodies. However, our specificity results were lower than the Zilla et al. study which had high (100%) specificity using 184 samples in the total antibody assay using the Siemens Vista [18]. The manufacturer also claims 99.86% specificity for CV2T and 100% specificity for CV2G using 1529 and 1509 healthy samples collected before COVID-19 pandemic, respectively. Although these discordances may be due to difference in samples, the difference may be negligible.

Comparing Group M and Group S, the antibody levels were significantly higher in Group S in the second and third weeks. These results are similar to a study by Long et al., which compared severe and non-severe groups from three hospitals in China although significant difference was obtained only in days 8–14 [23]. Several studies also demonstrated positive correlations between disease severity and antibody levels [9, 17, 24, 25] while others did not [20, 26]. These contradicting results may have occurred due to difference in sample size, the target for antibody detection (S protein or N protein or S1-RBD), or the genetic diversity of patients.

The results of CV2T and CV2G are not comparable because the CV2G is semi-quantitative assay, and the CV2T is qualitative assay. In our study, the CV2G assay seems to yield higher antibody levels than the CV2T assay in both Group S and Group M. However, further investigation was not able to perform because of the lack of reference material.

The CDC recommend serology testing for suspectable COVID-19 patients who are PCR negative and at least two weeks post onset of symptoms or for seroprevalence survey [27]. Importantly, the CDC also pointed out that antibody assays with positive result targeting the S antigen and negative result targeting other antigen suggest previous infection or reactions to vaccinations [27]. COVID-19 patients develop several types of antibodies against viral proteins, and as time passes the antibodies decrease and may become seronegative. Thus, negative N-specific antibodies do not deny post exposure. If a person has antibodies against N, E, or M proteins, not only S proteins, it would indicate post exposure to SARS-CoV-2. On the other hand, S-specific antibodies can be developed by exposure to SARS-CoV-2 or vaccination because most vaccines are expected to elicit S-specific antibodies. Hence, the assessment of optimal timing to measure S-specific antibodies that can act as neutralizing antibodies, not N-specific antibodies, is essential to evaluate effectiveness of the vaccination.

Concordant with the previous studies [18, 23, 28], our data showed that sensitivity reached to almost 100% in days 14–20. Also, a statistical analysis demonstrated antibody levels significantly increased during the first 14 days. Chew et al. also reported antibody levels were higher in the second week after onset of symptoms compared to the early phase [29]. Taken together, two to three weeks can be suitable timing for detecting seroconversion [9, 10, 11, 12].

This study has some limitations: 1) the present study was performed in a single university hospital, and number of samples were relatively small; 2) since this was a retrospective study, the sampling time varies by patients. Thus, we could not track the change of antibody level in all day groups; 3) clarification of the onset of symptoms was merely dependent upon the clinical charts, which was subjective and might have discrepancy with true onsets of viral infection; and 4) we have not performed neutralizing assay since it requires live virus and BSL-3 equipment. Thus, we were not able to determine whether the detected antibodies act as neutralizing antibodies. Some studies have demonstrated that the avidity assay detects the binding strength of antibodies as an alternative to neutralizing assay in determining protective humoral immunity [30, 31], which requires further investigation; 5) the sole use of Dimension EXL 200 is another potential limitation. There are several assays to detect SARS-CoV-2 S1-RBD-specific antibodies with different measurement principles [15, 16, 17, 18], which can lead to different results.

In conclusion, the Siemens CV2T and CV2G could detect the S1-RBD specific antibodies against SARS-CoV-2 at approximately 14 days after onset with a high sensitivity. Though current use of this assay as a screening tool is limited as described in the CDC recommendations, it may be useful to determine the post vaccine effect and the necessity of booster shots since they have a potential to detect neutralizing antibodies. However, a complete understanding of the antibody kinetics of these two assays is required for use as a predictor of vaccine response. This requires larger samples from various regions and prospective studies over longer timeframes, such as before and after vaccination.

Declarations

Author contribution statement

Gene Igawa: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Wrote the paper.

Takamasa Yamamoto: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data.

Yuna Baba, Konomi Shinozuka, Maiko Yuri: Performed the experiments.

Mitsuru Wakita: Conceived and designed the experiments; Contributed reagents, materials, analysis tools or data.

Shigeki Misawa, Takashi Miida: Contributed reagents, materials, analysis tools or data.

Tomohiko Ai: Analyzed and interpreted the data; Wrote the paper.

Yoko Tabe: Conceived and designed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Data availability statement

Data will be made available on request.

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.