Clinical Validation and Performance Evaluation of the Automated Vitros Total Anti-SARS-CoV-2 Antibodies Assay for Screening of Serostatus in COVID-19.

OBJECTIVES: Evaluation of serostatus against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as an important tool in identification of exposure to coronavirus disease 2019 (COVID-19). We report on the validation of the Vitros Anti-SARS-CoV-2 Total (CoV2T) assay for qualitative serologic testing of SARS-CoV-2 antibodies.
METHODS: We performed validation studies according to Commission of Office Laboratories Accreditation guidelines, using samples previously tested for SARS-CoV-2 by reverse transcription-polymerase chain reaction (RT-PCR). We evaluated precision, analytical interferences, and cross-reactivity with other viral infections; evaluated concordance with molecular and other serologic testing; and evaluated seroconversion.
RESULTS: The Vitros CoV2T assay exhibited acceptable precision and did not exhibit cross-reactivity with other acute respiratory virus infections. The CoV2T assay exhibited 100% negative predictive agreement (56/56) and 71% positive predictive agreement (56/79) with RT-PCR across all patient samples and was concordant with other serologic assays. Concordance with RT-PCR was 97% more than 7 days after symptom onset. The CoV2T assay was robust to icterus and lipemia but had interference from significant hemolysis.
CONCLUSIONS: The Vitros CoV2T assay was successfully validated in our laboratory. We anticipate it will be a useful tool in screening for exposure to SARS-CoV-2; however, the use of the CoV2T and other serologic assays in the clinical management of patients with COVID-19 is unknown and must be evaluated in future studies. © American Society for Clinical Pathology, 2020. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

Multiple serologic assays for detection of anti–SARS-CoV-2 antibodies have received US Food and Drug Administration emergency use authorizations, but few data have been published on the performance of these assays.

The Vitros Anti–SARS-CoV-2 Total assay is a total antibody test to be used as a serologic screen for exposure to COVID-19. This assay is comparable to other automated serologic tests.

We identified that the Vitros Anti–SARS-CoV-2 Total assay has positive interference from hemolysis at concentrations greater than 250 mg/dL that can produce false-positive results.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is a novel coronavirus identified in China in late 2019.[1] The resultant disease, coronavirus disease 2019 (COVID-19), has a variety of presentations from mild disease to severe pneumonia requiring mechanical ventilatory support, as well as thrombotic strokes, multi-inflammatory syndrome, and others.[2] COVID-19 has become a global pandemic and continues to spread, producing substantial morbidity, mortality, and economic impact.[3] Accurate epidemiologic information is important for the development of effective public health measures for containment and mitigation of COVID-19; however, case identification has been complicated by the wide range of clinical presentations of COVID-19. The mainstay of diagnostic testing for COVID-19, as with most respiratory viral infections, is molecular detection of SARS-CoV-2 viral RNA in respiratory swab specimens. Molecular diagnostic testing is preferred given its high specificity and the ability to yield a rapid diagnosis during acute infection, but preanalytical challenges (eg, nasopharyngeal swab sampling) and analytical challenges have limited the utility of molecular testing as a means to screen for exposure to SARS-CoV-2.[4,5] The abundance of detectable nucleic acid also decreases over time; other studies have suggested the RNA-positive rate may decline to under 30% by 3 weeks after symptom onset.[6] Consequently, the extent of COVID-19 infection remains unknown in many populations, limiting the ability to assess case fatality rates and hampering efforts to effectively quarantine infected individuals and limit spread.

Serologic testing is used for the identification and management of many infectious diseases to provide evidence that a person has had exposure to a pathogen and mounted an immune response. The laboratory workflow for serologic testing is typically simpler, faster, and less expensive than that of molecular testing, and specimen collection is more reproducible, which offers advantages for widespread screening. Serologic testing has been suggested as a means of surveillance to determine actual numbers of COVID-19 infections, which can subsequently inform public health strategies. Previous studies have indicated that seropositivity in COVID-19 begins to occur approximately 7 days after symptom onset, although how long seropositivity remains after recovery and to what extent it indicates immunity to reinfection are not yet established.[7]

Recommendations from the Centers for Disease Control and Prevention and the US Food and Drug Administration (FDA) indicate the use of total (IgG and IgM) anti–SARS-CoV-2 antibodies as evidence of previous viral exposure.[8] Only a few serologic assays have been granted emergency use authorization (EUA) by the FDA for this purpose, but limited data are available on the analytical or clinical performance of these tests. In this study, we describe validation of one of the first assays to receive EUA on an automated platform, the Vitros Anti–SARS-CoV-2 Total (CoV2T; Ortho Clinical Diagnostics) antibody assay, for screening of previous exposure to SARS-CoV-2 in our patient population.

Materials and Methods

The Vitros CoV2T assay detects total IgG and IgM directed against SARS-Cov-2 and was evaluated for use on the Vitros 5600 automated chemistry analyzer (Ortho Clinical Diagnostics). The CoV2T assay uses a solid-phase SARS-CoV-2 spike protein antigen to capture antibodies in the patient specimen and horseradish peroxidase–labeled recombinant SARS-CoV-2 antigen as a detection reagent. The assay is qualitative and reports results as reactive or nonreactive based on a manufacturer-defined signal-to-cutoff ratio (S/Co) of 1.00, with reactive values falling above this decision limit and nonreactive values below. Numerical values for S/Co are provided by the instrument but are not used in patient reports.

Specimens for validation were obtained with informed consent from healthy volunteers and known patients with COVID-19 under an approved protocol from our local institutional review board (number H47459, Federalwide Assurance number FWA 00000286). Known positive patients were previously diagnosed with COVID-19 by reverse transcription–polymerase chain reaction (RT-PCR) methods at our hospital or by molecular methods at other local laboratories within our large academic medical center. The time between PCR positivity and serologic testing ranged from 0 to 35 days. Patient specimens were collected by venipuncture into K2EDTA tubes or serum separator tubes and processed on receipt by the laboratory, with plasma or serum aliquoted and stored at 4°C until analysis (up to 5 days).

Precision studies were performed using a vendor-provided positive and negative control and a known negative sample and a known positive patient sample. Intra-assay precision was assessed by running the manufacturer-provided positive and negative control and 1 positive and negative sample 10 times in a single run. Interassay precision was assessed by running these samples on separate runs 5 times a day over 2 days. Precision was assessed as the percentage of coefficient of variation (CV) using numerical S/Co values provided by the instrument.

Accuracy studies were performed using 57 healthy volunteers who were negative for SARS CoV-2 by RT-PCR and who had no known exposure, travel history, or symptoms of COVID-19 and 79 patient samples that were positive for SARS CoV-2 by RT-PCR. Samples were tested on different days and by different operators. Accuracy was assessed as concordance with the positive or negative status of the specimen as assessed by molecular (RT-PCR) testing.

A subset of these specimens and additional specimens (totaling 80 negative and 48 positive samples tested by PCR) were also tested for concordance of the CoV2T assay with a semiquantitative SARS-CoV-2 IgG and IgM assay (SARS-CoV2 IgG or IgM ELISA; Ansh Laboratories) on the Dynex DS2 (Dynex Technologies), using the manufacturer’s cutoff value as a threshold for positive or negative results. An additional 18 samples were split and tested for concordance with results from a reference laboratory (Viracor Eurofins IgG and IgM panel).

Seroconversion in our patient population was assessed by correlation of chart review of 55 patients known to be positive for SARS-CoV-2 by RT-PCR and known date of symptom onset with sample reactivity by the CoV2T assay. Specimens were grouped by the number of days elapsed since the first reported symptom per patient history. Sensitivity was determined in samples that were taken after day 7 of symptom onset.

Analytical specificity was assessed by testing 14 different patient samples known to be positive for other viruses by molecular testing (including influenza A virus, influenza B virus, respiratory syncytial virus, adenovirus, rhinovirus, or other coronaviruses) but negative for SARS-CoV-2 by RT-PCR. Specimens for SARS-CoV-2 serologic testing were collected concurrently with molecular testing specimens.

Interference testing was also performed by spiking known negative or positive samples with known concentrations of hemoglobin, conjugated bilirubin, and triglyceride-rich lipid (Sun Diagnostics). Results were considered acceptable if spiked sample reactivity was concordant with the neat sample.

The effect of tube type was assessed by collecting specimens from 5 volunteers into serum separator or K2EDTA tubes and measuring the resultant serum or plasma on the CoV2T assay. Concordance between tube types was assessed, with results considered acceptable if concordant.

Statistical analysis was performed in the EP Evaluator. Results are given as mean (SD).

Results

The Vitros CoV2T assay exhibited acceptable precision. The CoV2T is a qualitative assay and does not report numerical results for patient use, but we examined the CV using the S/Co. In intra-assay experiments, a known negative specimen was reported as nonreactive over 10 replicates, with an S/Co CV of 10.9%. A known positive specimen was reported as reactive, with an S/Co CV of 1.6%. Interassay experiments yielded CVs of 9.7% for a negative specimen and 3.3% for a positive specimen .

The CoV2T assay was concordant with negative and known positive cases of COVID-19 at our and neighboring institutions. Of healthy volunteer specimens, 100% (57/57) of tests were nonreactive by the CoV2T assay, with an S/Co (SD) of 0.38 (0.26) reported for these specimens. Of patient specimens confirmed positive for SARS-CoV-2 by PCR, 70.9% (56/79) of all specimens tested were reactive by CoV2T assay . S/Co (SD) values were 0.13 (0.12) in the nonreactive subset (range, 0.02-0.50) and 144 (146) in the reactive subset of the positive population (range, 1.43-540). Seroconversion in our patient population appeared to occur on days 4 to 7 after the onset of initial symptoms, and 96% of specimens collected on or after day 4 yielded a reactive result . Only 6 of 17 (35%) specimens collected in the first 3 days of symptom onset were reactive by the CoV2T assay.

Sensitivity (positive predictive agreement) of the assay 7 days after onset of symptoms was 100%, with the exception of 1 specimen that was reported to be positive by PCR at an outside location but was tested as negative in our hospital. Specificity (negative predictive agreement) was found to be 100% because neither healthy volunteers (n = 57) nor patients under investigation who tested negative by RT-PCR for SARS-CoV-2 (n = 35) were reactive by the CoV2T assay.

There is no reference standard for serologic testing of SARS-CoV-2, but we sought to compare the CoV2T assay with other serologic methods for concordance. Eighteen split specimens (6 nonreactive, 12 reactive) were tested at a reference laboratory using a qualitative assay and were 100% concordant with the CoV2T assay. For the semiquantitative IgG method, the CoV2T assay was 91.2% concordant for negative samples (n = 80) and 91.6% concordant for positive samples (n = 48) .

Specimens from 14 patients with acute infections, previously tested to be negative for SARS-CoV-2 by RT-PCR but positive for another respiratory viral infection by molecular analysis, were nonreactive by the CoV2T assay. Interference studies identified no changes in sample reactivity when spiked with conjugated bilirubin or triglyceride-rich lipid and no changes in positive sample reactivity when spiked with hemoglobin. However, positive interference that affected sample reactivity was noted in nonreactive specimens spiked with hemoglobin at concentrations greater than 125 mg/dL .

The CoV2T instructions for use indicate that either serum or K2EDTA plasma may be used for analysis. Across 5 specimens, we verified that reactivity of all specimens collected were concordant regardless of tube type used (data not shown).

Discussion

The availability of reliable serologic testing for SARS-CoV-2 will be an important public health tool to mitigate the impact of COVID-19. Several serologic assays have received EUA from the FDA, and in this article, we describe the validation of the Vitros CoV2T total antibody assay in our hospital laboratory. The CoV2T assay exhibited good analytical precision, good sensitivity (concordance with RT-PCR testing for SARS-CoV-2), and good concordance with other serologic assays. We also confirmed that the assay had good specificity, was robust to most common analytical interferences, and exhibited similar performance between serum and K2EDTA plasma samples. In addition, discrimination between reactive and nonreactive specimens was adequate, as S/Co values reported for nonreactive and reactive specimens by the CoV2T assay did not overlap. Studies are ongoing at our institution to further evaluate the performance of the assay around the manufacturer’s cutoff value, and future testing strategies using an orthogonal serologic method to evaluate borderline reactive specimens are being evaluated.

We noted that nonreactive specimens are prone to positive interference from lower concentrations of hemoglobin than indicated in the manufacturer’s instructions for use—that document indicates no interference at 1,000 mg/dL, whereas we identified interference at concentrations as low as 250 mg/dL (Table 5).[9] We note that this can affect specimen reactivity and suggest that samples with significant hemolysis be rejected for testing. Our other analytical specificity studies did not identify cross-reactivity of the CoV2T assay with cases of acute infection with respiratory viruses or coronaviruses, but these studies were limited by the small number of available validation specimens in our laboratory and do not address possible cross-reactivity with convalescent plasma in these cases. Further testing, particularly with specimens banked before the emergence of SARS-CoV-2, will be useful to identify the possibility of cross-reactivity from other related viruses in this assay. Other studies have indicated low cross-reactivity of their serologic assays with other coronaviruses using specimens taken before the emergence of SARS-CoV-2, but this might differ slightly between assay manufacturers.[10] Until the possibility of cross-reactivity is further evaluated, a positive antibody test using the CoV2T assay should be considered only presumptively exposed to SARS-CoV-2.

Table 5

Analytical Specificity Studies With Interference and Other Respiratory Virus–Positive Specimens

Interferenta	Negative Sample	Positive Sample
Positive for influenza A, influenza B, respiratory syncytial virus, adenovirus, rhinovirus, or other coronaviruses (n = 14)	0.15 (0.15; nonreactive)	None
Hemolysate, mg/dL (index value)
125 (272)	Nonreactive	Reactive
250 (930)	Reactive	Reactive
500 (>1,000)	Reactive	Reactive
Conjugated bilirubin, mg/dL (index value)
30 (13)	Nonreactive	Reactive
40 (>25)	Nonreactive	Reactive
Triglyceride-rich lipid, mg/dL (index value)
250 (29)	Nonreactive	Reactive
2,000 (221)	Nonreactive	Reactive

aFor interference studies, known concentrations of interferent were spiked into either a known nonreactive or reactive sample. The instrument index values are provided in parentheses. For each concentration of interferent, the specimen reactivity is reported. For other virus-positive specimens, patient specimens previously tested positive for another virus by molecular methods were tested by the Vitros Anti–SARS-CoV-2 Total assay for reactivity.

Our studies demonstrated excellent specificity (100%) with sensitivity of 70.9% in our complete validation sample set, with samples collected from 0 to 35 days after onset of symptoms (Table 2). The single specimen in our validation set that was discordant in a patient who tested positive (by PCR at >13 days after onset, negative by CoV2T) was considered positive on the basis of an outside test result. When tested using our in-house RT-PCR assay, with a detection limit of as few as 40 copies of viral RNA per reaction, this specimen was negative for SARS-CoV-2. However, these data are preliminary. There is no standard reference method for serologic testing with which to compare the CoV2T results, and the predictive value of the CoV2T assay when used as a screening test will be dictated by the actual prevalence of COVID-19 in our population and the timing of specimen collection relative to the onset of symptoms. The performance of the CoV2T assay appears to be similar to that reported previously for other serologic assays for COVID-19, with other methods internationally yielding sensitivity of 73.3% to 100% and specificity of 90.6% to 100%.[10-15] Nevertheless, performance characteristics are likely to vary based on epitope specificity and selected cutoff value for positive or reactive specimens. Further clinical performance studies are ongoing in our laboratory.

Table 2

Concordance of the CoV2T Assay With Positive or Negative Results by RT-PCR for SARS-CoV-2

	RT-PCR Positive	RT-PCR Negative	Total
CoV2T reactive (Ab positive)	56	0	56
CoV2T nonreactive (Ab negative)	23	57	80
Total	79	57	136

Ab, antibody; Cov2T, Vitros Anti–SARS-CoV-2 Total assay; RT-PCR, reverse transcription–polymerase chain reaction; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

The higher overall specificity of the CoV2T assay relative to its overall sensitivity in our studies likely reflects the fact that a patient who tests positive for SARS-CoV-2 by RT-PCR may be in the early stages of infection and may not yet be seropositive. Studies of our patient population suggest that seropositivity as measured by the CoV2T assay may occur within the first 7 days and show 97% sensitivity for SARS-CoV-2 at or beyond 7 days after the onset of symptoms (n = 31). These data align with the existing literature, which reports that seropositivity for anti–SARS-CoV-2 antibodies occurs in only 50% of patients by 7 days after peak viral replication.[7,11] The clinical utility of serologic testing in the management of patients with COVID-19 is not yet known; however, we concur that serologic testing is likely to be of limited use for initial detection and diagnosis of symptomatic infections, largely because of variability in time between infection and detectable immune response. In addition, in patients with mild or asymptomatic disease, seropositivity may be delayed beyond this time frame; in immunocompromised patients, any testing for anti–SARS-CoV-2 antibodies may not be reliable. Consequently, molecular testing methods for viral RNA detection should be preferred as a diagnostic method.

At the time of writing, the utility and interpretation of serologic testing for SARS-CoV-2 antibodies is evolving. Serology testing will likely be useful for epidemiologic purposes, for investigational virology, and for correct attribution of the etiology of post–COVID-19 secondary syndromes.[8,16-18] It is not yet clear whether tests such as the CoV2T assay correlate with neutralizing antibody titers, what antibody titer is considered protective, whether antibodies confer immunity at all, and how long immunity might persist. Moreover, the CoV2T assay is qualitative, but confirmatory testing strategies with the ability to provide quantitative or isotype-specific information may be useful to distinguish between acute infection and convalescence. Consequently, it is especially important to stay abreast of current literature and testing guidelines, which will ultimately define the utility and interpretation of serologic testing.

In summary, we validated the Vitros CoV2T assay as a qualitative screening method for seropositivity against SARS-CoV-2 in our laboratory. We anticipate that this assay will be a useful method of determining true COVID-19 infection rates in our population and may be useful as a predictor of immune status against COVID-19. Our studies highlight the importance of careful validation of EUA assays, and while serologic testing should be evaluated and implemented judiciously, these assays should ultimately provide a useful tool for surveillance and containment efforts in the ongoing COVID-19 pandemic.

Table 1

Intra- and Interassay Precision Study Results

Sample	Intra-assay S/Coa	Interassay S/Co
Nonreactive (S/Co <1.0)	0.065 (0.007); 10.9	0.171 (0.017); 9.7
Reactive (S/Co >1.0)	245 (3.9); 1.6	3.051 (0.099); 3.3

S/Co, signal-to-cutoff ratio.

aValues are mean (SD); percentage of coefficient of variation.

Table 3

Antibody Reactivity by CoV2T Assay in Patients Positive for SARS-CoV-2 by RT-PCR, Grouped by Number of Days Since the First Reported Symptom

Days After Onset	No.	Reactivea
<3	17	35 (6/17)
4-7	7	86 (6/7)
8-13	8	100 (8/8)
>13	23	96 (22/23)b

Cov2T, Vitros Anti–SARS-CoV-2 Total assay; RT-PCR, reverse transcription–polymerase chain reaction; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

aValues are presented as percentage (frequency). Positive predictive agreement at >7 days after symptom onset was 98%.

bOne patient in the >13-day group was reported positive at an outside hospital but tested negative by RT-PCR at our institution.

Table 4

Method Comparison With Semiquantitative SARS-CoV-2 IgG and IgM ELISA

	IgG		IgM
CoV2T	Positive	Negative	Positive	Negative	Total
Reactive	44	7a	28	23	51
Nonreactive	4	73	1	76	77
Total	48	80	29	99	128

Cov2T, Vitros Anti–SARS-CoV-2 Total assay; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

aReactive specimens by CoV2T that were not IgG positive were not found to be positive for IgM by semiquantitative ELISA. All data from split sample testing with Viracor reference laboratories were concordant.