Pushing beyond specifications: Evaluation of linearity and clinical performance of the cobas 6800/8800 SARS-CoV-2 RT-PCR assay for reliable quantification in blood and other materials outside recommendations.

BACKGROUND: The ongoing SARS-CoV-2 pandemic presents a unique challenge to diagnostic laboratories. There are preliminary studies correlating qRT-PCR results from different materials to clinical outcomes, yet, comparability is limited due to the plethora of different assays used for diagnostics. In this study we evaluate clinical performance and linear range for the SARS-CoV-2 IVD (cobas6800/8800 system, a fully automated sample-to-result platform) in different clinically relevant matrix materials outside official specifications.
METHODS: Assay performance was assessed in human plasma, BAL/BL and transport medium following chemical inactivation. For analytical evaluation, respective matrix materials were spiked with SARS-CoV-2 RNA in ten-fold dilution series. The efficacy of chemical inactivation by guanidine hydrochloride solution was confirmed in cell culture infectivity experiments. For correlation, a total of 289 predetermined clinical samples including respiratory swabs, plasma and lower respiratory tract specimens were subjected to the SARS-CoV-2 IVD test and results were compared.
RESULTS: The SARS-CoV-2 IVD showed excellent linearity over four to six log steps depending on matrix material. Chemical inactivation resulted in a reduction in plaque forming units of at least 3.5 log steps, while having no significant impact on assay performance. Inter-run consistency from three different testing sites demonstrated excellent comparability of RT-PCR results (maximum deviation was 1.53 CT). Clinical evaluation for respiratory swabs showed very good agreement with the comparator assay (Positive agreement 95.7 %, negative agreement 98.9 %).
CONCLUSION: The SARS-CoV-2 IVD test for the cobas6800/8800 systems offers excellent linear range and inter-run consistency for quantification of SARS-CoV-2 RNA in different matrices outside official specifications.

Introduction

The novel coronavirus SARS-CoV-2 has been subject of extensive study since its emergence in late December 2019 [1,2]. Diagnostic RT-PCR was quickly determined as the gold standard for detecting the new pathogen in patients, in large part due to the rapid dissemination of complete virus sequences from the assumed origin of the outbreak in China and consecutive publication of specific PCR assays [3,4]. Besides merely confirming the diagnosis, there exists evidence for a correlation between viral loads and clinical outcomes for various respiratory viruses, including Influenza and the original SARS-CoV [5,6]. In the case of SARS-CoV-2 there has already been a flurry of publications describing viral load dynamics in different clinical specimens [7,8]. It can be assumed that this topic will further grow in relevance in the coming months.

Reliable quantification of RT-PCR signals is highly dependent on assay specifications and reference materials [9]. In the absence of an international standard to serve as reference, quantification of SARS-CoV-2 RNA is ultimately based on a variety of different methods and standards used in the individual labs [8,10], resulting in inherently poor comparability and reproducibility between testing sites. However, the increasing availability of commercial SARS-CoV-2 RT-PCR kits for widely used fully automated systems offers the opportunity to generate highly consistent PCR results which can then be used for quantification.

The cobas6800 system is a fully automated sample-to-result high-throughput RT-PCR platform, capable of performing 384 tests in an 8 h shift and requiring minimal hands-on time [11]. Until recently, laboratory developed tests (LDT) using the open mode (utility channel) had to be employed to use the system for SARS-CoV-2 testing [10]. However, Roche has since received “Emergency use authorization” by the FDA for their own SARS-CoV-2 IVD assay and have made the test available commercially [12]. This provides a common ground across different testing sites for quantification of RT-PCR data, especially when taking the high inter-run consistency of the instrument into account, as previously demonstrated for other clinical targets [13].

The aim of this study was to provide clinical evaluation of the new SARS-CoV-2 IVD assay by Roche and further validate clinical specimen types outside specifications for use in clinical studies. In this context, efficiency and linear range was to be determined in different clinical materials. Lastly, comparability of results was to be evaluated using a multicenter approach for inter-run variability.

Materials and methods

Cell culture and virus quantification

Vero cells (ATCC CCL 81) were propagated in DMEM containing 10 % FCS, 1 % Penicillin/ Streptomycin, 1 % l-Glutamine, (200 mM), 1 % Sodium pyruvate and 1 % non-essential amino acids (all Gibco/ Thermo Fisher, Waltham, USA). A SARS-CoV-2 isolate was rescued from an oropharyngeal swab as described previously [14]. For virus quantification, plaque-assays were performed as follows: A tenfold serial dilution series of the virus containing solution was incubated on Vero cells seeded in 24-well plates (500 μL per well). After adsorption at 37 °C for 1 h, cells were washed once with PBS and overlaid with DMEM containing 1% methylcellulose to prevent virus spreading within the supernatant. After 5 days of incubation at 37 °C, cells were fixed in 4 % formaldehyde (30 min at room temperature), washed once with H2O and stained with crystal violet solution. Virus titers in initial (stock) solutions were assessed by counting plaques.

Assessment of the virus inactivation capability of 40 % guanidinehydrochloride

In order to determine the virus inactivation capability of the Roche PCR Media Kit (≤ 40 % guanidinehydrochloride in Tris−HCl), 100 μL of the SARS-CoV-2 stock solution (with an estimated 3 × 10 [7] PFU/mL) were diluted 1:10 in either UTM or E-swab medium (modified Amies medium in Tris−HCl). 500 μL of each dilution were supplemented 1:1 with Roche PCR-Media, the remaining 500 μL were supplemented 1:1 with either UTM or E-swab medium. After 30 min of incubation at room temperature, ten-fold serial dilution series of each mixture were used for plaque assays as described above.

Cobas 6800 SARS-CoV-2 IVD test setup

The SARS-CoV-2 IVD dual-target test for the cobas6800/8800 system was obtained from Roche and used according to instructions issued by the manufacturer. Target-1 (ORF1ab) and Target-2 (E gene) properties were analysed separately, though the entire assay was deemed positive as long as one Target returned a positive result. Apart from loading the ready-to-use SARS-CoV-2 IVD-test cartridges onto the device, there are no manual steps required in the assay setup.

Assessing linearity in different materials

For determining linear range in different matrix materials, an initial 1:20 dilution of cell culture supernatant containing SARS-CoV-2 (isolate HH-1) was prepared using UTM (with or without chemical inactivation) or normal human plasma (SARS-CoV-2 RNA negative) as diluent. For BAL/BL, clinical specimens previously tested PCR-negative for SARS-CoV-2 were pooled to serve as matrix material. The initial stock was used to generate tenfold dilution series, which were subjected to the SARS-CoV-2 IVD test in 8 repeats for each step.

Comparing clinical samples

All tested clinical samples were predetermined positive or negative in routine diagnostics using the SARS-CoV-2 UCT (Utility Channel Test) on the cobas6800 system. This method was described in detail in a different study, see Pfefferle et al. [10]. Briefly, a published primer/probe set by Corman et al. [3] targeting the viral E-gene was modified and adapted for use with the cobas omni Utility Channel on the cobas 6800 instrument. This protocol served as comparator assay in this study. Oropharyngeal swabs or nasopharyngeal swabs were collected using eSwab collection kits (Copan, Italy). All samples were pre-treated by adding 1:1 “cobas PCR media” (Roche, ≤ 40 % guanidine hydrochloride in Tris−HCL buffer) for safe handling. To obtain plasma, EDTA-blood samples were centrifuged, followed by transfer of the plasma into sterile sample tubes. Lower respiratory tract specimens were bronchoalveolar lavage (BAL), bronchial lavage (BL) or tracheal aspirates (TA). Input volume was identical for both tests.

A total of 180 respiratory swab samples, 53 positive plasma samples and 56 positive BAL/BL/TA samples were retrieved from storage at −20 °C (< 1 month) and subjected to the SARS-CoV-2 IVD test. Clinical samples were not retested with the UCT following thawing. CT values of Target-1 (ORF1ab) and Target-2 (E gene) were compared to the prior result of the comparator assay (targeting the viral E gene).

This work was conducted in accordance with §12 of the Hamburg hospital law (§12 HmbKHG). The use of anonymized samples was approved by the ethics committee, Freie und Hansestadt Hamburg, PV5626.

Inter-run variability and comparability of results

Inter-run variability was compared between a total of four different instruments, located at 3 different testing sites. Triplicates of a solution containing diluted SARS-CoV-2 infected cell culture supernatant were prepared in a similar manner as described in section 3.4. Concentrations of SARS-CoV-2 RNA were estimated to be within linear range of the assay. Spiked sample sets were sent to “Labor Limbach”, Heidelberg and “Labor Stein”, Moenchengladbach (transported at 4 °C) for analysis with the SARS-CoV-2 IVD test using their own cobas6800 instruments. Each set contained three identical UTM samples and three identical plasma samples, as well as a negative control. CT values were reported and compared. Only 11/12 measurements returned results for each set (UTM and plasma) due to a system crash and loss of material at one of the sites.

Results

Supplementing swab medium with guanidinehydrochloride solution results in significant reduction of plaque formation

To confirm effective reduction of infectivity in clinical specimens, cell culture supernatants containing high concentrations of SARS-CoV-2 (HH-1, approximately 7.0E + 7 PFU/mL) were initially diluted (1:20) in UTM and eSwab medium, with and without adding 1:1 with Roche cobas PCR media (≤ 40 % guanidine hydrochloride in Tris−HCL buffer) to the mix. Stocks treated with guanidine hydrochloride solution showed a reduction in plaque forming units of at least 3.5 log steps compared to control (Fig. 1 ). No significant differences were observed when comparing PCR efficiency in UTM to UTM supplemented with guanidine hydrochloride solution (Fig. 2 ).

Fig. 1

Reduction of plaque forming units (PFU) by pretreatment with guanidine hydrochloride solution.

A stock solution of cell culture supernatant containing SARS-CoV-2 (HH-1) was diluted 1:20 in UTM or eSwab medium alone or supplemented 1:1 with cobas PCR Media. A 10-fold dilution series was prepared and 200 μL added to Vero cells grown to fluency in a 24 well plate.

Fig. 2

Linear range of Target-1 and Target-2 in different matrix materials.

Linear range of Target-1 and Target-2 in different matrix materials. A stock solution of cell culture supernatant containing SARS-CoV-2 (HH-1) was used to prepare a 10-fold dilution series within indicated materials. A total of 8 repeats was tested per dilution step, each measurement is represented by an individual dot. X-axis: Plaque forming units. Y-axis: PCR cycle threshold. Black dots: measurements within linear range, considered for trendline and slope. Faded dots: measurements outside linear range, not considered for trendline and slope. CPM: cobas PCR media (≤ 40 % guanidine hydrochloride in Tris−HCl). ND: “Not detected”.

Linear range for different matrix materials using cell culture stocks

Linear range of the assay was assessed in a variety of matrices, including swab medium with and without chemical inactivation, human plasma and BAL/BL. Both Targets exhibited slopes in the range of 3–3.3 CT in all tested materials except plasma, indicating very good efficiency (Fig. 2). Slopes were markedly higher in plasma, approximately 4 CT for both Targets. The same behavior was observed with the SARS-CoV-2 UCT (Supplementary Fig. 1).

Both assays demonstrated excellent linearity over a range of 4–6 log steps (6 log in UTM incl. guanidine hydrochloride, 5 log in plasma, 4 log in BAL/BL), with the Target-2 assay performing slightly better and being highly linear up to a CT of 35.

Clinical evaluation shows high agreement between the SARS-CoV-2 IVD and UCT results for detection of viral RNA in swab samples

For clinical validation, a set of predetermined samples was subjected to testing with the SARS-CoV-2 IVD and results were compared to the UCT. The SARS-CoV-2 IVD result was reported as “positive” as long as one of the two Targets returned a positive result. A total of 180 respiratory swabs (predetermined positive: 93; predetermined negative: 87) tested with the assay resulted in robust positive and negative agreement of 95.7 % and 98.9 % respectively (Table 1 ).

Table 1

Clinical evaluation of the SARS-CoV-2 IVD assay for 180 predetermined respiratory swab samples.

		UCT (comparator assay)
		positive	negative	Total
SARS-CoV-2 IVD	positive	89	1	90
	negative	4	86	90
	Total:	93	87	180

Table 1: A total of 180 predetermined respiratory swab samples (eSwab supplemented 1:1 with cobas PCR Media) were retrieved from storage (< 1 month, −20 °C) and tested with the SARS-CoV-2 IVD test. UCT: Utility Channel Test.

All discrepant positive (n = 1) and negative samples (n = 4) were at high CT values (> CT 30), indicating very low concentrations of viral RNA.

SARS-CoV-2 IVD results in clinical samples within linear range show excellent correlation with the comparator assay

For positive swab samples, as well as a further 53 predetermined positive plasma samples and 56 lower respiratory tract samples, CT values of the IVD test were compared to the UCT, which had previously been used to analyze these types of specimens, which are not recommended for the SARS-CoV-2 IVD. Target-2 (E gene) showed the best consistency with the reference assay, which is also targeting the viral E gene.

For both human plasma and lower respiratory tract specimens, results correlated very well with the comparator assay within linear range (Fig. 3 ).

Fig. 3

Clinical evaluation of the SARS-CoV-2 IVD compared to the comparator assay, SARS-CoV-2 UCT (utility channel test).

Clinical evaluation of the SARS-CoV-2 IVD compared to the comparator assay, SARS-CoV-2 UCT (utility channel test). Predetermined clinical samples previously analyzed in routine diagnostics were retrieved from storage at −20 °C and subjected to testing with the SARS-CoV-2 IVD. Sample input volume was 400 μL for both assays. X-axis: UCT CT. Y-axis: IVD CT. Blue dots: measurements within linear range of Target-2, considered for trendline and correlation. Orange dots: measurements outside linear range or one Target negative, not considered for trendline and correlation. ND: “Not Detected”.

Inter-run variability between different machines and testing sites

A set of samples spiked with SARS-CoV-2 (HH-1) was subjected to testing on a total of four different instruments at three different testing sites. Deviations of CT values did not exceed 1.53 CT in UTM and 0.53 CT in human plasma, indicating excellent comparability of results both between devices and testing sites (Fig. 4 ).

Fig. 4

Evaluation of inter-run variability on four different instruments.

Sets of seven identical spiked samples were sent to different laboratories in Germany for testing with the SARS-CoV-2 IVD. Material types were as indicated. UTM samples were not supplemented with cobas PCR Media. CT values were reported and compared. Hamburg-1: Instrument No1 at University Medical Center Hamburg-Eppendorf. Hamburg-2: Instrument No2 at University Medical Center Hamburg-Eppendorf. Labor Stein: Moenchengladbach. Labor Limbach: Heidelberg. Box: Median. Whiskers: Range.

Discussion

The recently released SARS-CoV-2 IVD for the cobas6800/8800 systems made automated high-throughput testing available to a wider range of diagnostic facilities. First studies providing clinical evaluation have demonstrated good agreement with established reference assays [12,15]. The benefits of automation for molecular diagnostics have been discussed extensively in previous studies [11,16]. While various sample-to-result systems were already used for SARS-CoV-2 diagnostics utilizing open modes during the early weeks of the pandemic [10,17,18], commercial test kits have since been released under emergency use authorization (EUA) for most automated PCR-platforms [12,19,20].

The SARS-CoV-2 IVD for the cobas 6800/8800 system provides a well standardized common ground for comparable quantification results and viral load monitoring, despite nominally being only a “qualitative” test. At present, the assay is limited by relatively narrow recommendations for matrix materials and lack of robust data on analytical and clinical performance. The aim of this study was to go beyond specifications and validate the assay on relevant clinical materials such as BAL/BL and blood, as well as evaluating the efficacy and performance impact of chemical inactivation procedures.

Safety has been a serious concern when handling clinical samples potentially containing SARS-CoV-2 in diagnostic laboratories. While Bio Safety Level 3 (BSL3) conditions are required for procedures that actively enrich or propagate infectious virus (i.e. cell culture), diagnostic samples are usually processed under BSL2 cabinets. Heat inactivation protocols are widespread in some parts of the world as there exists some evidence for their efficacy with SARS-CoV and MERS-CoV [21,22], however, there are also reports claiming that these procedures may lead to false negative PCR results [23]. Pastorino et al. showed that exposure to temperatures as high as 92 °C is necessary to completely abolish infectivity, also resulting in substantial degradation of viral RNA in samples [24]. Here we show that pre-treating samples by adding 1:1 guanidine hydrochloride solution (cobas PCR Media) reduces plaque formation by at least 3.5 log steps. This procedure did not impact assay performance in UTM (or eSwab medium). However, guanidine hydrochloride solution alone may not be sufficient to completely inactivate all infectious virus in samples [24,25].

Since the beginning of the outbreak, some clinical studies have reported on quantitative and semi-quantitative detection of SARS-CoV-2 RNA in different clinical materials over the course of treatment [7,8,26]. However, quantification results that are precise and reproducible across laboratories require highly standardized reference material, which is not yet available. Vogels et al. recently provided data on performance and linearity for the most commonly used SARS-CoV-2 inhouse assays, demonstrating that linear range and PCR efficiency does vary to a considerable degree [27]. However, other factors such as extraction methods, and choice of enzymes and cyclers will render comparability between sites difficult, even when using the same assays.

In this study we demonstrate that the SARS-CoV-2 IVD dual-target assay is highly efficient and offers a wide linear range in different types of matrix materials outside official specifications. Based on our data, it seems preferable to use the Target-2 (E gene) CT for quantification purposes as it performed slightly better than Target-1 (ORF1ab) in this trial. Compared to the inhouse assay SARS-CoV-2 UCT, the SARS-CoV-2 IVD assay showed good agreement for a total of 180 respiratory swab samples, which is in line with previously reported data [12]. Furthermore, good correlation was observed for 53 positive plasma and 56 positive lower respiratory tract samples within linear range of the IVD assay.

Fully automated RT-PCR systems performing nucleic acid extraction, PCR and signal analysis provide a standardized common ground between testing sites. In order to evaluate to what degree results are comparable between different instruments, predetermined identical sample-sets were subjected to testing in four different devices at three different testing sites in Germany. There was only minimal variance in CT values for human plasma samples (range dCT: 0.53) and slightly higher in UTM samples (range dCT: 1.53). This further underpins the reproducibility of results obtained from the SARS-CoV-2 IVD which may prove very useful in the context of viral-load monitoring and in clinical multicentre studies.

Conclusion

In this study we provide evaluation for different matrix materials and clinical performance of the SARS-CoV-2 IVD for the cobas6800/8800 systems. We observed excellent linearity over four to six log steps in different clinically relevant materials outside assay specifications, including human plasma samples, BAL/BL and swab medium pre-treated for chemical inactivation. Furthermore, deviations in CT values were minimal when identical samples were analyzed at different testing sites. The assay showed excellent correlation with the comparator assay for all specimen types analyzed, indicating compatibility of a variety of clinical materials despite not being officially recommended for use by the manufacturer. Overall, this data demonstrates the viability of the assay for providing reproducible quantification of SARS-CoV-2 RNA in respiratory samples and human plasma for viral load monitoring and clinical studies.

Author contribution

ML, SP, LO, DW, SK, NF and MA conceptualized and supervised the study. DN, AF and UE performed the experiments. DN, ML and SP wrote the initial draft of the manuscript. All authors agreed to the publication of the final manuscript.

Declaration of Competing Interest

UE, AF and ML received speaker honoraria and related travel expenses from Roche Diagnostics.

Parts of the material used for this study were provided by Roche Molecular Solutions (Pleasanton, USA).

SK reports grants and personal fees from Pfizer, personal fees from Biotest, personal fees from Cytosorbents, personal fees from Gilead, personal fees from MSD, personal fees from Bayer, personal fees from Astellas, personal fees from Baxter, personal fees from Fresenius, outside the submitted work

All other authors declare no conflict of interest.