Comparison of Two High-Throughput Reverse Transcription-PCR Systems for the Detection of Severe Acute Respiratory Syndrome Coronavirus 2.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has emerged as the cause of a worldwide pandemic. Many commercial SARS-CoV-2 reverse transcription-PCR (RT-PCR) assays have received Emergency Use Authorization from the U.S. Food and Drug Administration. However, there are limited data describing their performance, in particular the performance of high-throughput SARS-CoV-2 RT-PCR systems. We analyzed the diagnostic performance of two high-throughput systems: cobas 6800 and Panther Fusion, and their associated RT-PCR assays, with a collection of 389 nasopharyngeal specimens. The overall agreement between the platforms was 96.4% (375/389). Cohen's kappa analysis rated the strength of agreement between the two platforms as "almost perfect" (κ = 0.922; standard error, 0.051). Furthermore, there was no significant difference between corresponding cycle threshold values generated on the two systems (P value = 0.88; Student's t test). Taken together, these data imply that the two platforms can be considered comparable in terms of their clinical performance. We believe that this information will be useful for those who have already adopted these platforms or are seeking to implement high-throughput RT-PCR testing to stem the SARS-CoV-2 pandemic.

INTRODUCTION

In December 2019, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) emerged as the cause of a devastating respiratory tract disease, termed coronavirus disease 2019 (COVID-19) (1). Subsequently, this novel coronavirus has caused a worldwide pandemic with >2 million confirmed cases, >150,000 confirmed deaths, and >200 countries, areas, or territories with confirmed cases (2). New York City, the location of our medical center, is currently the epicenter for this infection, with more confirmed cases and deaths from COVID-19 than any other city in the world (3).

SARS-CoV-2 RNA can be detected in clinical specimens using reverse transcription-PCR (RT-PCR), and the most common specimen types assayed are nasopharyngeal (NP) and/or oropharyngeal swabs (4). On 4 February 2020, the Centers for Disease Control and Prevention (CDC) received Emergency Use Authorization (EUA) from the U.S. Food and Drug Administration (FDA) for an RT-PCR assay to detect SARS-CoV-2 in a range of respiratory specimens (5). Subsequently, many additional assays have received EUA from the FDA, including the cobas SARS-CoV-2 RT-PCR assay used in conjunction with the cobas 6800 or 8800 system (Roche Molecular Systems, Inc., Branchburg, NJ) and the Panther Fusion SARS-CoV-2 RT-PCR assay operated in combination with the Panther Fusion system (Hologic, Inc., San Diego, CA). Both platforms are automated, high-throughput systems that can process >1,000 specimens in 24 h. Key characteristics of these high-throughput platforms and their associated SARS-CoV-2 RT-PCR assays are shown in Table 1.

TABLE 1

Comparison of key characteristics displayed by the two high-throughput systems and their associated RT-PCR assays analyzed in this study

Characteristic	cobas SARS-CoV-2 RT-PCR (Roche Molecular Systems, Inc.)	Panther Fusion SARS-CoV-2 RT-PCR (Hologic, Inc.)
Instrument	cobas 6800	Panther Fusion
Specimen type tested in study	Nasopharyngeal swab	Nasopharyngeal swab
Minimum vol required (ml)	0.6	0.5
Specimen inactivated prior to loading on the instrument	No	Yes
Loci detected	ORF1ab (SARS-CoV-2 specific), E gene (pan-Sarbecovirus)	ORF1ab regions 1 and 2 (SARS-CoV-2 specific)
Limit of detection (TCID50/ml)a	ORF1ab, 0.007; E gene, 0.004	ORF1ab regions 1 and 2, 0.01
Test throughput/24 h (assuming 100% efficiency)	1,440	1,150
Time to first result (h)	∼3.5	∼2.5
Loading	Batchb	Random accessc

TCID50, tissue culture infectious dose which infects 50% of cells.

Batch-based processing allows loading of the specimens to be processed in batches using the same test or multiple tests with the same amplification conditions. With respect to the cobas 6800 system, batches of specimens (up to 96) can be continuously loaded and assays can be continuously processed.

Random access permits loading of specimens in any order at any time (including different specimen types) to be processed using different tests. With respect to the Panther Fusion system, specimens can be continuously loaded (including one at a time) and continuously analyzed.

Recent studies assessing the comparative performances of a number of commercial SARS-CoV-2 molecular assays have been described (6–8). However, to the best of our knowledge, there have been no reports documenting the direct comparison of high-throughput SARS-CoV-2 RT-PCR systems. Therefore, the primary aim of this study was to compare the performances of two high-throughput SARS-CoV-2 RT-PCR platforms that have been widely adopted in the United States: the cobas SARS-CoV-2 RT-PCR assay, associated with the cobas 6800 platform, and the Panther Fusion SARS-CoV-2 RT-PCR, associated with the Panther Fusion system.

MATERIALS AND METHODS

cobas SARS-CoV-2 RT-PCR.

The cobas SARS-CoV-2 test was performed on the cobas 6800 platform according to the default manufacturer’s instructions (9). The assay amplifies two loci within the SARS-CoV-2 genome: ORF1ab, a SARS-CoV-2-specific target (target 1), and the E gene (target 2), a pan-Sarbecovirus target. If detected, a cycle threshold (C) value is determined for each target. Amplification of an RNA internal control is performed to assess specimen processing, amplification, and detection. Results are classified as not detected or detected. For the purpose of this study, a detected result was recorded if both targets were detected, only target 1 was detected, or only target 2 was detected. No repeat testing was performed.

Panther Fusion SARS-CoV-2 RT-PCR.

The Panther Fusion RT-PCR was used in conjunction with the Panther Fusion system according to the manufacturer’s instructions (10). The loci for amplification are regions 1 and 2 within ORF1ab. Although two regions within ORF1ab are amplified, the products are detected by probes labeled with the same dye. Amplification of either one or both regions contributes to a single C value. An internal control is added to each reaction to monitor processing, amplification, and detection. For the purpose of this study, results were interpreted as not detected or detected. No repeat testing was performed.

Xpert Xpress SARS-CoV-2 RT-PCR.

The Xpert Xpress SARS-CoV-2 assay (Cepheid, Inc., Sunnyvale, CA) was used to adjudicate discrepancies between the cobas 6800 and Panther Fusion systems. Testing was performed according to the manufacturer’s instructions (11) as soon as possible after the discrepancy was observed (within 24 h). The assay permits the detection of two loci: the N2 gene (a SARS-CoV-2-specific target) and the E gene (a pan-Sarbecovirus target). A sample processing control is associated with each test and ensures that sample processing is adequate. Results are interpreted as not detected or detected. For the purpose of this study, a detected result was recorded if both targets were detected, only the N2 gene was detected, or only the E gene was detected. No repeat testing was performed.

Retrospective specimens.

A collection of 176 archived NP swab specimens obtained from independent subjects and collected in viral transport media (Becton, Dickinson and Company, Franklin Lakes, NJ, or Hardy Diagnostics, Santa Maria, CA) and tested on the cobas 6800 system were selected for testing on the Panther Fusion platform. These specimens were specifically selected based upon their resultant C values and spanned the dynamic range of the cobas 6800 system. The collection was composed of 81 specimens in which the virus was not detected and 95 specimens in which SARS-CoV-2 was detected by the cobas SARS-CoV-2 RT-PCR (both targets 1 and 2 detected). The 95 detected specimens spanned a range of target 1 (the SARS-CoV-2-specific locus) C values and were classified into three groups: high (suggestive of a high level of viral burden) (C value, <20; n = 29), medium (suggestive of a medium level of viral burden) (C value, 20 to 30; n = 35), and low (suggestive of a low level of viral burden) (C value, >30; n = 31). These specimens were collected between 30 March 2020 and 19 April 2020 and frozen once at –80°C prior to analysis on the Panther Fusion platform (the cobas 6800 system was operationalized initially, hence the order of testing). Prior to testing, specimens were thawed at room temperature and briefly vortexed.

Prospective specimens.

A collection of 213 NP swab specimens (transported in viral transport media) consecutively received for clinical testing from 207 subjects were assayed in parallel (within 24 h) on both the cobas 6800 and Panther Fusion systems. The specimens were not frozen and, per manufacturer recommendations, were not vortexed prior to testing. The NP swab specimens were obtained between 21 April 2020 and 23 April 2020.

Electronic medical record review.

For discordant specimens, the electronic medical record was reviewed to determine if the individual associated with the discordant specimen had any additional SARS-CoV-2 tests that were performed for routine clinical care or any symptoms compatible with COVID-19. All additional specimens (collected either before or after the discordant specimen) were tested using the cobas SARS-CoV-2 RT-PCR assay (our test of record for SARS-CoV-2 at the time of this study).

Statistical analyses.

All statistical analyses were conducted using Stata, version 15.0 (StataCorp, LLC, College Station, TX).

This study was performed under a Weill Cornell Medicine Institutional Review Board (IRB)-approved protocol (20-03021671).

RESULTS

Of the 176 retrospective NP swab specimens, 173 yielded the same result on both platforms, resulting in an agreement of 98.3% (173/176). Ninety-four specimens were detected on both platforms (cobas 6800 positive/Panther Fusion positive), 1 was detected by cobas 6800 only (cobas 6800 positive/Panther Fusion negative), 2 were detected by Panther Fusion only (cobas 6800 negative/Panther Fusion positive), and 79 were not detected by either platform (cobas 6800 negative/Panther Fusion negative). Cohen’s kappa analysis comparing the performances of the two platforms was rated “almost perfect” (12) (κ = 0.966; standard error [SE], 0.075). When comparing the 213 prospective specimens, 202 generated the same result for an agreement of 94.8% (cobas 6800 positive/Panther Fusion positive, 39; cobas 6800 positive/Panther Fusion negative, 6; cobas 6800 negative/Panther Fusion positive, 5; and cobas 6800 negative/Panther Fusion negative, 163). The agreement between the two systems was rated “strong” (12) (κ = 0.844; SE, 0.069). Finally, when retrospective and prospective specimens (n = 389) were combined, the strength of the agreement was rated “almost perfect” (12) (κ = 0.922; SE, 0.051) and the agreement was 96.4% (375/389) (cobas 6800 positive/Panther Fusion positive, 133; cobas 6800 positive/Panther Fusion negative, 7; cobas 6800 negative/Panther Fusion positive, 7; and cobas 6800 negative/Panther Fusion negative, 242). Interestingly, a small subset of prospective specimens analyzed on the cobas 6800 platform were positive for either target 1 only (n = 2) or target 2 only (n = 9). For these specimens, the C values were >35. None of the target-1-only specimens were detected on the Panther Fusion, while six of the target-2-only specimens were detected on the Panther Fusion.

To further assess the agreement between the two platforms, we analyzed the relationship between the SARS-CoV-2-specific loci detected on each system. The C values for the cobas 6800 SARS-CoV-2-specific target were between 14 and 35.3 for retrospective specimens and 14.1 and 36.1 for prospective specimens, while the Panther Fusion SARS-CoV-2-specific regions ranged between 13 and 38.3 for retrospective specimens and 14.6 and 38.4 for prospective specimens. A total of 127 specimens (retrospective, 94; prospective, 33) had C values for the same target (ORF1ab) detected by each assay and were directly compared. Data are displayed graphically in Fig. 1. Importantly, no significant difference between the C values was observed (P value = 0.88; Student’s t test). The median C values for ORF1ab for the retrospective specimens assayed on the cobas 6800 and Panther Fusion platforms were 24.7 (interquartile range [IQR], 19.5 to 31.1) and 24.6 (IQR, 18.2 to 32), respectively. The median C values for ORF1ab for the prospective specimens assayed on the cobas 6800 and Panther Fusion platforms were 28.5 (IQR, 21.2 to 32.8) and 27.9 (IQR, 20.2 to 33.3), respectively. Finally, the median C values for ORF1ab for the combined retrospective and prospective specimens assayed on the cobas 6800 and Panther Fusion platforms were 26.2 (IQR, 19.6 to 31.5) and 25.3 (IQR, 18.9 to 32.4), respectively.

FIG 1

Box-and-whisker plot of 127 positive specimens (retrospective, 94, and prospective, 33) that had C values for the same target (ORF1ab) detected by each assay. The points along the y axis are presented from 40 through 10, as the C value is inversely proportional to the viral burden, with a specimen exhibiting a C value of 10 having a higher viral burden than one with a C value of 40.

There were 14 specimens (from a total of 389) with discordant results (Table 2), 3 associated with the retrospective group and 11 associated with the prospective group. The C values of the target loci for the discordant specimens were >35, implying low viral burden. To investigate these discrepancies, we analyzed each of the 14 specimens using the Xpert Xpress SARS-CoV-2 RT-PCR. We reasoned that an independent method that requires minimal specimen volume (especially given the small amount of remnant specimen) may adjudicate in favor of a given platform. Postanalysis, nine of the specimens tested by the Xpert Xpress SARS-CoV-2 test were in agreement with the cobas 6800 result, while five agreed with the Panther Fusion assay (Table 2). Finally, we reviewed the electronic medical records of the patients associated with the discordant specimens to understand if additional SARS-CoV-2 RT-PCR tests were performed and if their symptoms were compatible with COVID-19 (13) (Table 2). For the majority of cases (10/14), an additional SARS-CoV-2 RT-PCR for a specimen taken before or after the discordant specimen was positive and/or clinical symptoms were suggestive of COVID-19.

TABLE 2

Characteristics of the discordant specimens and associated patients

Specimen no.	cobas 6800 SARS-CoV-2 RT-PCR result				Panther Fusion SARS-CoV-2 RT-PCR result			Discrepancy test resultb	Result of additional SARS-CoV-2 testc	Symptoms compatible with COVID-19
	Result	Target 1 CT value	Target 2 CT value	IC CT value	Result	CT value	IC CT value
Retrospective
53	D	35.26	37.69	33.52	ND	NA	30.7	D	NA	NA
66	ND	NA	NA	34.21	D	36.5	31	D	NA	NA
151	ND	NA	NA	33.53	D	38.1	31.1	D	D	Yes
Prospective
212	ND	NA	NA	33.89	D	38.1	32.4	ND	NA	Yes
240	ND	NA	NA	35.78	D	36.9	32.5	D	D	Yes
275	D	NA	38.26d	34.1	ND	NA	31.8	D	NA	Yes
300	ND	NA	N/A	33.9	D	38	30.2	De	D	Yes
326	D	35.37	NA	33.82	ND	NA	31	D	NA	NA
333	D	NA	37.64d	34.13	ND	NA	31.3	D	D	Yes
335	D	NA	38.46d	34.28	ND	NA	30.6	De	D	Yes
338	D	35.84	38.05	34.28	ND	NA	30.7	D	NA	NA
361	ND	NA	NA	33.71	D	38	32.1	ND	NA	Yes
366	ND	NA	NA	33.96	D	38.1	32.5	D	D	NA
382	D	36.13	NA	33.95	ND	NA	32.5	D	D	Yes

Abbreviations: C, cycle threshold; COVID-19, coronavirus disease 19; D, detected; IC, internal control; NA, not applicable or available; ND, not detected.

The method used for discrepancy testing was the Xpert Xpress SARS-CoV-2 RT-PCR.

The additional SARS-CoV-2 RT-PCR was performed on the cobas 6800 platform (our test of record for SARS-CoV-2 at the time of this study).

This result would be classified as presumptively positive according to the manufacturer (Roche Molecular Systems, Inc.), indicating that the E gene was detected but the ORF1ab locus was not detected. However, for the purpose of this study this result was interpreted as detected.

This result would be classified as presumptively positive according to the manufacturer (Cepheid, Inc.), indicating that the E gene was detected but the N2 gene was not detected. For the purpose of this study this result was interpreted as detected.

DISCUSSION

In this study, we compared the diagnostic performances of the cobas 6800 and Panther Fusion high-throughput RT-PCR systems for the detection of SARS-CoV-2 RNA in 389 NP swab specimens, the predominant specimen type employed for SARS-CoV-2 RT-PCR (4). In the absence of clinical trial information, which is to be expected for assays that receive EUA from the FDA, these data are especially important for the diagnostic and clinical communities. The overall percent agreement between the two systems was excellent (96.4%), and Cohen’s kappa analysis rated the strength of the agreement between systems as “almost perfect” (12). Therefore, we posit that the two platforms display similar performance characteristics in the clinical setting.

Very recently, studies describing the performance of several SARS-CoV-2 molecular assays (including the cobas SARS-CoV-2 RT-PCR) which have received EUA from the FDA have been reported (6–8). These reports suggest that the diagnostic performances of most of these assays are equivalent, although a significant difference in the performance between the ID NOW COVID-19 assay (Abbott Diagnostics, Scarborough, ME) and the Abbott RealTime SARS-CoV-2 assay performed on the Abbott m2000 system (Abbott Molecular Inc., Des Plaines, IL) was observed (6). The RealTime SARS-CoV-2 assay yielded more detected results (ID NOW COVID-19 positive/RealTime SARS-CoV-2 negative, 2, and ID NOW COVID-19 negative/RealTime SARS-CoV-2 positive, 47 [n = 524]). However, to the best of our knowledge, no study has evaluated the performance characteristics of the Panther Fusion SARS-CoV-2 RT-PCR assay or directly compared two high-throughput systems.

This study does have limitations. We were unable to compare our data to a reference method, such as the CDC EUA RT-PCR assay, due to a lack of available resources. Nevertheless, there is precedent for this in the literature as neither Moran and colleagues nor Harrington and coworkers employed a reference method in their comparison of SARS-CoV-2 assays (6, 7). In addition, we were unable to compare all high-throughput platforms currently available for SARS-CoV-2 RT-PCR, with the notable omission of the Abbott m2000 system. However, the cobas 6800 and Panther Fusion systems and associated SARS-CoV-2 RT-PCR assays are widely adopted throughout the United States.

In conclusion, the cobas 6800 and Panther Fusion systems and their associated SARS-CoV-2 tests are comparable in terms of their performance characteristics in the clinical setting. Both platforms can analyze >1,000 NP swab specimens per day and have the potential to ensure that medical center, reference, and public health laboratories have the capability to efficiently and expediently process and analyze very high volumes of NP swab specimens for the detection of SARS-CoV-2 RNA and thus stem the global scourge of COVID-19. Ultimately, we believe that data presented herein are important and useful for those who have adopted or are seeking to implement high-throughput RT-PCR platforms in the midst of the COVID-19 pandemic.