Comparison of performance between three SARS-CoV-2 molecular assays (Aptima™, Laboratory Developed Test-Fusion, and R-GENE®) with special attention to turnaround time, a key point in laboratory management.

The emergence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) highlights the importance of rapid diagnostic testing to identify individuals with SARS-CoV-2 infections and to limit the spread of the virus. Many molecular assays have become commercially available to cope with this surging demand for timely diagnosis of COVID-19 cases, but identifying individuals requires accurate diagnostic tools. We compared the performance of three molecular SARS-CoV-2 assays: Aptima™ SARS-CoV-2 assay running on the Panther system (Hologic), an in-house assay (Laboratory Developed Test, LDT) running on the Fusion module of the Panther Fusion system (LDT-Fusion; Hologic), and the R-GENE® SARS-CoV-2 assay (bioMérieux). In addition, we also evaluated the turnaround time. This parameter is crucial to managing the SARS-CoV-2 diagnosis and represents a key point in the quality management at the laboratory. Aptima™ and LDT-Fusion assays exhibited an excellent positive percent agreement (PPA) (100.0%), while the R-GENE® assay showed a slightly decreased PPA (98.2%). The Hologic assays have a higher throughput with less hands-on time than the R-GENE® assays (24-25 vs. 71 min). Both Hologic assays are used on a fully automated random-access testing system with on-demand testing capabilities that avoid run series, unlike the R-GENE® assay. Automated random-access testing systems should be preferred during periods of high SARS-CoV-2 prevalence.

INTRODUCTION

For almost 2 years, the severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) outbreak has been upgraded to the level of a global pandemic by the World Health Organization. This pandemic has highlighted the importance of rapid, reliable, and comparable diagnostic testing to detect infected individuals, follow the pandemic evolution, and limit the spread. , The implementation of molecular diagnostic tools over the past few years has provided shorter turnaround times (TATs) and higher sensitivity and specificity for the detection of viral agents compared to conventional diagnostic assays. Many molecular assays have commercially available to cope with this surging demand for timely diagnosis of COVID‐19 cases.

At our institution, three molecular assays are used interchangeably to cope with the testing demand: the Aptima™ SARS‐CoV‐2 assay running on the Panther system (Hologic), an in‐house assay (Laboratory Developed Test, LDT) running on the Fusion module of the Panther‐Fusion system (herein referred as LDT‐Fusion; Hologic) and the R‐GENE® SARS‐CoV‐2 assay (bioMérieux).

We have had concerns before the integration of R‐GENE® and Aptima™ assays in our laboratory. Aptima™ is a transcription‐mediated amplification (TMA) assay and does not provide a semiquantitative result with a threshold amplification cycle (C t) value. R‐GENE® assay requires a second real‐time reverse transcriptase‐polymerase chain reaction (rRT‐PCR) in case of equivocal results.

Aptima™ assay was recently evaluated and compared with other assays for the detection of SARS‐CoV‐2 in clinical respiratory specimens. , , , One study mentioned that the Fusion Open Access protocol allows the use of LDT SARS‐CoV‐2 and one study compared it to Aptima™ assay. Nevertheless, this is the first study comparing the R‐GENE® assay to any other assays for SARS‐CoV‐2 testing. In this study, we compared, therefore, the performance of three assays (two reverse transcriptase‐polymerase chain reaction [RT‐PCR] assays and one TMA assay) for the detection of SARS‐CoV‐2 in clinical specimens. In addition, we also evaluated the TAT which is crucial to managing the SARS‐CoV‐2 diagnosis.

MATERIALS AND METHODS

Samples

We assayed 85 frozen specimens (stored for 2 months at −80°C after initial collection) from the 2020 winter season in parallel on three assays. Among 85 specimens, 76 nasopharyngeal swabs, 6 nasopharyngeal aspirates, and 3 bronchoalveolar lavages were assayed. The median age of the cohort was 62 years old. The gender distribution of the cohort was 33 women and 52 men. These samples came from patients who consulted at the University Hospital of Angers.

Methods

The Aptima™ assay targets two virus sequences located on the ORF1ab gene. A virus transport medium (VTM) (500 µl) was manually placed in the appropriate specimen lysis tube containing 710 µl of lysis buffer. On the Panther system, 360 µl of this mix was used for the lysis and capture of nucleic acids.

The LDT‐Fusion assay corresponds to an in‐house RT‐PCR assay running on the Fusion module of the Panther‐Fusion system using primers (nCoV_IP2 and nCoV_IP4) targeting two virus sequences on the RNA‐dependent RNA polymerase (RdRP) gene and uses primer/probe sets from the National Reference Center of respiratory viruses (Institut Pasteur). Samples were prepared as described for the Aptima™ assay.

The R‐GENE® assay includes two multiplexes rRT‐PCR: first targeting nucleocapsid and RdRp genes and performed for the screening of SARS‐COV‐2; and subsequently, a second targeting Sarbecovirus E gene and performed for diagnostic confirmation in cases of equivocal results. Nucleic acid extraction from 200 µl of VTM was performed on the NucliSENS® easyMAG® system (bioMérieux) and amplification on the Mx3005P QPCR System (Agilent technologies).

Performance evaluation and workflow analysis

For performance evaluation of assays, the consensus result was based on the majority results of the three assays and was defined as follows: consensus positive equals a positive result in two of three assays; consensus negative equals a negative result in two of three assays. A target gene with a C t > 40 was considered as negative for R‐GENE® and LDT‐Fusion assays.

The C t‐values of the RdRp gene were compared between R‐GENE® and LDT‐Fusion assays (C t‐value of the R‐GENE® RdRp gene compared to the average C t‐value of the LDT‐Fusion RdRp gene).

TAT of analytical step and hands‐on time (HOT) were evaluated for six specimens for each assay. TAT was defined as the time interval between the beginning of the sample processing and the availability of the final result at the laboratory (excluding the pre‐and post‐analytical steps). A time study was carried out by laboratory staff to record HOT in the laboratory with times for each step of the assays recorded. The viral inactivation is chemically processed in a lysis tube and the inactivation step is included in the TATs for three assays.

Ethics statement

The study was carried out in accordance with the Declaration of Helsinki. This study was a noninterventional study, with no alterations of the usual sampling procedures. Biological material data were obtained only for standard viral diagnosis following physicians' prescriptions (no specific sampling, no modification of the sampling protocol). Data analyses were carried out using an anonymized database.

Statistical analysis

Statistical analysis was performed using IBM® SPSS® 15.0 Statistics (Statistical Package for Social Sciences; IBM Corp).

RESULTS

Concordances between R‐GENE® and Aptima™ assays, between R‐GENE® and LDT‐Fusion, and between Aptima™ and LDT‐Fusion were high: 98.8% (84/85), 98.8% (84/85), and 97.6% (83/85), respectively.

Nevertheless, the R‐GENE® assay required further investigation for nine samples (9/54 positive samples, 16.7%) for which we observed initially discordant results for the first rRT‐PCR: no C t‐value for RdRp gene and C t < 40 for nucleocapsid gene. The second rRT‐PCR targeting E gene confirmed only two positive results (low viral load with C t > 38 for Sarbecovirus E gene) and seven negative results. LDT‐Fusion confirmed low viral load because C t > 35. LDT‐Fusion and Aptima™ assays exhibited a perfect positive percent agreement (PPA) (100.0%), while the R‐GENE® assay had a PPA of 98.2% (Table 1). Cohen's κ values were higher than 0.81, all of which indicate an “almost perfect” level of agreement with the consensus result. One discordant result was noted for R‐GENE®: negative result whereas positive result (Aptima™; relative light unit [RLU] = 1152) and C t > 35 (LDT‐Fusion) were observed, suggesting lower viral titer in this sample (nasopharyngeal swab). One discordant result was noted for Aptima™: positive result with a low value for RLU = 713 and negative results for the other two assays (LDT‐Fusion and R‐GENE®). This agrees with the positivity of one target out of two with Aptima™ and lower viral titer in this sample. Deming regression analysis showed a good correlation of paired positive R‐GENE® and LDT‐Fusion C t‐values (slope 1.01, Pearson's r = 0.97) (n = 54).

Table 1

Clinical performance comparison of three SARS‐CoV‐2 assays

Molecular assay	Consensus result		κ [95% CI]	PPA (%) [95% CI]	NPA (%) [95% CI]
	No. of positive samples	No. of negative samples
R‐GENE®
Positive	54	0	0.974 [0.925–1.023]	98.2 [95.4–1.0]	100.0 [1.0–1.0]
Negative	1	30
LDT‐Fusion
Positive	55	0	1.000 [1.000–1.000]	100.0 [1.0–1.0]	100.0 [1.0–1.0]
Negative	0	30	[1.000]
Aptima™
Positive	55	1	0.974 [0.923–1.025]	100.0 [1.0–1.0]	96.7 [92.9–1.0]
Negative	0	29

Abbreviations: CI, confidence interval; LDT, Laboratory Developed Test; NPA, negative percent agreement; PPA, positive percent agreement; SARS‐CoV‐2, severe acute respiratory syndrome coronavirus 2.

Figure 1 summarizes the workflow characteristics of each assay. The time of automation was approximately similar for R‐GENE® and LDT‐Fusion assays, whereas it was longer for the Aptima™ (1h20 of additional time). The overall laboratory TAT was most affected by the HOT with the R‐GENE® (71 min). The HOT was reduced with LDT‐Fusion and Aptima™ because the extraction process is integrated into the automated PCR platform and the following steps were removed: extraction launching, recovery of eluates, deposits of reagents and samples, PCR preparation, and PCR launching.

Figure 1

Workflow comparison of each assay. Dark gray color corresponds to hands‐on time and the light gray color corresponds to automation time. The workflow of each assay represented on the graph corresponds to the time for six specimens per run. The NucliSENS® easyMAG® system (the automated process of extraction) can extract 24 samples per run. To process the 120 samples, 5 extraction runs of 24 samples and 2 PCR runs must be considered, that is, a turnaround time of 16h13. LDT, Laboratory Developed Test; PCR, polymerase chain reaction

DISCUSSION

In summary, the three assays provided comparable qualitative results and confirmed previous evaluations results for LDT‐Fusion and Aptima™ assays. , , We reported the results of a first performance evaluation for the R‐GENE® assay not previously evaluated and compared. The negative percent agreement ranged from 96.7% to 100.0%, suggesting that each assay has good specificity.

Due to the high testing demand occurring in the pandemic context, it is essential to choose assays with reduced TAT to meet health requirements legacy regarding the timeliness of results for physicians and to lead to a significant improvement of patients’ care. Rigorous TAT analysis taking into account the human and technical resources is a key point in laboratory management; it is important to estimate it when setting up a new assay and then it is often used as a quality indicator. A limitation of the R‐GENE® assay is the manual setup of the multiwell plate (not automated) and the longer TAT for the nucleic acid extraction step. The TAT of R‐GENE® could be even longer in some cases because 9/85 samples in our study required further investigation to confirm equivocal results.

Both Hologic assays are used on a fully automated random‐access testing system with on‐demand testing capabilities that avoid run series, unlike the R‐GENE® assay. Random access offers the capability to simultaneously load a significant number of samples on a single instrument at any time and specimens can be continuously analyzed. Our data show that implementing the Panther/Fusion platform in our laboratory was associated with a great reduction in TAT that was largely due to the difference in the HOT when compared to the R‐GENE®. The time of automation is longer with Aptima™ than LDT‐Fusion. However, the Aptima™ assay has the advantage of continuous loading of reagents and specimens during the process, which offers time‐saving. One of the limitations of the Aptima™ assay is that it does not provide a semiquantitative assessment of the amount of virus. Thus, an RLU value around 900 suggests the positivity of both targets but does not predict low or high viral load.

Many laboratories perform rRT‐PCR molecular testing using separate instruments for nucleic acid extraction and amplification that are relatively time‐consuming and cause human resources deployment. R‐GENE® assay offers the advantage of being a flexible open system applicable to five different extraction and amplification platforms and has enabled to cope urgently with the demand for testing at the beginning of the pandemic with the systems already available in the laboratories. This assay may again be very useful at the end of the pandemic when laboratories will be confronted with a smaller quantity of samples, thus reducing the costs of consumables and reagents.

CONFLICTS OF INTEREST

Hélène Le Guillou‐Guillemette has received honoraria from being a consultant or speaker from Hologic and research grants from Hologic (outside the context of the submitted work). Other authors declare no conflicts of interest.

AUTHOR CONTRIBUTIONS

Adeline Pivert and Hélène Le Guillou‐Guillemette conceptualized the study. Caroline Lefeuvre, Adeline Pivert, and Emilie Przyrowski collected the data. Caroline Lefeuvre and Adeline Pivert conducted the data analyses. Caroline Lefeuvre wrote the first draft of the manuscript. Elise Bouthry, Estelle Darviot, Rafaël Mahieu, Françoise Lunel‐Fabiani, Alexandra Ducancelle, and Hélène Le Guillou‐Guillemette revised the manuscript. All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.