Quantitative analysis of different respiratory specimens on two automated test systems for detection of SARS-CoV-2 RNA.

Molecular testing of SARS-CoV-2 RNA is essential during the pandemic. Here, we compared the results of different respiratory specimens including anterior nasal swabs, pharyngeal swabs, saliva swabs, and gargle lavage samples to nasopharyngeal swabs on two automated SARS-CoV-2 test systems. Samples were collected and tested simultaneously from a total of 36 hospitalized symptomatic COVID-19 patients. Detection and quantification of SARS-CoV-2 was performed on cobas®6800 (Roche) and NeuMoDx™ (Qiagen) systems. Both assays showed reliable detection and quantification of SARS-CoV-2 RNA, with nasopharyngeal swabs showing the highest sensitivity. SARS-CoV-2 RNA concentrations in other respiratory specimens were lower (mean 2.5 log10 copies/ml) or even undetectable in up to 20%. These data clearly indicate that not all respiratory materials are equally suitable for the management of hospitalized patients, especially, in the late phase of COVID-19, when the viral phase subsides and inflammation becomes the predominant factor, making detection of even lower viral loads increasingly important.

Introduction

The global spread of severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) and the required expansion of testing capacities was an enormous global effort. Early identification and sequencing enabled very rapid development of SARS-CoV-2 detection protocols [1]. However, most in-house protocols are labor-intensive, and the massive use of quantitative RT-PCR (qPCR) protocols resulted in shortage of extraction and qPCR reagents [2], [3], [4]. In parallel, commercial assays were developed including fully automated SARS-CoV-2 systems for high-throughput testing [5]. In spring 2020, Roche's cobas®SARS-CoV-2 test received an Emergency Use Authorization (EUA) by the US Food and Drug Administration (FDA) for the qualitative detection of SARS-CoV-2 RNA in nasopharyngeal and oropharyngeal swabs on Roche's fully automated cobas® 6800 and cobas® 8800 systems [6]. During 2020, additional commercial SARS-CoV-2 RNA assays received EUA status for in vitro diagnostic devices (IVD), including the NeuMoDx™ SARS-CoV-2 test (Qiagen), automated on the NeuMoDx random-access platform [7, 8].

To control the spread of SARS-CoV-2 infection, massive screening measures were initiated to identify both asymptomatic SARS-CoV-2 infections and infections before the onset of symptoms [9], [10], [11]. Since SARS-CoV-2 infections in the younger population and in children are often asymptomatic, and vaccinations are not yet recommended in all age groups, schools and child care facilities are of particular interest [12, 13]. In this age group, the recommended nasopharyngeal and throat swabs are difficult to perform and must be performed by trained medical personnel. Therefore, several screening studies were initiated using respiratory materials such as saliva swabs, anterior nasal swabs or gargle lavage, which can be obtained in a minimally invasive procedure [[9], [10], [11], [14], [15], [16]]. Notably, the performance of high-throughput test systems with such alternative specimens from the upper respiratory tract is still unclear. In hospitalized patients, monitoring of SARS-CoV-2 RNA concentrations is important for the management of antiviral therapy as well as decision making in hospital hygiene. Reliable quantification of SARS-CoV-2 RNA is therefore particularly relevant in the hospital setting.

The objective of this study was therefore to directly compare detection rates and quantitative results of the recommended nasopharyngeal and throat swabs to different minimally invasive respiratory specimens on the fully automated SARS-CoV-2 test systems cobas® 6800 and NeuMoDx™.

Material and Methods

Study cohort

All study participants were hospitalized for symptomatic SARS-CoV-2 infection, but none required intensive care. A total of 36 inpatients with confirmed SARS-CoV-2 infection were recruited for the study. To participate, patients had to have a confirmed SARS-CoV-2 infection on a nasopharyngeal swab and a written informed consent from the patient or guardian to participate in the study. Approval from the local ethics committee was obtained (Study-ID.: 2021-1291).

The study participants were recruited in autumn 2020. Since no vaccines were available at that time and the spread of new clinically relevant SARS-CoV-2 variants was not documented until February 2021, we assume unvaccinated patients with SARS-CoV-2 wild-type (B.1 variant) infections.

Respiratory specimens

For the comparative analysis, different respiratory materials were obtained simultaneously from the study participants. While the recommended nasopharyngeal swab (NPS) and throat swabs were performed by medical personal, the anterior nasal swabs, the saliva swabs, and the gargle lavage using 10 ml NaCl2 were self-collected by the study participants. Detection of SARS-CoV-2 in the NPS was mandatory for comparative analysis.

Sample preparation

All respiratory materials obtained were processed on the same working day. To minimize the influence of inhibitory interfering factors in respiratory materials on PCR, samples were diluted and centrifuged prior to PCR (c6800: 1:2.5; NMDx: 1:4.3). The dilutions were performed with cell culture medium (DMEM). The samples were tested according to the manufacturer's instructions.

RT-PCR platforms

Detection of SARS-CoV-2 RNA was performed using two different commercial RT-qPCR assays, automated on the cobas®6800 (c6800, Roche) and on the NeuMoDx™ (NMDx, Qiagen). Both are dual target assays and detect two different target genes of SARS-CoV-2, one structural and one non-structural protein (c6800: ORF1, E gene; NMDx: NSP2, N gene).

Quantification of SARS-CoV-2 RNA

In order to compare different test systems directly with each other, it is necessary to use reference material, so-called standards. To compare the cobas 6800 and the NeuMoDx SARS-CoV-2 assays in terms of their sensitivity, the cycle threshold (Ct) values must be correlated with quantitative reference samples. For the direct comparison between the cobas 6800 and NMDx the quantitative SARS-CoV-2 reference samples provided by INSTAND e.V. (Society for the Promotion of Quality Assurance in Medical Laboratories) were used to compare Ct-values [17]. The quantitative reference samples are SARS-CoV-2 positive cell culture supernatants with 107 copies/ml and 106 copies/ml, respectively. To determine the SARS-CoV-2 RNA concentration for a given Ct-value, the quantitative reference samples were used in serial dilutions and a standard curve was established. The calculated SARS-CoV-2 RNA concentrations for two target genes are given as mean values.

Statistics

Statistical analysis was performed using GraphPad Prism 9.01. Ct-values and SARS-CoV-2 RNA concentrations were tested for normal distribution with the Shapiro-Wilk method. Multiple Wilcoxon matched-pairs signed rank tests were used to compare individual specimen types between the NMDx and the c6800 assays and the adjusted p-value was calculated with the Holm-Šídák method. Differences between specimen types within one assay were compared by Friedman test followed by Dunn´s multiple comparisons test. A p-value <0.05 was considered statistically significant.

Results

In total, 180 respiratory samples of 36 patients infected with SARS-CoV-2 (f=13 (36.1%), m=23 (63.9%)) with a median age of 56.5 years (19-90 years) were analyzed.

The SARS-CoV-2 analyses of the various respiratory materials showed different detection rates compared to the NPS as the gold standard (Table 1 ). While both the nasal and the throat swab showed a high detection rate of 91.7%, detection rates were lower with the saliva swab (c6800: 83.3% and NMDx: 80.6%) and the gargle lavage (80.6% and 72.2%). Notably, in these latter two specimens the detection rate of SARS-CoV-2 RNA was reproducibly lower with the NMDx assay than with the c6800 assay (Table 1).

Table 1

Agreement of SARS-CoV-2 detection rate of different respiratory specimens compared to the nasopharyngeal swab (NPS)

Concordance NPS	Nasal Swab % (n)	Throat Swab % (n)	Saliva Swab % (n)	Gargle lavage % (n)
c6800	91.7 (33)	91.7 (33)	83.3 (30)	80.6 (29)
NMDx	91.7 (33)	91.7 (33)	80.6 (29)	72.2 (25)

When comparing the concordance of SARS-CoV-2 RNA detection between the two assays, an agreement of 100% was observed with the anterior nasal swab and throat swab (Table 2 ). There was also agreement between the two systems regarding the negative result in 3 identical swabs from the anterior nasal and pharyngeal cavities. In contrast, the saliva swab and the gargle lavage only achieved overall agreements of 86.1% and 88.9%, respectively, with the gargle lavage showing the highest false negative rate of 19.4% in both SARS-CoV-2 assays. Overall, the c6800 assay performed slightly better with saliva swabs and gargle lavages compared to the NMDx assay, where three and four samples (8.3% and 11.1%, respectively) were tested positive on the c6800 and negative in the NMDx assay. In turn, two saliva swabs (5.6%) were tested positive in the NMDx assay despite being tested negative in the c6800 assay. Notably, discordant results were associated with high Ct-values (Ct >32; Supplemental table 1).

Table 2

Agreement of SARS-CoV-2 detection between c6800 and NMDx SARS-CoV-2 assays.

c6800/NMDx	NPS % (n)	Nasal Swab % (n)	Throat Swab % (n)	Saliva Swab % (n)	Gargle lavage % (n)
Concordance	100 (36)	100 (36)	100 (36)	86.1 (31)	88.9 (32)
+/+	0	91.7 (33)	91.7 (33)	75 (27)	69.4 (25)
-/-	0	8.3 (3)	8.3 (3)	11.1 (4)	19.4 (7)
Disconcordance	0	0	0	13.9 (5)	11.1 (4)
+/-	0	0	0	8.3 (3)	11.1 (4)
-/+	0	0	0	5.6 (2)	0

A detailed comparison of the Ct-values determined in both SARS-CoV-2 assays showed minimal differences between individual target genes, but clear differences related to the respiratory material (Figure 1 , Supplemental table 1). Regardless of the target gene, the NPS samples showed the lowest median Ct-values (Ct: 24.76) followed by the throat swab and nasal swab (CT: 27.56 and Ct: 29.39, respectively). Both saliva swabs and gargle lavage samples provided median Ct-values >32 (Ct: 32.89 and Ct: 33.04, respectively). Overall, the non-structural proteins Nsp2 and ORF1 yielded lower Ct-values than the structural proteins, independent of the material used (Figure 1). These data clearly demonstrate the limitations of a quantitative statement based on individual Ct-values when the assay is not calibrated to a quantitative standard. Accordingly, for direct comparison of the two SARS-CoV-2 assays on the different platforms the SARS-CoV-2 RNA was quantified using a standardized reference material.

Figure 1

Ct-values determined with the SARS-CoV-2 assays for the c6800 and the NMDx system separated to the target gene and the respiratory specimen. The c6800 SARS-CoV-2 assay detects the non-structural ORF1 gene and the structual E gene. The NMDx SARS-CoV-2 assay detects the non-structural Nsp2 gene and the structual N gene. The number of PCR cycles is limited to 40 in both assays (dotted line). Undetected samples are set to a Ct-value of 41. Median Ct-value and IQR (25-75%) are indicated in bars. NPS: nasopharyngeal swab

Overall, similar SARS-CoV-2 RNA concentrations were detected with both analyzed automated SARS-CoV-2 assays ranging from 3.932 to 6.379 log10 cop/ml with the c6800 and from 3527 to 6.415 log10 cop/ml with the NMDx assay, depending on the respiratory material (Figure 2 , Table 3 ).

Figure 2

Comparison of SARS-CoV-2 RNA concentrations of different respiratory specimens quantified using a standardized reference material determined with the SARS-CoV-2 assays for the c6800 and the NMDx system. Direct comparison of the determined SARS-CoV-2 RNA concentrations with the c6800 (gray) or the NMDx (blue) SARS-CoV-2 assay is shown in A. Comparison of the SARS-CoV-2 RNA concentrations determined for the different respiratory specimens with the c6800 SARS-CoV-2 assay is shown in (B) or with the NMDx SARS-CoV-2 assay in (C). Median viral load and IQR (25-75%) are indicated in bars. NPS: nasopharyngeal swab

Table 3

Comparison of the determined SARS-CoV-2 RNA concentrations between different respiratory specimens analyzed with the c6800 and the NMDx assays.

	NPS	Nasal Swab	Throat Swab	Saliva Swab	Gargle lavage
c6800 log10 cop/ml (median) (ΔNPS)	6.379	5.061 (-1.318)	5.585 (-0.794)	3.932 (-2.447)	4.114 (-2.264)
NMDx log 10 cop/ml (median) (ΔNPS)	6.415	4.802 (-1.577)	5.476 (-0.904)	3.792 (-2.587)	3.527 (-2.852)

NPS = naspharyngeal swab

Looking at the differences in respiratory specimen, according to the determined Ct-values, the NPS provided the highest SARS-CoV-2 RNA concentrations with a median of 6.379 log10 copies/ml (IQR: 5.144-7.922) with the c6800 assay and 6.415 log10 cop/ml (IQR: 5.142-8.015) with the NMDx assay (Figure 2 and Table 3). Both the saliva swabs and the gargle lavages yielded in average 2.5 log10 lower SARS-CoV-2 RNA concentrations compared to the NPS (c6800: 3.932 log10 cop/ml and 4.114 log10 cop/ml, and NMDx: 3.792 log10 cop/mL and 3.527 log10 cop/ml, p<0.0001, respectively). Of note, the nasal and throat swabs had similar median SARS-CoV-2 RNA concentrations regardless of the assay (c6800: 5.061 log10 cop/ml and 5.585 log10 cop/ml, and NMDx: 4.802 log10 cop/ml and 5.476 cop/ml, p>0.9999, respectively), but concentrations were on average 0.8 (throat swab) and 1.4 (anterior nasal swab) log10 levels lower compared to the NPS.

Discussion

During the COVID-19 pandemic, it was necessary to develop reliable assays for the detection of SARS-CoV-2 infections in a very short period of time. This has resulted in many commercial SARS-CoV-2 tests coming to market shortly after the onset of the pandemic [7, 18]. In order to make more efficient use of laboratory capacity, tests for automated platforms have been developed in addition to standard real-time PCR assays [19].

In this study, the SARS-CoV-2 assay from Roche for the c6800 and from Qiagen for the NMDx platform were compared. For this purpose, not only the generally recommended specimens such as nasopharyngeal swabs or throat swabs were used, but also less invasive and easier to obtain respiratory specimens such as anterior nasal swabs, saliva swabs, and gargle lavages, which have been used primarily in children but also in many cases for SARS-CoV-2 screening [11, [20], [21], [22], [23]].

While the c6800 system is well established in diagnostic workflows for many years, the NMDx system is available since the end of 2018 and therefore the latest platform in the field of molecular virus diagnostics. Nevertheless, open channel reagents were offered for this platform prior to the availability of commercial kits, enabling automation of laboratory-developed real-time RT-PCR test (LDT) for SARS-CoV-2 as early as autumn 2020 [1]. Commercial reagents for SARS-CoV-2 for the c6800 and the NMDx platform became available in summer 2020. Both platforms use a dual-target system and achieve full automation and thus higher sample throughput. Both assays showed convincing analytical and clinical performance compared to other commercial assays [24, 25].

Comparative testing of 180 respiratory specimens from 36 different SARS-CoV-2-positive study participants using both platforms revealed no significant differences in detection rates between the c6800 and NMDx SARS-CoV-2 assays when using recommended specimens such as nasopharyngeal swabs, anterior nasal swabs, or throat swabs. The c6800 assay has been shown in previous comparative analyses to be effective in qualitative detection of SARS using nasopharyngeal swabs [26] as well as various other clinical samples such as sputum, bronchial aspirate, bronchoalveolar lavage and anal swabs [25]. In contrast we saw substantially lower detection rates in our study using saliva swabs and gargle lavages.

While anterior nasal and pharyngeal swabs were only slightly less sensitive than the NPS, as previously observed in other studies [27, 28], both the saliva swab and gargle lavage showed 20% lower detection rates on average compared to the NPS. This is particularly evident when looking at the agreement of SARS-CoV-2 detection between the two assays. The concordances for saliva swabs and gargle lavage were <90% compared to the other materials with 100% concordances. Here, the disconcordance could primarily be attributed to the non-detection with the NMDx assay. However, it should be considered that the discrepancies between the assays were only observed in samples with low viral loads, i.e. Ct-values >32. An inferior analytical sensitivity in the low-viremic range is expected, however, the sensitivity may differs between different assays [29].

To address whether the discordant results between the two assays were due to differences in the sensitivity of individual target genes used in the assays, we compared not only the Ct-values of the different materials but also the Ct-values of the individual targets. We observed minor differences in analytical sensitivity between the different targets, which were primarily differences between the genes coding for structural and non-structural proteins and not between the selected target genes of the assays. At lower viral loads (high Ct-values), which we detected with saliva swabs and gargle lavages, the number of undetected targets increased. Together, no individual target predominantly failed in saliva swabs or gargle lavages suggesting that detection of all targets were similarly affected in these specimens.

Although other studies have shown that both saliva and gargle lavage are suitable alternative respiratory materials for the detection of SARS-CoV-2 [30, 31], as already mentioned above, these two materials were clearly inferior in our study. This was most likely due to the fact that we directly, in contrast to the other studies, compared our detection rates to the gold standard, the NPS, which still provides the most sensitive detection for SARS-CoV-2 [32, 33], and in addition to the timing of sample collection during the course of infection. As shown by Jamal et al. and Savela and colleages, saliva specimen present high viral loads only at the onset of infection as differences in sensitivity were greatest for sample pairs collected later in illness [34, 35]. Notably, for screening of early infections, saliva may still be an excellent respiratory material, however, in hospitalized and symptomatic patients at a later stage of COVID-19 infection, SARS-CoV-2 RNA concentrations in the upper respiratory tract are typically low or even already undetectable, which needs to be considered for decision making in the hospital.

The SARS-CoV-2 RNA concentration plays an important role for the patient management in terms of isolation measures and treatment decisions. Quantitative analysis of SARS-CoV-2 RNA is therefore common practice. Overall, similar SARS-CoV-2 RNA levels were detected with both platforms depending on the respiratory specimen. As shown in other studies [35, 36], the NPS showed the highest SARS-CoV-2 RNA concentrations in the quantitative comparisons between the different specimen, followed by nasal and throat swab with an average of one log level lower SARS-CoV-2 RNA concentrations according to the determined Ct-values. Thus, as expected, both saliva swab and gargle lavage were inferior in this quantitative evaluation in our comparative study.

Taken together, both SARS-CoV-2 assays showed comparable results in both detection and quantification of SARS-CoV-2 RNA. Differences were only found between the different respiratory specimens analysed. Possible reasons for reduced sensitivity of saliva swabs and gargle lavages are less viral shedding in this compartment, lower stability of the SARS-CoV-2 RNA, and/or a higher dilution factor for the SARS-CoV-2 RNA, related to the specimen heterogeneity[37, 38].

It should be noted that only samples from 36 study participants were examined, which represents a relatively small study cohort and represents a limitation of our study. A comparative analysis with a larger cohort could possibly dissect out the reasons for discordant results in the low viremic samples in more detail. Importantly, both SARS-CoV-2 assays utilized in this study were validated and CE-marked for nasal, nasopharyngeal and pharyngeal swabs only. In light of our results these specimens indeed showed higher SARS-CoV-2 RNA concentrations compared to minimally invasive materials and should be considered as the gold standard for COVID-19 diagnostics. Nevertheless, there may be a role for SARS-CoV-2 quantification from minimally invasive materials in certain patient groups, if it is ensured that the same specimens are longitudinally compared.

Conclusions

This study shows that both the c6800 and the NMDx SARS-CoV-2 assays are highly reliable with the recommended specimens and generate valid results. When a quantitative reference material was used as a standard, no relevant differences could be observed in the quantification of the respiratory materials between the assays, while saliva swabs and gargle lavages were clearly inferior in our study. Importantly, these data indicate that not all respiratory specimens are equally suitable for the management of hospitalized patients, especially when dealing with the late phase of COVID-19 in hospitalized patients, when viral replication is declining and inflammation becomes the predominant factor, which makes both the detection of lower levels of virus but also the quantitative longitudinal courses of SARS-CoV-2-RNA increasingly important.

Author statement

Manuscript title: Quantitative analysis of different respiratory specimens on two automated test systems for detection of SARS-CoV-2 RNA

All persons who meet authorship criteria are listed as authors, and all authors certify that they have participated sufficiently in the work to take public responsibility for the content, including participation in the concept, design, analysis, writing, or revision of the manuscript. Furthermore, each author certifies that this material or similar material has not been and will not be submitted to or published in any other publication before its appearance in the Diagnostic Microbiology & Infectious Disease.

The authors confirm contribution to the paper as follows:

Nadine Lübke: project administration, conceptualization, investigation, data curation, validation, writing original draft. Katharina Repges: investigation, data curation, review and editing of draft. Christopher Menne: formal analysis, validation, review and editing of draft. Andreas Walker: validation, review and editing of draft. Björn Jensen: conceptualization, resources, review and editing of draft. Noemi F. Freise: resources, review and editing of draft. Smaranda Gliga: resources, review and editing of draft. Simon Eickhoff: formal analysis, review and editing of draft. Hans Martin Bosse: conceptualization, resources, review and editing of draft. Ortwin Adams: formal analysis, review and editing of draft. Jörg Timm: supervision, conceptualization, validation, review and editing of draft.

Author contributions

Nadine Lübke: project administration, conceptualization, investigation, data curation, validation, writing original draft. Katharina Repges: investigation, data curation, review and editing of draft. Christopher Menne: formal analysis, validation, review and editing of draft. Andreas Walker: validation, review and editing of draft. Björn Jensen: conceptualization, resources, review and editing of draft. Noemi F. Freise: resources, review and editing of draft. Smaranda Gliga: resources, review and editing of draft. Simon Eickhoff: formal analysis, review and editing of draft. Hans Martin Bosse: conceptualization, resources, review and editing of draft. Ortwin Adams: formal analysis, review and editing of draft. Jörg Timm: supervision, conceptualization, validation, review and editing of draft.

Funding

This study was funded through intramural research funds.

Declaration of Competing Interest

All authors declare no competing interests.