Performance of the Abbott ID NOW rapid SARS-CoV-2 amplification assay in relation to nasopharyngeal viral RNA loads.

Introduction

The coronavirus disease 2019 (COVID-19) pandemic caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has resulted in large worldwide mortality, morbidity, and economic devastation. Rapid diagnosis of COVID-19 by laboratory tests is critical for timely initiation of therapy, isolation, and epidemiologic interventions to combat the disease. Rapid identification of patients shedding large numbers of SARS-CoV-2 infectious particles is essential to implement timely isolation measures that protect other patients, frontline healthcare workers and family members . The SARS-CoV-2 ID NOW assay (ID-NOW) (Abbott, Scarborough, ME) uses rapid qualitative isothermal amplification to detect SARS-CoV-2 RNA. Given the turnaround time as short as 2 min for a positive result and 15 min for a negative result, we implemented the ID-NOW assay to address clinical situations that require rapid diagnosis and identification of infectious patients.

Previous studies reported lower sensitivity of the ID-NOW assay compared to real time RT-PCR assays [1], [2], [3], [4]. Sensitivity was decreased by dilution of nasopharyngeal swabs in transport media and consequently the manufacturer and the FDA have recommended direct dry swab sampling since May 14, 2020 [5]. We have confirmed the lower analytic and clinical sensitivity of the ID-NOW as compared to highly sensitive RT-PCR methods, including the Xpert Xpress SARS-CoV-2 (Cepheid, Inc., Sunnyvale, CA) and a CDC-based RT-PCR assay. To further validate the ID-NOW in our clinical practice, we obtained two parallel nasopharyngeal samples from each patient, a dry swab for ID-NOW and a swab in viral transport media (VTM) for the RT-PCR assays. We assayed the dry swab with the ID-NOW and reflexed all negative results to an RT-PCR assay. This parallel sampling approach provided an opportunity to assess the sensitivity of the ID-NOW assay as compared to RT-PCR in a larger number of samples. Further, we used the cycle threshold (C) values of the RT-PCR assays as a surrogate to relate viral RNA loads with the ID-NOW clinical sensitivity.

Methods

Patients

A total of 993 patients were tested with both the ID-NOW SARS-CoV-2 and the Xpert assay either with parallel swabs collected at the same time or within 4 h of each other. We collected a dry swab for ID-NOW and a swab placed in 3 mL of VTM for the Xpert assay. Initially we collected two simultaneous nasopharyngeal samples from 144 patients who presented to the George Washington University Hospital. Subsequently, we followed the manufacturer and FDA recommended protocol and reflexed an initial ID-NOW presumptive negative result to the Xpert assay in 817 patients. In addition, 32 patients initially positive by ID-NOW were also tested by Xpert within 4 h of the ID-NOW test. Clinical indications for ordering ID-NOW included acute or critical disease such as trauma, acute coronary syndrome, stroke, and precipitous labor or undergoing emergent caesarian section. Demographics and patient characteristics are shown on Supplementary Table-1 .

Table 1

Comparison of ID-NOW with Xpert assay performance. Numbers in parenthesis are percentage of total cases (993).

ID-NOW	Xpert
	Positive	Negative	Total
Positive	77 (7.8%)	4 (0.4%)	81 (8.2%)
Negative	33 (3.3%)	879 (88.5%)	912 (91.8%)
Total	110 (11.1%)	883 (88.9%)	993 (100%)
Agreement	Positive (Sensitivity): 77/110 = 70.0%	Negative (Specificity): 879/883 = 99.5%	Overall Agreement: 956/993 = 96.3%

Assays

ID-NOW uses isothermal nucleic acid amplification for rapid detection of the target SARS-CoV-2 RdRp gene. Nasopharyngeal swabs (Sterile Foam Tipped Applicator, 25–1506 1PF 100, Puritan, Guilford, ME) were collected in sterile tubes (Vacuette, 6 mL, no additive, Greiner Bio-one, Monroe, NC) and tested according to the manufacturer's instructions, except that the swab samples were mixed with the elution/lysis reagent for 30 instead of 10 s to resolve issues of increased viscosity samples. Swabs were kept at room temperature for a maximum of 2 h before testing and were processed in BSL-2 biosafety hoods.

The Xpert Xpress assay amplifies two regions of SARS-CoV-2 genome, the nucleocapsid gene (N2-target) unique to SARS-CoV-2 and the envelop gene (E-target), common to both Sarbecovirus, SARS-CoV-2 and SARS-CoV-1. Samples were collected with either the Xpert

Nasopharyngeal Sample Collection Kit for Viruses (Cepheid) or with a nylon swab (Copan Diagnostics, Murrieta, CA), placed in VTM (Cepheid or Copan, respectively), vigorously mixed, transferred to a cartridge, and analyzed with the Cepheid Infinity system according to the manufacturer's instructions. Both ID-NOW and Xpert assays include internal amplification controls. External positive and negative controls were run with every new reagent shipment or lot and every 30 days of use.

Calibration of Xpert C values

To calibrate sample viral loads on our Xpert instrument we performed serial dilutions of two purified cultured SARS-CoV-2 viral stocks of known concentration and assayed with the Xpert and ID-NOW (Supplementary Methods). We then fitted a linear model to the observed Xpert C values from each dilution to estimate the nasopharyngeal viral loads based on the Xpert C values obtained from clinical samples (Fig.-1 ).

Fig. 1

Calibration of Xpert assay using serial dilutions of purified SARS-CoV-2 viral stock (NY: New York strain in circles, WA: Washington strain in triangles) and comparison with the ID-NOW assay. For the Xpert assay, 200 µL of each diltution was added to the Xpert cartridge, whereas for the ID-NOW, a foam tipped swab was dipped into each dilution and the manufacturer's protocol was followed. The gray-shadded bands represent the 95% confidence intervals of the fitted linear model for each target. ID-NOW results are plotted in the lower panel for each dilution to represent a concentration in genomic equivalents (GE) per mL of VTM equivalent to the Xpert results, with positive results in blue and negative results in orange. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Viral culture

We cultured 8 nasopharyngeal samples with Xpert N2 C values ≥25, of which 3 were Cepheid positive and ID-NOW negative, as well as 5 samples positive by both methods which had N2 C values <25. Samples in VTM were taken into the BSL-3 laboratory and cultured in Vero CCL.81 cells (ATCC) in serum-free EMEM for 5 days at 37 °C and 5% CO2. Supernatants were passed into another flask of Vero cells at 48 h after which cytopathic effects were visualized. If no cytopathic effect was seen, cultures were incubated for up to 14 days. All cultures with positive cytopathic effect were confirmed by real time RT-PCR specific for SARS-CoV-2 RNA.

Results

We tested 993 individuals with both ID-NOW and Xpert at our institution between April 17 and August 7, 2020. There were 114 positive results by either Xpert or ID-NOW, representing an overall positivity rate of 11.5%. There were no statistically significant differences between SARS-CoV-2 positive and negative patients in age, sex, or visit type (Supplementary Table-1). As a result of our criteria for testing with ID-NOW, most patients were seen in the emergency department or admitted to the hospital. Although positive patients had higher mortality, shorter testing to death interval, and longer testing to discharge interval, these differences were not statistically significant after adjustment for multiple comparisons.

The overall agreement between ID-NOW and Xpert was 96.3% (Cohen's kappa = 0.786), the percent positive agreement was 70.0% (95% CI = 60.5–78.4%) and the overall percent negative agreement was 99.5% (95% CI = 98.8–99.9%) (Table-1). We considered a sample positive when either the N2 or the E targets were amplified with a C of 45 or lower. In 28 (2.8%) Xpert results only the N2 target was detected and there were no samples with only the E gene detected. Four positive ID-NOW samples were not detected by Xpert in a concurrent sample. One of these patients was previously positive with the Xpert assay and therefore can safely be considered true positive. For the other 3 patients, we only had one test performed on each instrument, so they could be ID-NOW false positive or Xpert false negative results. However, the specificity of amplification assays for SARS-CoV-2 is very high [3,4,[6], [7], [8]] and we favor that these were Xpert false negatives, probably due to sampling inadequacy. -

There were 33 patients with negative ID-NOW and positive Xpert results, and their N2 C values were above 37. Only one patient (3%) had a clinical profile consistent with COVID-19 pneumonia at the time of testing but this patient also had acute coronary syndrome and decompensated heart failure complicating evaluation of symptom onset. Two patients had prolonged (> 12 days) history of symptoms of COVID-19 and another three patients had COVID-19 diagnosed > 1 month before testing. The remaining patients (27/33, 81.8%) had no symptoms of COVID-19.

We also performed viral culture from nasopharyngeal samples collected in VTM from 3 patients with Xpert N2 C of 41.7/42.1/39.5, and concurrent negative ID-NOW. Viral growth was not seen up to 14 days of culture from any of the 3 ID-NOW negative samples and 5 additional samples with Xpert N2 C values  ≥25, whereas 5 nasopharyngeal samples with N2 C values <25 grew virus by culture confirmed by RT-PCR (results not shown).

Fig.-2 and Table-2 show ID-NOW results compared with Xpert C values. All false negative ID-NOW results had N2 C 37.3 and E C 35.2. The agreement between ID-NOW and Xpert results with E or N2 C values 35 or 37, respectively, was 100%, and for E or N2 results > 35 or 37 the agreement was 31.6% and 37.7%, respectively.

Fig. 2

Violin plots of the distribution of cycle threshold (C) values and corresponding genomic equivalents (GE) per mL of VTM obtained by the Xpert assay. The Xpert assay targets the N2 (left) or E (right) SARS-CoV-2 genes. Nasopharyngeal samples collected in parallel with dry swab samples were resulted as positive (red) or negative (blue) by the ID-NOW assay. The width of each violin plot represents the density distribution and the lines within each violin plot represent percentiles 0, 25, 50 (median), 75, and 100 of the C values. Each dot is a test result.

Table 2

Comparison of ID-NOW results with Xpert results by cycle threshold value (C) of the amplification targeting either the N2 or the E target regions. PPA: Percentage of positive agreement.

ID-NOW	Xpert N2 CT Values			Xpert E CT Values
	<=37	>37	Total	<=35	>35	Total
Positive	57 (51.8%)	20 (18.2%)	77 (70%)	63 (76.8%)	6 (7.3%)	69 (84.1%)
Negative	0 (0%)	33 (30%)	33 (30%)	0 (0%)	13 (15.9%)	13 (15.9%)
Total	57 (51.8%)	53 (48.2%)	110 (100%)	63 (76.8%)	19 (23.2%)	82 (100%)
PPA	100%	37.7%	70%	100%	31.6%	81.1%

We used a linear model to calculate the estimated nasopharyngeal viral loads in samples with positive Xpert results. The calibration shows high linearity (R2>0.982) and reproducibility between two different strains (Fig.-1). Fig.-3 shows N2-estimated viral loads between April 17 and August 7, 2020, grouped by week, and color coded by ID-NOW result. The horizontal line in the graph shows that the ID-NOW detected estimated viral loads  945 GE/mL (N2 C ≤37.0) with 100% sensitivity corresponding to 100% sensitivity of the ID-NOW. Similarly, testing dilutions of purified viral stocks show that the ID-NOW was positive for the NY strain at 320 GE/mL (Fig.-1) and the WA strain at 103 GE/mL. The lowest Xpert viral load that was also detected by ID-NOW in clinical samples was 15 GE/mL, corresponding to a N2 C of 43.4.

Fig. 3

Distribution of C values of the Xpert SARS-Cov-2 N2 target per week, color coded by the parallel ID-NOW results as positive (red) or negative (blue). Boxes represent the median (horizontal line) and interquartile range (top and bottom of the box) for each week. The horizontal line in the graph shows the Xpert threshold in GE/mL viral load (VL) and corresponding Xpert C value corresponding to 100% sensitivity of the ID-NOW. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Discussion

This report demonstrates that while the overall sensitivity of the ID-NOW assay compared to RT-PCR was 70%, the ID-NOW has very high sensitivity for detection of patients with high levels of SARS-CoV-2 RNA (100% for estimated viral loads ≥ 945 GE/mL). The clinical impact of rapid SARS-CoV-2 detection methods with lower sensitivity, such as the ID-NOW and point-of-care antigen tests, depends on the balance between the advantages of saving time in detecting infected patients and the negative consequences of not detecting patients with false negative results. The diagnostic sensitivity of all SARS-CoV-2 assays may be impaired by timing of the sampling relative to the course of the disease, inappropriate sampling technique, type of swabs used, transportation media, and other pre-analytical factors. In a study from New York City during the first surge, the clinical sensitivity of a SARS-CoV-2 RT-PCR assay was estimated to be as low as 58% in repeat tested patients, possibly due to sampling too early [9], whereas in another study clinical sensitivity was 82% to 97% [10]. Other contributors to loss of sensitivity include pre-analytical issues such as the dilutional effect of VTM versus dry swabs for the ID-NOW [1] and the mixing time of the swabs with viscous samples, which we determined to require 30 s (results not shown).

Our results further demonstrate that ID-NOW false negative patients have low viral loads (<945 GE/mL). This is supported by our findings that cultured swabs from 3 patients who were RT-PCR positive (N2 C values over 39) and negative by ID-NOW were unable to yield viral growth in cell culture assays. Similarly, other studies also reported the inability to culture virus from cases with high C values [11], [12], [13], [14]. The first important consideration when failing to identify patients with very low viral loads is the likelihood of developing severe disease. In hospitalized patients, high viral loads were associated with more severe disease and higher mortality [15,16]. In Italy [17], New York City [18], and Detroit [19] a significant reduction in viral load since the surge was associated with significantly decreased mortality.

We noticed that the average weekly SARS-CoV-2 C values obtained with Xpert increased and correspondingly the viral loads decreased significantly since the first surge in March and April 2020 (Fig.-3 and Supplementary Figs. 1 and 2). An exception was the first week of August, with 4 patients with high viral loads, all positive by ID-NOW. Two of the patients had symptoms consistent with COVID-19 and one was admitted with severe hypoxemia and COVID-19 pneumonia. While our study was not designed to relate C values to infectivity or severity, these results provide support for reporting and tracking C values. Importantly, reporting of individual C values must disclose all the limitations associated with using C values to estimate nasopharyngeal viral loads [20], including timing and variability of sample collection and processing, differences between assays, and unavailability of calibration and commutable standard materials. Notably, the FDA has recently recognized the potential usefulness of C value reporting [21].

The second important consideration regarding patients with very low viral loads is their infectivity. Several studies have failed to culture SARS-CoV-2 from samples with low levels of viral RNA [11], [12], [13], [14]. The lowest level of RNA in samples from which SARS-CoV-2 was isolated varies from 103 [12,13], to 105 [14] and 106 RNA copies/mL [11]. In general, it is known that cells shed a large number of viral nucleic acid fragments for each viable virion. In the study reported by Bullard et al. [12] a C level of 20, which represents about 107–108 RNA copies/mL ( [22] and this study), corresponded to about 1000 culturable virions/mL, suggesting that there are about 104–105 detectable RNA copies for each viable virion. This ratio is likely to increase as the infection progresses into the recovery phase, since cultivable virus was isolated only from the first 5–8 days post-symptom onset, even when viral RNA loads were greater than 105 copies/mL [23]. Shedding of viral RNA persists in many patients for more than a month [9,24,25] and is associated with high frequency of SARS-CoV-2 antibodies [26]. There is a significant correlation between viral loads and infectivity [27], and after 5–7 days of symptom onset both viral loads [28], ability to culture the virus, and infectivity measured from transmission events [29,30] sharply decline. However, it is inappropriate to base protective and preventive measures solely on C values as some studies reported the presence of culturable virus in a small proportion of patients with high C values and up to 32 days post symptom onset, especially in patients with severe disease [31,32].

While our study was not designed to detect infectivity, all but one of the patients with low viral load and negative ID-NOW results (32/33, 97%) had either no symptoms of COVID-19 or were in the recovery phase of symptomatic COVID-19 infection for at least 12 days, when they were less likely to be infectious. Additionally, we cultured 3 of the samples with low viral loads and negative ID-NOW results and failed to grow SARS-CoV-2, lending further support to the notion that patients with high C values on RT-PCR tests and negative ID-NOW results may be non-infectious. Together, these data support current manufacturer's recommendations that ID-NOW is indicated in the first 7 days of symptoms.

In summary, our study demonstrates that the ID-NOW assay detected patients with viral loads that have been associated with higher infectivity or risk of severe disease in other studies in that the ID-NOW had 100% sensitivity at viral loads ~ 103 GE/mL or greater as assessed by a highly sensitive RT-PCR method. Our study further substantiates the possibility that reporting estimated viral loads from molecular SARS-CoV-2 testing and using rapid assays such as the ID-NOW to detect individuals with high viral loads may assist in the timely identification of outbreaks allowing for aggressive contact tracing and containment.

Declaration of Competing Interest

The authors have no conflicts of interest to declare.