Evaluation of Abbott ID NOW COVID-19 POC test performance characteristics and integration in the regional health network workflows to improve health care delivery.

With the recent global surge of SARS-CoV-2 Delta variant, there continues to be high demand for COVID-19 diagnostic testing. Abbott ID NOW is a rapid, CLIA-waived, COVID-19 diagnostic test ideally suited for use in urgent care settings or where access to diagnostic testing is limited. In this study we describe the results of rigorous validation of ID NOW and post-implementation study of POC test utilization patterns within community hospitals and clinics. Performance of ID NOW was validated by comparison of the results from 207 consecutive, paired, specimens tested on the ID NOW and on the m2000/Alinity m platforms. Once validated, ID NOW devices were placed for clinical use at four regional hospitals and clinics. We found that the ID NOW and m2000/Alinity m positive and negative percent agreement were 94.5% (95% CI, 85.1% to 98.1%) and 99.3% (95% CI, 96.4% to 99.9%), respectively. As of August 2021, a total of 2,301 tests were performed by ID NOW at individual regional network sites. The population tested consisted of 55.5% White and 42.9% Black patients, with Black patients presenting predominantly in the hospitals, while White patients were more evenly distributed between hospital and clinic sites. Disease prevalence observed among patients tested by ID NOW (12.3%) was aligned with overall prevalence seen at regional sites (11.3%). In summary, the ID NOW test can provide rapid and accurate results in a variety of near-to-patient and POC settings. If used correctly, it could serve as a valuable diagnostic tool to enable equal access to care and improve healthcare delivery within large health network systems.

Introduction

With the recent surge of SARS-CoV-2 Delta variant incidence across the US during the spring and summer of 2021, there continues to be high demand for COVID-19 diagnostic testing. To meet demands for population-wide COVID-19 testing, many clinical laboratories, including our own, have implemented several SARS-CoV-2 diagnostic tests ranging from highly sensitive RT-PCR to rapid but less sensitive antigen tests. Although readily available, molecular testing is mostly limited to high complexity, Clinical Laboratory Improvement Amendment (CLIA) licensed clinical laboratories. Many regional, community and rural hospital laboratories do not have capability to perform molecular COVID-19 diagnostic testing due to their size or lack of expertise. In these settings availability of rapid and reliable CLIA-waived diagnostic tests is very important to help facilitate adequate health care delivery. Several rapid molecular tests have received Federal Drug Administration (FDA) Emergency Use Authorization (EUA) for CLIA-waived settings, such as testing at the point of care (POC) by minimally trained staff [1]. With capability to produce results in minutes, these platforms are best fit for clinical environments where timely result is necessary, such as patients presenting to EDs, urgent care facilities, or community hospitals and clinics that lack rapid access to diagnostic testing.

MUSC Health is an integrated health delivery system where our institution (MUSC Charleston) serves as the hub connected to four regional medical centers across the state of South Carolina (SC), comprising Regional Health Network (RHN). The RHN hospitals serve the following counties: Marion, Florence, Lancaster and Chester. According to the SC Department of Health data, the communities served by RHN medical centers have consistently had higher COVID-19 incidence compared to Charleston County, served by our institution [2]. Thus, in addition to expedited centralized testing performed at MUSC Charleston, an access to rapid testing that could be performed at POC (ie at each individual RHN site) in symptomatic patients was necessary to help proactively contain further outbreaks. Abbott ID NOW seemed to be an obvious POC testing choice due to its CLIA waived status and short turn-around-time of 13 min or less. However, there have been controversial reports regarding the clinical performance of this test, with reports of clinical sensitivity ranging from 55% to 91% [3], [4], [5], [6], [7], [8], [9], [10], [11]. These reports necessitated a comprehensive evaluation of ID NOW performance in our patient population prior to implementation.

In this study we describe the results of rigorous validation of ID NOW in COVID-19 symptomatic patients presenting to the ED of the small South Carolina regional hospital and effectiveness of the combined diagnostic strategy approach in improving healthcare delivery within communities.

Materials and methods

Abbott ID NOW validation study population and sample collection

Paired dry nasal swabs (NS) and nasopharyngeal swabs (NPS) placed in 3 mL sterile saline were collected from 207 consecutive, symptomatic patients presenting to the ED from 6/3/20 to 7/19/20 during the second wave of COVID-19 in South Carolina. The swabs provided with the ID NOW test kit were used to collect nasal swabs from both nostrils according to manufacturer’s instructions. Dry nasal swabs were replaced in packaging and transported to the local laboratory for immediate testing. Flocked NP swabs were used and collected according to hospital protocol in 3 mL of sterile saline. These specimens were transported on the cold packs to the central laboratory where the testing was performed within 24 h of collection. The signs and symptoms used to qualify patients followed the CDC guidelines in place at the time and included fever, cough, shortness of breath, nausea, vomiting, diarrhea, fatigue, headache and loss of taste and/or smell. All testing was performed by qualified laboratory personnel.

Abbott ID NOW validation testing

Dry NS samples provided with the kit were tested using the Abbott ID NOW (Abbot Diagnostics, Scarborough, ME), a CLIA-waived POCT that uses an isothermal nicking enzyme amplification reaction and fluorescently labeled molecular beacons for detection of SARS-CoV-2 RNA. A unique region of SARS-CoV-2 RdRp gene and an internal control are targeted by this test. It delivers test results in 5 to 13 min. The manufacturer’s claimed limit of detection is 125 copies/mL [12], [13]. All validation testing was performed at one of the RHN sites, MUSC Health Marion Medical Center Clinical Laboratory.

The NPS samples were transported via courier to MUSC Charleston on ice and tested by the Abbott RealTime SARS-CoV-2 test using either the m2000 sp/rt or Alinity m systems (Abbott Molecular, Des Plaines, IL) within 24 h of collection. Real-time RT-PCR amplification and detection of three targets, SARS-CoV-2 RdRp gene, SARS-CoV-2 N gene, and an internal control (hydroxypyruvate reductase gene from the pumpkin plant), takes place simultaneously in the same reaction. The test has a reported limit of detection of 100 copies/ml [12], [14]. Although the Abbott RealTime SARS-CoV-2 test is a qualitative test we have established the relationship between the cycle number (Cn) values and genomic copies/ml using dilutions of commercially available reference material quantified by digital droplet PCR (AccuPlex SARS-CoV-2 verification panel, SeraCare) [15].

Abbott ID NOW POC clinical testing implementation

In this study each individual health network site was defined as a POC site. ID NOW devices were placed at all four RHN hospitals and associated family medicine, primary care and/or urgent care clinics, where significant influx of symptomatic patient has already been seen or was expected. All other patients were continued to be tested by centralized molecular testing at MUSC Charleston site. The patients tested at hospital sites were symptomatic patients who presented predominantly to the ED. The clinic patients presented mostly with mild/moderate symptoms consistent with COVID-19 infection. Clinical data for ID NOW was summarized for the time period ranging from 09/01/2020 to 08/23/2021.

Data analysis

Positive percent agreement (PPA) and negative percent agreement (NPA) along with corresponding Wilson 95% confidence intervals (CI) were calculated using a clinical method validation software package (Analyze-it). The paired analysis of positive PCR results was performed using univariate distribution plots (Analyze-it software). Population demographics and PCR testing statistics were provided by MUSC Health Analytics using Tableau software (Mountain View, CA) and EPIC SlicerDicer (Verona, WI).

Results

Total number of patients tested at MUSC Marion during the validation process (June and July 2020) was 793, with 122 positive results, resulting in overall community disease prevalence of 15.4%. The patient population tested consisted of 49.8% Black, 47.1% White, and approximately 3% others. A total of 207 symptomatic patients presented to MUSC Marion ED during this time period. The parallel testing of these patients on ID NOW and m2000/Alinity m revealed that the incidence of COVID-19 among these patients is 26.5%, significantly higher than overall community positivity rates.

PPA and NPA for the Abbott ID NOW in our test population are summarized in Table 1 . Among a total of 207 consecutive symptomatic patients tested, there were three false negative and one false positive results on ID NOW, resulting in PPA of 94.5% (95% CI, 85.1% to 98.1%) and NPA of 99.3% (95% CI, 96.4% to 99.9%).

Table 1

Positive and negative percent agreement between ID NOW and m2000/Alinity m assays.

	ID NOW
m2000/Alinity m	Positive	Negative	Total
Positive	52	3	55
Negative	1	151	152
Total	53	154	207
PPA (95% CI)	94.5% (85.1% to 98.1%)
NPA (95% CI)	99.3% (96.4% to 99.9%)

To further evaluate three false negative ID NOW results, distribution of the RT-PCR cycle number (Cn) values was examined for the positive NP swab samples. Although the Abbott SARS-CoV-2 assays have very similar performance characteristics and the same LOD on both the m2000 and Alinity m systems the Cn values were on average 14.1 lower on the m2000, due to the fact the m2000 protocol does not collect data during the first 10 cycles and software modifications on the Alinity m [15]. All positive results obtained on Alinity m (n = 21) were concordant with ID NOW, with Cn values ranging from 16.65 to 38.35. The highest Cn on the Alinity m corresponds to a viral burden of approximately 400 copies/ml. The positive agreement between ID NOW and m2000 methods (n = 34) is summarized in Fig. 1 . As shown in the figure, the false negative results on ID NOW occur at the high Cn values, which correspond to low viral copy values. The Cn values ranged from 2.44 to 23.53 for concordant samples, while Cn values for discordant samples were 24.85, 26.08 and 30.73, corresponding to viral burdens between 1,000 and < 100 copies/ml.

Fig. 1

Distribution of all RT-PCR positive results relative to the cycle number (Cn) values. Only values analyzed on Abbott m2000 analyzer (n = 34) are shown here. All positive values obtained on Alinity m analyzer (n = 21), with Cn values ranging from 16.65 to 38.35, were concordant with Abbott ID NOW.

As of August 2021, a total of 62, 550 patients were tested across different MUSC RHN locations, including both hospitals, primary and urgent care clinics. Of those, 2,301 were performed by ID NOW at individual RHN sites. The population testing statistics and demographics are summarized in Table 2 . The population tested by ID NOW consisted of 55.5% White, 42.9% Black patients and approximately 3% others. Overall disease prevalence in all RHN patients was 11.3%. Disease prevalence observed among patients tested by ID NOW was 12.3%. Analysis of ID NOW utilization at RHN hospitals vs clinics, revealed that while overall testing volumes were approximately the same, the racial distribution differed between hospitals and clinics. Black patients were seen predominantly in the hospitals, whereas White patients were more evenly distributed between hospitals and clinics (Fig. 2 ). Also, as shown in Fig. 2, utilization of ID NOW in clinics where patients presented with generally milder symptoms resulted in the same positivity rates as seen in more severely ill patients presenting to the hospitals.

Table 2

Summary statistics of SARS CoV-2 testing across all MUSC RHN locations.

	Number of Patients Tested	Number of Positive Tests	% Positive Tests (range)
All RHN Patients-All Tests	62,550	7,051	11.3%
All RHN Patients -ID NOW Testing	2,301	283	12.3%
Black	988
White	1,276
Other	37

Fig. 2

Abbott ID NOW testing utilization and positivity rates across RHN hospitals and clinics. *Note: The positivity rates for Black patients seen at RHN clinics could not be calculated due to low number (<10) of positive patients at each individual testing site.

Discussion

In this study we evaluated performance characteristics of ID NOW and conducted post-implementation clinical utilization analysis of POC COVID-19 testing to determine the effectiveness of this approach within racially and socioeconomically diverse communities served by the MUSC Health network. Our goal was to ensure more equitable access to care for our entire patient population by providing a rapid diagnostic test for symptomatic patients residing, particularly those in remote and/or underserved area.

The performance of ID NOW was evaluated by analysis of 207 paired dry NS samples tested on ID NOW and NPS collected in 3 mL saline and tested on m2000 and Alinity m instruments. The samples were collected from symptomatic patients meeting clinical criteria of COVID-19. Of note, ID NOW testing was preformed within the local clinical laboratory by several experienced and well-trained laboratory technical staff. The overall positivity rate in our validation patient population was 26.5%. Our study design, methods evaluated, and population tested are very similar to the studies recently conducted by Harrington et al and McDonald et al [3], [10]. Both groups reported significantly lower positive agreement (75–79%) than our findings of 94.5%. While McDonald et al did not characterize false negative results, Harrington et al reported Cn values for discordant samples as low as 6.79 and concordant samples with Cn values as high as 31.01. This suggests that, aside from inferior analytical sensitivity, specimen collection and/or integrity may be important contributors to false negative ID NOW results. In contrast, as shown in Fig. 1, we observed more consistent distribution of concordant and discordant data across Cn values. All discordant results observed were at Cn values > 24. Based on our results, the analytical sensitivity of ID NOW appears to be between 500 and 1,000 copies/mL, which is in line with ID NOW limit of detection of 511 copies/mL, reported by Fung et al [16] and significantly better than the other estimates published in literature [3], [8].

In addition, a recent Cochrane systematic review of studies of rapid-point-of-care molecular tests for SARS-CoV-2 reported sensitivity and specificity for ID Now of 76.8% (95% CI 72.9% to 80.3%) and 99.6% (95% CI 98.4% to 99.9%), respectively for the six studies reviewed [17]. They also reported that ID NOW sensitivity rose to 100% when only samples with cycle treshold (Ct) values ≤ 30 were considered which is consistent with our findings. Moreover, clinical significance of detecting low levels of viral RNA is still not clear. Additional clinical studies are needed to determine if the false negative ID NOW results truly reflect the failure to detect actionable, active disease.

We had a single false-positive ID NOW result that may have resulted from specimen carry-over cross-contamination, poor collection of the NP swab or degradation of a low level of SARS-CoV-2 RNA during transport to the central laboratory. Taken together, our findings indicated that the performance characteristics of the ID NOW are comparable to the Abbott SARS-CoV-2 RNA assays run on the m2000 and Alinity m instruments in symptomatic patients.

Limitations to our validation study include that ID NOW was performed in the laboratory rather than as a true POC test by non-laboratory staff, the prevalence of infection among our study population was high, the ED population may represent the more severe end of the disease spectrum with higher viral burdens, and the confounding variable of the difference is in sample types used for ID NOW and the comparator tests. Comparing across anatomical sites, nasal cavity versus nasopharynx, may say more about the preferred anatomical niche of the virus rather than any meaningful difference in sensitivities between the assays used in this study.

Despite the validation study drawbacks outlined above, our post-implementation data of POC testing in various clinical settings suggest that this approach is suitable for community-wide deployment to help facilitate COVID-19 infection diagnosis. This is particularly important in the states such as South Carolina where poverty rates are above the national average and the proportion of Black or other racial/ethnic minority patient population is significant [18]. Several recent studies have reported racial disparities in SARS-CoV-2 prevalence as well as correlation of SARS-CoV-2 test positivity and lower socioeconomic status [19], [20]. Hsiao et al reported that black race is associated with higher SARS-CoV-2 test positivity, while Palacio et al reported that disease positivity was correlated with lower socioeconomic status but not race. These studies highlight the need for testing in underserved, economically disadvantaged, racially and ethnically diverse areas. As shown in Fig. 2, our ID NOW utilization patterns revealed fairly equal distribution of patients between RHN hospitals and clinics in terms of total number of patients tested. However, racial distribution was quite different with Black patients presenting predominantly in the hospitals, while White patients were more evenly distributed. Disease prevalence of 12.3% in population tested by ID NOW is very much in line with the overall disease prevalence of 11.3% established by molecular testing in RHN patient population, suggesting that ID NOW can be effectively used in patients presenting with wide range and severity of symptoms. This is further supported by equal positivity rates observed between the hospitals and clinics (Fig. 2).

By customizing our rapid testing approach to the needs of our community hospitals and clinics, we were able to provide immediate access to testing to symptomatic patients residing in high disease prevalence areas across the state. Our hope is that this strategy will not only help limit disease outbreaks within these communities but also lead to improved outcomes. A large cohort study conducted at the US Department of Veteran Affairs (VA), including close to 6 million patients revealed that while Black and Hispanic patients were more likely to test positive for COVID-19 than White patients, 30-day mortality did not differ by race [21]. Since VA patients generally have more equal access to care compared to general population, these results suggest that access to care rather than the race is stronger predictor of mortality in COVID-19 patients. This study also highlights a need for tailored testing strategy approach, such as described here, to contain and prevent disease outbreaks in racial and ethnic minority communities.

Conclusion

In summary, rapid testing has become an important tool in our COVID-19 diagnostic toolkit. It is now deployed within MUSC Health system as a true POC test for rapid identification and isolation of infected individuals, appropriate use of personal protective equipment, and initiation of special workflows required for emergency care and in time-sensitive outpatient settings. Future population-based outcome studies are needed to evaluate the effectiveness of this approach in enabling equal access to care and improving healthcare delivery to the communities across our health network.