Utility of COVID-19 antigen testing in the emergency department.

BACKGROUND: The BinaxNOW coronavirus disease 2019 (COVID-19) Ag Card test (Abbott Diagnostics Scarborough, Inc.) is a lateral flow immunochromatographic point-of-care test for the qualitative detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) nucleocapsid protein antigen. It provides results from nasal swabs in 15 minutes. Our purpose was to determine its sensitivity and specificity for a COVID-19 diagnosis.
METHODS: Eligible patients had symptoms of COVID-19 or suspected exposure. After consent, 2 nasal swabs were collected; 1 was tested using the Abbott RealTime SARS-CoV-2 (ie, the gold standard polymerase chain reaction test) and the second run on the BinaxNOW point of care platform by emergency department staff.
RESULTS: From July 20 to October 28, 2020, 767 patients were enrolled, of which 735 had evaluable samples. Their mean (SD) age was 46.8 (16.6) years, and 422 (57.4%) were women. A total of 623 (84.8%) patients had COVID-19 symptoms, most commonly shortness of breath (n = 404; 55.0%), cough (n = 314; 42.7%), and fever (n = 253; 34.4%). Although 460 (62.6%) had symptoms ≤7 days, the mean (SD) time since symptom onset was 8.1 (14.0) days. Positive tests occurred in 173 (23.5%) and 141 (19.2%) with the gold standard versus BinaxNOW test, respectively. Those with symptoms >2 weeks had a positive test rate roughly half of those with earlier presentations. In patients with symptoms ≤7 days, the sensitivity, specificity, and negative and positive predictive values for the BinaxNOW test were 84.6%, 98.5%, 94.9%, and 95.2%, respectively.
CONCLUSIONS: The BinaxNOW point-of-care test has good sensitivity and excellent specificity for the detection of COVID-19. We recommend using the BinasNOW for patients with symptoms up to 2 weeks.

BACKGROUND

Although specific dates vary per different reports, on December 29, 2019, 4 cases of “pneumonia of unknown etiology” were officially reported by local Chinese hospitals. Bronchoalveolar lavage fluid samples revealed a virus with features typical of the beta‐coronavirus 2B lineage of coronavirus, and on December 31, 2019, Chinese authorities alerted the World Health Organization (WHO). By January 8, 2020, a novel coronavirus was officially announced as the cause of the rapidly spreading illness. The first case reported outside of China was in Thailand on January 13, 2020. On January 20, 2020, the Centers for Disease Control and Prevention (CDC) confirmed the first US case. Ten days later, the WHO declared a global health emergency, and coronavirus disease 2019 (COVID‐19) was declared a pandemic on March 11, 2020. As of August 1, 2021, this pandemic has spread to >200 countries, areas, or territories across the world, with an estimated 197 million cases and 4.2 million deaths.

Current COVID‐19 testing strategy

A key aspect in controlling the spread is the need for accurate and rapid COVID‐19 testing. Molecular testing, as currently employed, is focused on symptomatic individuals and captures only a portion of those who are contagious. Based on CDC estimates, 40% of infections are asymptomatic, and the percentage of transmission occurring before symptom onset is 50%. Thus, these asymptomatic individuals may unknowingly infect others with whom they come into contact. Laboratory‐based reverse transcriptase (RT) polymerase chain reaction (PCR) platforms with high sensitivity and specificity are the gold standard for the direct detection of viruses. However, the numbers of platforms, technical personnel requirements, reagent and collection supply limitations, turnaround times, and costs have presented significant challenges. In many cases, including in the emergency department (ED), it takes several days or more before results are reported to the patient. This can increase the spread of contagion as a person awaits results. Acknowledging the important roles currently served by these tests, the fact remains that diagnostic testing, as currently implemented, will not stop this pandemic.

What is needed?

A paradigm shift is urgently required to address the fundamental need of identifying individuals who are contagious. A new strategy should require a widely deployable diagnostic test with rapid time to results. It must be cost‐effective, use a readily accessible sample type (eg, nasal swab), require no instrumentation, and be rapidly scalable (tens of millions per month) to enable frequent patient testing. In addition to these criteria, a simple‐to‐use test that directly detects the severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) antigen would be ideal. Although sensitivity is important, the primary goal is to detect individuals who shed high levels of infectious virus. Evidence suggests that most individuals with low levels of virus pose a limited threat of transmission. , , , , , , , , A highly specific, rapid, point‐of‐care ED available test could provide improved SARS‐CoV‐2 containment thru more efficient detection and subsequent isolation of individuals.

In this multicenter study of 767 emergency department patients with COVID‐19 symptoms or exposure, a point‐of‐care antigen test showed good sensitivity (84.6%) and excellent specificity (98.5%) for the detection of COVID‐19.

Testing strategy evaluated by this study

The BinaxNOW COVID‐19 Ag Card point‐of‐care test (Abbott Diagnostics Scarborough, Inc.) is an excellent candidate as a first line of defense to identify people who are currently symptomatic, or at risk, and should isolate themselves to help prevent disease spread. It is an immunochromatographic membrane assay that uses highly sensitive antibodies to detect the SARS‐CoV‐2 nucleocapsid protein from nasal swab specimens. SARS‐CoV‐2‐specific antibodies and a control antibody are immobilized onto a membrane support as 2 distinct lines that, combined with other reagents/pads, construct a test strip. This test strip, and a well to hold the swab specimen, are mounted on opposite sides of a cardboard, book‐shaped hinged test card (Figure 1). It is intended for the qualitative detection of the nucleocapsid protein antigen from SARS‐CoV‐2 in nasal swabs from individuals suspected of COVID‐19 within the first 7 days of symptom onset. This test delivers results in 15 minutes with no instrumentation using lateral flow technology. If accurate, a test with these characteristics would provide great operational and public health advantages for the ED. Our purpose was to determine the sensitivity, specificity, and negative and positive predictive values of this test in a population of suspected patients with COVID‐19.

FIGURE 1

CONSORT Flow diagram demonstrating study enrollment

METHODS

This was a prospective (defined as the data were gathered before the gold standard diagnosis was known), institutional review board–approved, multicenter study that enrolled a convenience sample of patients from 27 EDs and hospital wards. The specific sites are listed in the appendix. Eligible patients were suspected by a healthcare practitioner of having a COVID‐19 infection or exposure. Patients were excluded if they had active nose bleeds or acute facial injuries, were currently enrolled in a study to evaluate any investigational drug not cleared by the US Food and Drug Administration (FDA), had already participated in this study, were unable or unwilling to provide informed consent, or were a member of a vulnerable population deemed inappropriate for inclusion by the site's principal investigator. Vulnerable groups are frequently defined to include children, the disabled, patients with HIV/AIDS, the elderly, indigenous peoples, refugees, and prisoners. There was no age exclusion. Vaccination status was not recorded as no FDA cleared vaccines were available during data collection for this trial.

Performance of the test

After informed consent and collection of the gold standard nasopharyngeal swab, research staff collected 2 nasal swabs from each patient. One nasal swab was tested immediately (within 10 minutes) using the BinaxNOW COVID‐19 Ag test without elution into viral transport media (VTM). To perform the test, 6 drops of extraction reagent were added to the top hole of the swab well from a dropper bottle. The patient's nasal sample swab was then inserted into the test card through the bottom hole of the swab well and firmly pushed upward until the swab tip was visible through the top hole. Next, the swab was rotated 3 times and the card was closed, bringing the extracted sample into contact with the test strip. Test results were interpreted visually at 15 minutes based on the determination of the presence or absence of visually detectable pink/purple‐colored lines (with no allowance for a gray zone), and results were recorded using photographs.

The second nasal swab was placed in VTM and stored at −2°C to −8°C until shipped to the core lab. The order of nasal swab collection was randomized according to subject identification (ID) number. For those with an even‐numbered subject ID, the first nasal swab collected was placed in the BinaxNOW device and the other swab was eluted into VTM. For patients with an odd‐numbered subject ID, the opposite sequence occurred.

Nasal swab sample collection was performed using gentle rotation, with the swab pushed into the nostril until resistance was met at the level of the turbinate (<1 in. into the nostril). The swab was then rotated several times against the nasal wall, then slowly removed, and collection was repeated using the same swab in the opposite nostril. The same process was repeated with a second swab; however, the samples were obtained in the opposite nare order than the first swab.

Eluted VTM nasopharyngeal samples were tested at the core laboratory, blinded to the BinaxNOW results, on the Abbott RealTime SARS‐CoV‐2 platform (Abbott Diagnostics) provided by the sponsor. All reference testing was performed according to product instructions. The PCR cycle threshold (Ct) reported by the Abbott RealTime SARS‐CoV‐2 does not include the first 10 thermal cycles that are dark cycles; therefore, the categorical analysis of low viral RNA levels is performed with the Abbott RealTime SARS‐CoV‐2 test Ct ≥23, which corresponds to Ct ≥33 for other PCR methods. Each patient's demographic information, self‐reported symptom data, reference results, and BinaxNOW COVID‐19 Ag test results were recorded in an electronic data capture (EDC) system.

Blinding and external controls

Test operators were clinical staff blinded to the reference standard results, representative of intended users, and did not have professional training as laboratory technicians. Each day BinaxNOW COVID‐19 Ag testing was performed, external control testing was done before subject testing using positive and negative control swabs. Results were recorded in a control testing log and entered into the EDC system.

Analytic methods

Study enrollment was planned to continue until 120 patients symptomatic for ≤7 days and positive for COVID‐19 by the reference method were identified from an estimated 2200 patients. Planned data analysis was for positive and negative agreement, as well as sensitivity and specificity, between the reference standard and the BinaxNOW assay for a diagnosis of SARS‐CoV‐2 infection. Alternative diagnoses were not recorded nor adjudicated. An interim analysis was conducted after 30 reference positives were enrolled and used for an FDA Emergency Use Authorization (EUA) submission (which was approved). Finally, operation and ease of results interpretation of the BinaxNOW COVID‐19 Ag testing was evaluated by a questionnaire completed by the staff who performed the test (Table 1).

TABLE 1

Ease of use questionnaire responses and responses

Question	Strongly agree	Agree	Neutral	Disagree	Strongly disagree	No answer
Test procedures instructions were easy to follow	57 (85.1)	9 (13.4)	0	0	0	1 (1.5)
It was easy to process sample (addition of reagent to card)	60 (89.6)	7 (10.4)	0	0	0	0
It was easy to see and understand results	48 (71.6)	10 (14.9)	2 (3.0)	6 (8.6)	1 (1.5)	0
The control line was always easy to read	56 (83.6)	7 (10.4)	4 (6.0)	0	0	0
The instructions clearly explained how to tell if a test is invalid	61 (91.0)	5 (7.4)	1 (1.5)	0	0	0
I needed help from someone the first time I ran the test	17 (25.4)	19 (28.3)	10 (14.9)	9 (13.4)	12 (17.9)	0

Note: Data are provided as n (%).

RESULTS

From July 20 to October 28, 2020, 767 patients were enrolled; of these, 32 (4.2%) were considered unevaluable and were excluded from the analysis. Reasons for exclusions included the BinaxNOW test performance starting >60 minutes after sample collection (n = 8), reference testing conducted >72 hours after sample collection (n = 8), subject eligibility criteria were not met (n = 3), the BinaxNOW test operator was ineligible by training (eg, had prior formal laboratory training [n = 3] or was not trained in the performance of the BinaxNOW test [n = 1]), the reference sample was not sent to the reference laboratory (n = 4), the reference sample quantity was insufficient to conduct reference testing (n = 3), the BinaxNOW test result was invalid (n = 1), or the written informed consent form was invalid (n = 1). There were no adverse events related to the testing procedures.

Of the 735 evaluable patients, the mean (SD) age was 47 (16.6) years (Table 2), of which 57.4% were women and 49.1% were White, 41.4% were Black, 1.8% were Asian, and 11.3% were of Hispanic ethnicity. A total of 625 (84.8%) patients had COVID‐19 symptoms, most commonly shortness of breath (54.7%), cough (42.7%), and fever (34.4%). These were followed, in order of decreasing frequency, by fatigue (31.1%), headache (24.8%), myalgia (23.8%), diarrhea (18.4%), lack of taste/smell (13.6%), rhinorrhea (12.2%), and sore throat (11.6%). Rates of symptoms within a temporal cohort were relatively consistent with the passage of time (see Table 2). Although 460 (62.6%) had symptoms for ≤7 days, the mean (SD) time since symptom onset was 8.1 (14.0) days. The longest duration of symptoms reported was 182 days, and 56 (7.6%) patients had symptoms for ≥14 days.

TABLE 2

Demographics and testing results, stratified by symptom duration

	Symptoms, n = 623 (84.8%) a
	Overall, N = 735	<8 days, n = 460	8–14 days, n = 105	>14 days, n = 56	No symptoms, n = 112
Characteristic
Age, y, mean (SD)	47 (16.6)	48 (16.9)	51 (16.0)	51 (13.5)	36 (12.3)
Female sex, n (%)	422 (57.4)	253 (55.0)	58 (55.2%)	31 (55.4)	78 (69.6)
Race/ethnicity, n (%)
White	362 (49.1)	210 (45.7)	43 (41.0)	21 (37.5)	88 (78.6)
Black	304 (41.4)	217 (47.2)	50 (47.6)	19 (33.9)	16 (14.3)
Asian	13 (1.8)	4 (0.9)	1 (1.0)	1 (1.8)	7 (6.3)
Hispanic	83 (11.3)	40 (8.7)	15 (14.3)	22 (39.3)	6 (5.4)
Symptoms, n (%)
Any COVID‐19 symptom	623 (84.8)	460 (100.0)	105 (100.0)	56 (100.0)	0 (0)
Shortness of breath	402 (54.7)	286 (62.2)	80 (76.2)	36 (64.3)	0 (0)
Cough	314 (42.7)	221 (48.0)	60 (57.1)	33 (58.9)	0 (0)
Fever	253 (34.4)	193 (42.0)	42 (40.0)	18 (32.1)	0 (0)
Fatigue	229 (31.1)	159 (34.6)	42 (40.0)	28 (50.0)	0 (0)
Headache	182 (24.8)	132 (28.7)	26 (24.8)	24 (42.9)	0 (0)
Myalgias	175 (23.8)	126 (27.4)	32 (30.5)	17 (30.4)	0 (0)
Diarrhea	135 (18.4)	91 (19.8)	28 (26.7)	16 (28.6)	0 (0)
Lack of taste/smell	100 (13.6)	67 (14.6)	21 (20.0)	12 (21.4)	0 (0)
Rhinorrhea	90 (12.2)	68 (14.8)	15 (14.3)	7 (12.5)	0 (0)
Sore throat	85 (11.6)	64 (13.9)	12 (11.4)	9 (16.1)	0 (0)
Positive test
BinaxNOW	133 (18.1)	99 (21.5)	27 (25.7)	3 (5.4)	4 (3.6)
Gold standard	173 (23.5)	117 (25.4)	44 (41.9)	6 (10.7)	6 (5.4)
BinaxNOW versus gold standard, % (95% CI)
Sensitivity	76.9 (69.9–82.9)	84.6 (76.8–90.6)	61.4 (45.5–75.6)	50.0 (11.8–88.2)	66.7 (22.3–95.7)
Specificity	98.6 (97.2–99.4)	98.5 (96.6–99.5)	98.4 (91.2–100.0)	98.0 (89.4–99.9)	99.1 (94.9–100.0)
NPV	93.3 (90.9–95.1)	94.9 (92.1–97.0)	77.9 (67.0–86.6)	94.2 (84.1–98.8)	98.1 (93.4–99.8)
PPV	94.3 (89.1–97.5)	95.2 (89.1–98.4)	96.4 (81.7–99.9)	75.0 (19.4–99.4)	80.0 (28.4–99.5)

Abbreviations: CI, confidence interval; COVID‐19, coronavirus disease 2019; NPV negative predictive value; PPV, positive predictive value.

aTwo patients had symptom days missing.

Of the overall evaluable population, positive tests occurred in 173 (23.5%) and 133 (18.1%) patients with the gold standard and BinaxNOW, respectively. The majority of samples were collected in those >21 years of age, and the most samples were obtained in patients >60 years old. When stratified by symptom duration, those with symptoms <14 days had the highest rates of positive tests, 25.7% and 41.9%, for the BinaxNOW and the gold standard test, respectively. Earlier presentations, in those with symptoms <8 days, provided the greatest agreement between the BinaxNOW and the gold standard, 21.5% and 25.4%, respectively. Later presentations, in those with symptoms >2 weeks, had a positive test rate roughly one quarter of those with an earlier presentation regardless of testing platform. When symptoms were reported as >2 weeks, positive tests occurred in only 3 (5.4%) and 6 (10.7%) of the BinaxNOW and gold standard tests, respectively.

The overall sensitivity of the BinaxNOW test was 76.9%, but varied as a function of symptom duration. It was as high as 84.6% if symptoms were <8 days and declined thereafter (see Table 2). Specificity was consistently ≥98%, regardless of the time of presentation. Overall, positive and negative predictive values were >93% and were highest within the first week of symptoms.

Cycle time (Ct) describes the number of PCR cycles required to identify a virus, with fewer cycles equating to higher virus titers. When grouped by Ct value, independent of symptom status, the positive agreement of the gold standard and BinaxNOW tests for patients with Cts <33 was 90.6% (95% confidence interval [CI], 84.2%–95.1%). For patients with Cts ≥33, the positive agreement was 37.8% (95% CI, 23.8%–53.5%). When evaluated in patients who were symptomatic, the positive agreement was 90.3% (95% CI, 83.7%–94.9%) and 39.5% (95% CI, 25.0%–55.6%), between the gold standard and BinaxNOW tests, when Ct values were <33 and ≥33, respectively. When stratified between patients with symptom onsets of ≤5, 7, and 10 days, positive agreement improved, ranging from 94.5% to 96.7% for Ct values <33 and 27.8% to 48.1% when the Ct was ≥33. Among patients who were asymptomatic, the positive agreement was 100% (95% CI, 39.8%–100%) for Ct values <33 and 0.0% (95% CI, 0.0%–84.2%) for Ct values ≥33. The wide CIs in this later cohort are the result of very few reference patients who were positive and asymptomatic.

Finally, the ease of operations and interpretation of the results questionnaire (Table 1) was returned by 67 physicians, nurses, and research associates. The majority of answers regarding the ease of use, instructions, and interpretation of the BinaxNOW platform were all “strongly agree,” except for the question regarding needing help on first‐time use, for which only 25% “agreed.”

Limitations

Our study is not without limitations. Although the use of PCR as a gold standard provides good biochemical evidence of the presence of SARS‐CoV‐2, a clinically adjudicated diagnosis may have altered the outcomes of this investigation. Furthermore, the high negative and positive predictive values are a function of the disease prevalence when this study was performed, with different populations likely to have markedly different outcomes. Notably, although our study did not use laboratory experts, our results can only be applied to test performance by the clinical staff used in this study. Furthermore, although the BinaxNOW test requires user observation of indicator colors, collection of interrater reliability data was not obtained. Whether similar results could be obtained by laypeople using this assay in different environments is unclear. In addition, the population studied was a convenience sample of ED patients, and therefore a sample bias may have occurred. Finally, this study did not evaluate this assay's performance with newly arising variants; whether it will perform equally with different variants is unknown.

It should be noted that the enrollment rate per site may be considered low. This is because at the time of the pandemic that this study was performed, many EDs had simply furloughed their research staff to protect them from COVID‐19, thereby allowing only essential clinical staff into their departments. Thus, to gather these critical data required essentially volunteer research staff, who did so at significant personal risk to enroll at each institution, which did slow the process.

DISCUSSION

We found that nasal testing using the BinaxNOW point‐of‐care assay provides COVID‐19 results with high negative and positive predictive values compared with gold standard PCR testing. The clinical utility of the BinaxNOW is that it can be performed and resulted in 15 minutes by nonlaboratorians without significant equipment. The potential benefits of a rapid point‐of‐care test serially administered by a nonlaboratory technician and obtained by nasal swab include improved patient tolerability; broader, more efficient, and faster application; and improved identification of individuals infected with SARS‐CoV‐2 or those at risk by virtue of exposure to known infected individuals. The potential impact of this type of serial testing suggests that it may have application in environments where laboratory serial testing is not feasible.

Symptom duration, viral load, infectivity, and Ct values are closely related. Our results are highly correlated with “days of symptoms” and correspond to suspected viral load in patients with confirmed COVID‐19. Ct values >33 indicate low levels of virus that may be difficult to culture, suggesting the patient may no longer be infectious. , , , , , , , , , Achieving high positive agreement among patients with high Ct values may be of lesser clinical relevance compared with those with lower Ct values, as patients with high Ct values may pose a low risk of infecting others even if they were incorrectly classified as negative for the SARS‐CoV‐2 antigen and failed to self‐isolate.

We suggest that patients with low viral RNA levels, manifest by a Ct value of >33 (with the CDC RT‐PCR EUA test), are less likely to be infectious. Because it is impossible to know from 1 data point where a person is on the viral load curve (before or after the peak), frequent testing of suspected patients may be beneficial in identifying otherwise asymptomatic individuals. Although fluctuations in viral load may occur in a minority of cases, multiple rounds of testing have the potential to compensate for this variability and eventually identify an infected person. With these key findings in mind, frequent screening with an inexpensive, simple, rapid antigen test could identify silent contagious spreaders of SARS‐CoV‐2, thus aiding in the disruption of the COVID‐19 pandemic.

Although the BinaxNOW test has a lower sensitivity than the Abbott RealTime SARS‐CoV‐2 test, it has excellent specificity (98%) from day 1 to beyond 14 days of symptoms, allows rapid point‐of‐care testing, provides a result that can be obtained by a nonlaboratory technician, and has better patient tolerability than a nasopharyngeal test. These features may provide improved identification of individuals infected with SARS‐CoV‐2 and in those at risk by virtue of exposure to known infected individuals. Application of this strategy may represent another tool to help contain the SARS‐CoV‐2 pandemic.

CONFLICT OF INTEREST

Each author's institution received financial support for the performance of the study.

AUTHOR CONTRIBUTIONS

W. Frank Peacock wrote the first draft. Karina M. Soto‐Ruiz assembled study institutions, monitored the study, and critically reviewed the submission. Stacey L. House, Chad M. Cannon, Gary Headden, Brian Tiffany, Sergey Motov, Kian Merchant‐Borna, Anna Marie Chang, Claire Pearson, Brian W. Patterson, Alan E. Jones, Joseph Miller, Joseph Varon, Aveh Bastani, Carol Clark, Zubaid Rafique, Bory Kea, John Eppensteiner, James M. Williams, Simon A. Mahler, Brian E. Driver, Phyllis Hendry, Eugenia Quackenbush, David Robinson, Jon W. Schrock, James P. D'Etienne, Christopher J. Hogan, Anwar Osborne, and Ralph Riviello were responsible for patient recruitment at their respective institutions, and critically reviewed the submission. Stephen Young critically reviewed the submission and provided laboratory support.